1
|
Hallaj S, Chuter BG, Lieu AC, Singh P, Kalpathy-Cramer J, Xu BY, Christopher M, Zangwill LM, Weinreb RN, Baxter SL. Federated Learning in Glaucoma: A Comprehensive Review and Future Perspectives. Ophthalmol Glaucoma 2024:S2589-4196(24)00143-1. [PMID: 39214457 DOI: 10.1016/j.ogla.2024.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Revised: 08/20/2024] [Accepted: 08/23/2024] [Indexed: 09/04/2024]
Abstract
CLINICAL RELEVANCE Glaucoma is a complex eye condition with varied morphological and clinical presentations, making diagnosis and management challenging. The lack of a consensus definition for glaucoma or glaucomatous optic neuropathy further complicates the development of universal diagnostic tools. Developing robust artificial intelligence (AI) models for glaucoma screening is essential for early detection and treatment but faces significant obstacles. Effective deep learning algorithms require large, well-curated datasets from diverse patient populations and imaging protocols. However, creating centralized data repositories is hindered by concerns over data sharing, patient privacy, regulatory compliance, and intellectual property. Federated Learning (FL) offers a potential solution by enabling data to remain locally hosted while facilitating distributed model training across multiple sites. METHODS A comprehensive literature review was conducted on the application of Federated Learning in training AI models for glaucoma screening. Publications from 1950 to 2024 were searched using databases such as PubMed and IEEE Xplore with keywords including "glaucoma," "federated learning," "artificial intelligence," "deep learning," "machine learning," "distributed learning," "privacy-preserving," "data sharing," "medical imaging," and "ophthalmology." Articles were included if they discussed the use of FL in glaucoma-related AI tasks or addressed data sharing and privacy challenges in ophthalmic AI development. RESULTS FL enables collaborative model development without centralizing sensitive patient data, addressing privacy and regulatory concerns. Studies show that FL can improve model performance and generalizability by leveraging diverse datasets while maintaining data security. FL models have achieved comparable or superior accuracy to those trained on centralized data, demonstrating effectiveness in real-world clinical settings. CONCLUSIONS Federated Learning presents a promising strategy to overcome current obstacles in developing AI models for glaucoma screening. By balancing the need for extensive, diverse training data with the imperative to protect patient privacy and comply with regulations, FL facilitates collaborative model training without compromising data security. This approach offers a pathway toward more accurate and generalizable AI solutions for glaucoma detection and management. FINANCIAL DISCLOSURE(S) Proprietary or commercial disclosure may be found after the references in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Shahin Hallaj
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Benton G Chuter
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Alexander C Lieu
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Praveer Singh
- Division of Artificial Medical Intelligence, Department of Ophthalmology, University of Colorado School of Medicine, Aurora, Colorado
| | - Jayashree Kalpathy-Cramer
- Division of Artificial Medical Intelligence, Department of Ophthalmology, University of Colorado School of Medicine, Aurora, Colorado
| | - Benjamin Y Xu
- Roski Eye Institute, Keck School of Medicine, University of Southern California, Los Angeles, California
| | - Mark Christopher
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California
| | - Linda M Zangwill
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California
| | - Robert N Weinreb
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California
| | - Sally L Baxter
- Division of Ophthalmology Informatics and Data Science, Hamilton Glaucoma Center, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, La Jolla, California; Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California.
| |
Collapse
|
2
|
Christopher M, Hallaj S, Jiravarnsirikul A, Baxter SL, Zangwill LM. Novel Technologies in Artificial Intelligence and Telemedicine for Glaucoma Screening. J Glaucoma 2024; 33:S26-S32. [PMID: 38506792 DOI: 10.1097/ijg.0000000000002367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Accepted: 01/22/2024] [Indexed: 03/21/2024]
Abstract
PURPOSE To provide an overview of novel technologies in telemedicine and artificial intelligence (AI) approaches for cost-effective glaucoma screening. METHODS/RESULTS A narrative review was performed by summarizing research results, recent developments in glaucoma detection and care, and considerations related to telemedicine and AI in glaucoma screening. Telemedicine and AI approaches provide the opportunity for novel glaucoma screening programs in primary care, optometry, portable, and home-based settings. These approaches offer several advantages for glaucoma screening, including increasing access to care, lowering costs, identifying patients in need of urgent treatment, and enabling timely diagnosis and early intervention. However, challenges remain in implementing these systems, including integration into existing clinical workflows, ensuring equity for patients, and meeting ethical and regulatory requirements. Leveraging recent work towards standardized data acquisition as well as tools and techniques developed for automated diabetic retinopathy screening programs may provide a model for a cost-effective approach to glaucoma screening. CONCLUSION Leveraging novel technologies and advances in telemedicine and AI-based approaches to glaucoma detection show promise for improving our ability to detect moderate and advanced glaucoma in primary care settings and target higher individuals at high risk for having the disease.
Collapse
Affiliation(s)
- Mark Christopher
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
| | - Shahin Hallaj
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
| | - Anuwat Jiravarnsirikul
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Department of Medicine, Division of Biomedical Informatics, University of California San Diego, La Jolla, CA
| | - Sally L Baxter
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
- Department of Medicine, Division of Biomedical Informatics, University of California San Diego, La Jolla, CA
| | - Linda M Zangwill
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
| |
Collapse
|
3
|
Chen JS, Copado IA, Vallejos C, Kalaw FGP, Soe P, Cai CX, Toy BC, Borkar D, Sun CQ, Shantha JG, Baxter SL. Variations in Electronic Health Record-Based Definitions of Diabetic Retinopathy Cohorts: A Literature Review and Quantitative Analysis. OPHTHALMOLOGY SCIENCE 2024; 4:100468. [PMID: 38560278 PMCID: PMC10973665 DOI: 10.1016/j.xops.2024.100468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 01/04/2024] [Accepted: 01/11/2024] [Indexed: 04/04/2024]
Abstract
Purpose Use of the electronic health record (EHR) has motivated the need for data standardization. A gap in knowledge exists regarding variations in existing terminologies for defining diabetic retinopathy (DR) cohorts. This study aimed to review the literature and analyze variations regarding codified definitions of DR. Design Literature review and quantitative analysis. Subjects Published manuscripts. Methods Four graders reviewed PubMed and Google Scholar for peer-reviewed studies. Studies were included if they used codified definitions of DR (e.g., billing codes). Data elements such as author names, publication year, purpose, data set type, and DR definitions were manually extracted. Each study was reviewed by ≥ 2 authors to validate inclusion eligibility. Quantitative analyses of the codified definitions were then performed to characterize the variation between DR cohort definitions. Main Outcome Measures Number of studies included and numeric counts of billing codes used to define codified cohorts. Results In total, 43 studies met the inclusion criteria. Half of the included studies used datasets based on structured EHR data (i.e., data registries, institutional EHR review), and half used claims data. All but 1 of the studies used billing codes such as the International Classification of Diseases 9th or 10th edition (ICD-9 or ICD-10), either alone or in addition to another terminology for defining disease. Of the 27 included studies that used ICD-9 and the 20 studies that used ICD-10 codes, the most common codes used pertained to the full spectrum of DR severity. Diabetic retinopathy complications (e.g., vitreous hemorrhage) were also used to define some DR cohorts. Conclusions Substantial variations exist among codified definitions for DR cohorts within retrospective studies. Variable definitions may limit generalizability and reproducibility of retrospective studies. More work is needed to standardize disease cohorts. Financial Disclosures Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Jimmy S. Chen
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California
| | - Ivan A. Copado
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California
| | - Cecilia Vallejos
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California
| | - Fritz Gerald P. Kalaw
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California
| | - Priyanka Soe
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California
| | - Cindy X. Cai
- Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, Maryland
| | - Brian C. Toy
- Department of Ophthalmology, Roski Eye Institute, Keck School of Medicine, University of Southern California, Los Angeles, California
| | - Durga Borkar
- Department of Ophthalmology, Duke Eye Center, Duke University, Durham, North Carolina
| | - Catherine Q. Sun
- F.I. Proctor Foundation, University of California San Francisco, San Francisco, California
- Department of Ophthalmology, University of California San Francisco, San Francisco, California
| | - Jessica G. Shantha
- F.I. Proctor Foundation, University of California San Francisco, San Francisco, California
- Department of Ophthalmology, University of California San Francisco, San Francisco, California
| | - Sally L. Baxter
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California
| |
Collapse
|
4
|
Song A, Borkar DS. Advances in Teleophthalmology Screening for Diabetic Retinopathy. Int Ophthalmol Clin 2024; 64:97-113. [PMID: 38146884 DOI: 10.1097/iio.0000000000000505] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2023]
|
5
|
Cai CX, Halfpenny W, Boland MV, Lehmann HP, Hribar M, Goetz KE, Baxter SL. Advancing Toward a Common Data Model in Ophthalmology: Gap Analysis of General Eye Examination Concepts to Standard Observational Medical Outcomes Partnership (OMOP) Concepts. OPHTHALMOLOGY SCIENCE 2023; 3:100391. [PMID: 38025162 PMCID: PMC10630664 DOI: 10.1016/j.xops.2023.100391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 08/16/2023] [Accepted: 08/21/2023] [Indexed: 12/01/2023]
Abstract
Purpose Evaluate the degree of concept coverage of the general eye examination in one widely used electronic health record (EHR) system using the Observational Health Data Sciences and Informatics Observational Medical Outcomes Partnership (OMOP) common data model (CDM). Design Study of data elements. Participants Not applicable. Methods Data elements (field names and predefined entry values) from the general eye examination in the Epic foundation system were mapped to OMOP concepts and analyzed. Each mapping was given a Health Level 7 equivalence designation-equal when the OMOP concept had the same meaning as the source EHR concept, wider when it was missing information, narrower when it was overly specific, and unmatched when there was no match. Initial mappings were reviewed by 2 graders. Intergrader agreement for equivalence designation was calculated using Cohen's kappa. Agreement on the mapped OMOP concept was calculated as a percentage of total mappable concepts. Discrepancies were discussed and a final consensus created. Quantitative analysis was performed on wider and unmatched concepts. Main Outcome Measures Gaps in OMOP concept coverage of EHR elements and intergrader agreement of mapped OMOP concepts. Results A total of 698 data elements (210 fields, 488 values) from the EHR were analyzed. The intergrader kappa on the equivalence designation was 0.88 (standard error 0.03, P < 0.001). There was a 96% agreement on the mapped OMOP concept. In the final consensus mapping, 25% (1% fields, 31% values) of the EHR to OMOP concept mappings were considered equal, 50% (27% fields, 60% values) wider, 4% (8% fields, 2% values) narrower, and 21% (52% fields, 8% values) unmatched. Of the wider mapped elements, 46% were missing the laterality specification, 24% had other missing attributes, and 30% had both issues. Wider and unmatched EHR elements could be found in all areas of the general eye examination. Conclusions Most data elements in the general eye examination could not be represented precisely using the OMOP CDM. Our work suggests multiple ways to improve the incorporation of important ophthalmology concepts in OMOP, including adding laterality to existing concepts. There exists a strong need to improve the coverage of ophthalmic concepts in source vocabularies so that the OMOP CDM can better accommodate vision research. Financial Disclosures Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Cindy X. Cai
- Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, Maryland
| | - William Halfpenny
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Michael V. Boland
- Massachusetts Eye and Ear, Harvard Medical School, Boston, Massachusetts
| | - Harold P. Lehmann
- Division of Health Sciences Informatics, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
- Biomedical Informatics and Data Science, Division of General Internal Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Michelle Hribar
- Office of Data Science and Health Informatics, National Eye Institute, National Institute of Health, Bethesda, Maryland
- Department of Ophthalmology, Casey Eye Institute, Portland, Oregon
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon
| | - Kerry E. Goetz
- Office of Data Science and Health Informatics, National Eye Institute, National Institute of Health, Bethesda, Maryland
| | - Sally L. Baxter
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
| |
Collapse
|
6
|
Tavakoli K, Kalaw FGP, Bhanvadia S, Hogarth M, Baxter SL. Concept Coverage Analysis of Ophthalmic Infections and Trauma among the Standardized Medical Terminologies SNOMED-CT, ICD-10-CM, and ICD-11. OPHTHALMOLOGY SCIENCE 2023; 3:100337. [PMID: 37449050 PMCID: PMC10336190 DOI: 10.1016/j.xops.2023.100337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 05/10/2023] [Accepted: 05/19/2023] [Indexed: 07/18/2023]
Abstract
Purpose Widespread electronic health record adoption has generated a large volume of data and emphasized the need for standardized terminology to describe clinical concepts. Here, we undertook a systematic concept coverage analysis to determine the representation of clinical concepts in ophthalmic infection and ophthalmic trauma among standardized medical terminologies, including the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT), the International Classification of Diseases (ICD) version 10 with clinical modifications (ICD-10-CM), and ICD version 11 (ICD-11). Design Extraction of concepts related to ophthalmic infection and ophthalmic trauma and structured search in terminology browsers. Data Sources The American Academy of Ophthalmology Basic and Clinical Science Course (BCSC), SNOMED-CT, and ICD-10-CM terminologies from the Observational Health Data Sciences and Informatics Athena browser, and the ICD-11 terminology browser. Methods Concepts pertaining to ophthalmic infection and ophthalmic trauma were extracted from the 2022 BCSC free text and index terms. We searched terminology browsers to identify corresponding codes and classified the extent of semantic alignment as equal, wide, narrow, or unmatched in each terminology. The overlap of equal concepts in each terminology was represented in a Venn diagram. Main Outcome Measures Proportions of clinical concepts with corresponding codes at various levels of semantic alignment. Results A total of 443 concepts were identified: 304 concepts related to ophthalmic infection and 139 concepts related to ophthalmic trauma. The SNOMED-CT had the highest proportion of equal coverage, with 82.0% (249 of 304) among concepts related to ophthalmic infection and 82.0% (115 of 139) among concepts related to ophthalmic trauma. Across all concepts, 28% (124 of 443) were classified as equal in ICD-10-CM and 52.8% (234 of 443) were classified as equal in ICD-11. Conclusions The SNOMED-CT had significantly better semantic alignment than ICD-10-CM and ICD-11 for ophthalmic infections and ophthalmic trauma. This demonstrates opportunity for continuing advancement of representation of ophthalmic concepts in standardized medical terminologies.
Collapse
Affiliation(s)
- Kiana Tavakoli
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Fritz Gerald P. Kalaw
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Sonali Bhanvadia
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Michael Hogarth
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| | - Sally L. Baxter
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, California
| |
Collapse
|
7
|
Gu B, Sidhu S, Weinreb RN, Christopher M, Zangwill LM, Baxter SL. Review of Visualization Approaches in Deep Learning Models of Glaucoma. Asia Pac J Ophthalmol (Phila) 2023; 12:392-401. [PMID: 37523431 DOI: 10.1097/apo.0000000000000619] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 05/11/2023] [Indexed: 08/02/2023] Open
Abstract
Glaucoma is a major cause of irreversible blindness worldwide. As glaucoma often presents without symptoms, early detection and intervention are important in delaying progression. Deep learning (DL) has emerged as a rapidly advancing tool to help achieve these objectives. In this narrative review, data types and visualization approaches for presenting model predictions, including models based on tabular data, functional data, and/or structural data, are summarized, and the importance of data source diversity for improving the utility and generalizability of DL models is explored. Examples of innovative approaches to understanding predictions of artificial intelligence (AI) models and alignment with clinicians are provided. In addition, methods to enhance the interpretability of clinical features from tabular data used to train AI models are investigated. Examples of published DL models that include interfaces to facilitate end-user engagement and minimize cognitive and time burdens are highlighted. The stages of integrating AI models into existing clinical workflows are reviewed, and challenges are discussed. Reviewing these approaches may help inform the generation of user-friendly interfaces that are successfully integrated into clinical information systems. This review details key principles regarding visualization approaches in DL models of glaucoma. The articles reviewed here focused on usability, explainability, and promotion of clinician trust to encourage wider adoption for clinical use. These studies demonstrate important progress in addressing visualization and explainability issues required for successful real-world implementation of DL models in glaucoma.
Collapse
Affiliation(s)
- Byoungyoung Gu
- Division of Ophthalmology Informatics and Data Science and Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, US
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA, US
| | - Sophia Sidhu
- Division of Ophthalmology Informatics and Data Science and Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, US
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA, US
| | - Robert N Weinreb
- Division of Ophthalmology Informatics and Data Science and Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, US
| | - Mark Christopher
- Division of Ophthalmology Informatics and Data Science and Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, US
| | - Linda M Zangwill
- Division of Ophthalmology Informatics and Data Science and Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, US
| | - Sally L Baxter
- Division of Ophthalmology Informatics and Data Science and Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, US
- Division of Biomedical Informatics, Department of Medicine, University of California San Diego, La Jolla, CA, US
| |
Collapse
|
8
|
Constantin A, Atkinson M, Bernabeu MO, Buckmaster F, Dhillon B, McTrusty A, Strang N, Williams R. Optometrists' Perspectives Regarding Artificial Intelligence Aids and Contributing Retinal Images to a Repository: Web-Based Interview Study. JMIR Hum Factors 2023; 10:e40887. [PMID: 37227761 DOI: 10.2196/40887] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Revised: 01/23/2023] [Accepted: 04/19/2023] [Indexed: 05/26/2023] Open
Abstract
BACKGROUND A repository of retinal images for research is being established in Scotland. It will permit researchers to validate, tune, and refine artificial intelligence (AI) decision-support algorithms to accelerate safe deployment in Scottish optometry and beyond. Research demonstrates the potential of AI systems in optometry and ophthalmology, though they are not yet widely adopted. OBJECTIVE In this study, 18 optometrists were interviewed to (1) identify their expectations and concerns about the national image research repository and their use of AI decision support and (2) gather their suggestions for improving eye health care. The goal was to clarify attitudes among optometrists delivering primary eye care with respect to contributing their patients' images and to using AI assistance. These attitudes are less well studied in primary care contexts. Five ophthalmologists were interviewed to discover their interactions with optometrists. METHODS Between March and August 2021, 23 semistructured interviews were conducted online lasting for 30-60 minutes. Transcribed and pseudonymized recordings were analyzed using thematic analysis. RESULTS All optometrists supported contributing retinal images to form an extensive and long-running research repository. Our main findings are summarized as follows. Optometrists were willing to share images of their patients' eyes but expressed concern about technical difficulties, lack of standardization, and the effort involved. Those interviewed thought that sharing digital images would improve collaboration between optometrists and ophthalmologists, for example, during referral to secondary health care. Optometrists welcomed an expanded primary care role in diagnosis and management of diseases by exploiting new technologies and anticipated significant health benefits. Optometrists welcomed AI assistance but insisted that it should not reduce their role and responsibilities. CONCLUSIONS Our investigation focusing on optometrists is novel because most similar studies on AI assistance were performed in hospital settings. Our findings are consistent with those of studies with professionals in ophthalmology and other medical disciplines: showing near universal willingness to use AI to improve health care, alongside concerns over training, costs, responsibilities, skill retention, data sharing, and disruptions to professional practices. Our study on optometrists' willingness to contribute images to a research repository introduces a new aspect; they hope that a digital image sharing infrastructure will facilitate service integration.
Collapse
Affiliation(s)
- Aurora Constantin
- School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
| | - Malcolm Atkinson
- School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
| | - Miguel Oscar Bernabeu
- Centre for Medical Informatics, Usher Institute, University of Edinburgh, Edinburgh, United Kingdom
| | - Fiona Buckmaster
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Baljean Dhillon
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Alice McTrusty
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Niall Strang
- Department of Vision Sciences, Glasgow Caledonian University, Glasgow, United Kingdom
| | - Robin Williams
- School of Social and Political Science, University of Edinburgh, Edinburgh, United Kingdom
| |
Collapse
|
9
|
Baxter SL. Ocular Health and National Data Standards: A Case for Including Visual Acuity in the United States Core Data for Interoperability (USCDI). OPHTHALMOLOGY SCIENCE 2022; 2:100210. [PMID: 36570621 PMCID: PMC9767820 DOI: 10.1016/j.xops.2022.100210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
|
10
|
Domalpally A, Slater R, Barrett N, Voland R, Balaji R, Heathcote J, Channa R, Blodi B. Implementation of a Large-Scale Image Curation Workflow Using Deep Learning Framework. OPHTHALMOLOGY SCIENCE 2022; 2:100198. [PMID: 36531570 PMCID: PMC9754974 DOI: 10.1016/j.xops.2022.100198] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/20/2022] [Revised: 06/09/2022] [Accepted: 07/07/2022] [Indexed: 06/17/2023]
Abstract
PURPOSE The curation of images using human resources is time intensive but an essential step for developing artificial intelligence (AI) algorithms. Our goal was to develop and implement an AI algorithm for image curation in a high-volume setting. We also explored AI tools that will assist in deploying a tiered approach, in which the AI model labels images and flags potential mislabels for human review. DESIGN Implementation of an AI algorithm. PARTICIPANTS Seven-field stereoscopic images from multiple clinical trials. METHODS The 7-field stereoscopic image protocol includes 7 pairs of images from various parts of the central retina along with images of the anterior part of the eye. All images were labeled for field number by reading center graders. The model output included classification of the retinal images into 8 field numbers. Probability scores (0-1) were generated to identify misclassified images, with 1 indicating a high probability of a correct label. MAIN OUTCOME MEASURES Agreement of AI prediction with grader classification of field number and the use of probability scores to identify mislabeled images. RESULTS The AI model was trained and validated on 17 529 images and tested on 3004 images. The pooled agreement of field numbers between grader classification and the AI model was 88.3% (kappa, 0.87). The pooled mean probability score was 0.97 (standard deviation [SD], 0.08) for images for which the graders agreed with the AI-generated labels and 0.77 (SD, 0.19) for images for which the graders disagreed with the AI-generated labels (P < 0.0001). Using receiver operating characteristic curves, a probability score of 0.99 was identified as a cutoff for distinguishing mislabeled images. A tiered workflow using a probability score of < 0.99 as a cutoff would include 27.6% of the 3004 images for human review and reduce the error rate from 11.7% to 1.5%. CONCLUSIONS The implementation of AI algorithms requires measures in addition to model validation. Tools to flag potential errors in the labels generated by AI models will reduce inaccuracies, increase trust in the system, and provide data for continuous model development.
Collapse
Affiliation(s)
- Amitha Domalpally
- A-EYE Unit, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
- Wisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Robert Slater
- A-EYE Unit, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Nancy Barrett
- Wisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Rick Voland
- Wisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Rohit Balaji
- Wisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Jennifer Heathcote
- Wisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Roomasa Channa
- A-EYE Unit, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| | - Barbara Blodi
- Wisconsin Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, Wisconsin
| |
Collapse
|
11
|
Halfpenny W, Baxter SL. Towards effective data sharing in ophthalmology: data standardization and data privacy. Curr Opin Ophthalmol 2022; 33:418-424. [PMID: 35819893 PMCID: PMC9357189 DOI: 10.1097/icu.0000000000000878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE OF REVIEW The purpose of this review is to provide an overview of updates in data standardization and data privacy in ophthalmology. These topics represent two key aspects of medical information sharing and are important knowledge areas given trends in data-driven healthcare. RECENT FINDINGS Standardization and privacy can be seen as complementary aspects that pertain to data sharing. Standardization promotes the ease and efficacy through which data is shared. Privacy considerations ensure that data sharing is appropriate and sufficiently controlled. There is active development in both areas, including government regulations and common data models to advance standardization, and application of technologies such as blockchain and synthetic data to help tackle privacy issues. These advancements have seen use in ophthalmology, but there are areas where further work is required. SUMMARY Information sharing is fundamental to both research and care delivery, and standardization/privacy are key constituent considerations. Therefore, widespread engagement with, and development of, data standardization and privacy ecosystems stand to offer great benefit to ophthalmology.
Collapse
Affiliation(s)
| | - Sally L. Baxter
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, CA, USA
- Health Department of Biomedical Informatics, University of California San Diego, La Jolla, CA, USA
| |
Collapse
|
12
|
Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice. Prog Retin Eye Res 2021; 90:101034. [PMID: 34902546 DOI: 10.1016/j.preteyeres.2021.101034] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Revised: 12/03/2021] [Accepted: 12/06/2021] [Indexed: 01/14/2023]
Abstract
An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.
Collapse
|