1
|
Elvidge J, Hawksworth C, Avşar TS, Zemplenyi A, Chalkidou A, Petrou S, Petykó Z, Srivastava D, Chandra G, Delaye J, Denniston A, Gomes M, Knies S, Nousios P, Siirtola P, Wang J, Dawoud D. Consolidated Health Economic Evaluation Reporting Standards for Interventions That Use Artificial Intelligence (CHEERS-AI). VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:1196-1205. [PMID: 38795956 PMCID: PMC11343728 DOI: 10.1016/j.jval.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Revised: 04/22/2024] [Accepted: 05/04/2024] [Indexed: 05/28/2024]
Abstract
OBJECTIVES Economic evaluations (EEs) are commonly used by decision makers to understand the value of health interventions. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022) provide reporting guidelines for EEs. Healthcare systems will increasingly see new interventions that use artificial intelligence (AI) to perform their function. We developed Consolidated Health Economic Evaluation Reporting Standards for Interventions that use AI (CHEERS-AI) to ensure EEs of AI-based health interventions are reported in a transparent and reproducible manner. METHODS Potential CHEERS-AI reporting items were informed by 2 published systematic literature reviews of EEs and a contemporary update. A Delphi study was conducted using 3 survey rounds to elicit multidisciplinary expert views on 26 potential items, through a 9-point Likert rating scale and qualitative comments. An online consensus meeting was held to finalize outstanding reporting items. A digital health patient group reviewed the final checklist from a patient perspective. RESULTS A total of 58 participants responded to survey round 1, 42, and 31 of whom responded to rounds 2 and 3, respectively. Nine participants joined the consensus meeting. Ultimately, 38 reporting items were included in CHEERS-AI. They comprised the 28 original CHEERS 2022 items, plus 10 new AI-specific reporting items. Additionally, 8 of the original CHEERS 2022 items were elaborated on to ensure AI-specific nuance is reported. CONCLUSIONS CHEERS-AI should be used when reporting an EE of an intervention that uses AI to perform its function. CHEERS-AI will help decision makers and reviewers to understand important AI-specific details of an intervention, and any implications for the EE methods used and cost-effectiveness conclusions.
Collapse
Affiliation(s)
- Jamie Elvidge
- National Institute for Health and Care Excellence (NICE), England, UK.
| | - Claire Hawksworth
- National Institute for Health and Care Excellence (NICE), England, UK
| | - Tuba Saygın Avşar
- National Institute for Health and Care Excellence (NICE), England, UK
| | - Antal Zemplenyi
- Center for Health Technology Assessment and Pharmacoeconomic Research, University of Pécs, Pécs, Hungary; University of Colorado Anschutz Medical Campus, Denver, CO, USA; Syreon Research Institute, Budapest, Hungary
| | | | - Stavros Petrou
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, England, UK
| | | | - Divya Srivastava
- Department of Health Policy, London School of Economics and Political Science, London, England, UK
| | - Gunjan Chandra
- Biomimetics and Intelligent Systems Group, University of Oulu, Oulu, Finland
| | | | - Alastair Denniston
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, England, UK
| | - Manuel Gomes
- Department of Primary Care and Population Health, University College London, England, UK
| | - Saskia Knies
- National Healthcare Institute (ZIN), Diemen, The Netherlands
| | - Petros Nousios
- Dental and Pharmaceutical Benefits Agency (TLV), Stockholm, Sweden
| | - Pekka Siirtola
- Biomimetics and Intelligent Systems Group, University of Oulu, Oulu, Finland
| | - Junfeng Wang
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University, Utrecht, The Netherlands
| | - Dalia Dawoud
- National Institute for Health and Care Excellence (NICE), England, UK
| |
Collapse
|
2
|
Wong CYT, Antaki F, Woodward-Court P, Ong AY, Keane PA. The role of saliency maps in enhancing ophthalmologists' trust in artificial intelligence models. Asia Pac J Ophthalmol (Phila) 2024:100087. [PMID: 39069106 DOI: 10.1016/j.apjo.2024.100087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2024] [Revised: 07/17/2024] [Accepted: 07/23/2024] [Indexed: 07/30/2024] Open
Abstract
PURPOSE Saliency maps (SM) allow clinicians to better understand the opaque decision-making process in artificial intelligence (AI) models by visualising the important features responsible for predictions. This ultimately improves interpretability and confidence. In this work, we review the use case for SMs, exploring their impact on clinicians' understanding and trust in AI models. We use the following ophthalmic conditions as examples: (1) glaucoma, (2) myopia, (3) age-related macular degeneration, and (4) diabetic retinopathy. METHOD A multi-field search on MEDLINE, Embase, and Web of Science was conducted using specific keywords. Only studies on the use of SMs in glaucoma, myopia, AMD, or DR were considered for inclusion. RESULTS Findings reveal that SMs are often used to validate AI models and advocate for their adoption, potentially leading to biased claims. Overlooking the technical limitations of SMs, and the conductance of superficial assessments of their quality and relevance, was discerned. Uncertainties persist regarding the role of saliency maps in building trust in AI. It is crucial to enhance understanding of SMs' technical constraints and improve evaluation of their quality, impact, and suitability for specific tasks. Establishing a standardised framework for selecting and assessing SMs, as well as exploring their relationship with other reliability sources (e.g. safety and generalisability), is essential for enhancing clinicians' trust in AI. CONCLUSION We conclude that SMs are not beneficial for interpretability and trust-building purposes in their current forms. Instead, SMs may confer benefits to model debugging, model performance enhancement, and hypothesis testing (e.g. novel biomarkers).
Collapse
Affiliation(s)
| | - Fares Antaki
- Institute of Ophthalmology, University College London, London, United Kingdom
| | | | - Ariel Yuhan Ong
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Pearse A Keane
- Institute of Ophthalmology, University College London, London, United Kingdom.
| |
Collapse
|
3
|
Wu H, Jin K, Yip CC, Koh V, Ye J. A systematic review of economic evaluation of artificial intelligence-based screening for eye diseases: From possibility to reality. Surv Ophthalmol 2024; 69:499-507. [PMID: 38492584 DOI: 10.1016/j.survophthal.2024.03.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 03/04/2024] [Accepted: 03/12/2024] [Indexed: 03/18/2024]
Abstract
Artificial Intelligence (AI) has become a focus of research in the rapidly evolving field of ophthalmology. Nevertheless, there is a lack of systematic studies on the health economics of AI in this field. We examine studies from the PubMed, Google Scholar, and Web of Science databases that employed quantitative analysis, retrieved up to July 2023. Most of the studies indicate that AI leads to cost savings and improved efficiency in ophthalmology. On the other hand, some studies suggest that using AI in healthcare may raise costs for patients, especially when taking into account factors such as labor costs, infrastructure, and patient adherence. Future research should cover a wider range of ophthalmic diseases beyond common eye conditions. Moreover, conducting extensive health economic research, designed to collect data relevant to its own context, is imperative.
Collapse
Affiliation(s)
- Hongkang Wu
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Kai Jin
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Chee Chew Yip
- Department of Ophthalmology & Visual Sciences, Khoo Teck Puat Hospital, Singapore, Singapore
| | - Victor Koh
- Department of Ophthalmology, National University Hospital, National University of Singapore, Singapore
| | - Juan Ye
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China.
| |
Collapse
|
4
|
Zhang Z, Deng C, Paulus YM. Advances in Structural and Functional Retinal Imaging and Biomarkers for Early Detection of Diabetic Retinopathy. Biomedicines 2024; 12:1405. [PMID: 39061979 PMCID: PMC11274328 DOI: 10.3390/biomedicines12071405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 05/27/2024] [Accepted: 06/10/2024] [Indexed: 07/28/2024] Open
Abstract
Diabetic retinopathy (DR), a vision-threatening microvascular complication of diabetes mellitus (DM), is a leading cause of blindness worldwide that requires early detection and intervention. However, diagnosing DR early remains challenging due to the subtle nature of initial pathological changes. This review explores developments in multimodal imaging and functional tests for early DR detection. Where conventional color fundus photography is limited in the field of view and resolution, advanced quantitative analysis of retinal vessel traits such as retinal microvascular caliber, tortuosity, and fractal dimension (FD) can provide additional prognostic value. Optical coherence tomography (OCT) has also emerged as a reliable structural imaging tool for assessing retinal and choroidal neurodegenerative changes, which show potential as early DR biomarkers. Optical coherence tomography angiography (OCTA) enables the evaluation of vascular perfusion and the contours of the foveal avascular zone (FAZ), providing valuable insights into early retinal and choroidal vascular changes. Functional tests, including multifocal electroretinography (mfERG), visual evoked potential (VEP), multifocal pupillographic objective perimetry (mfPOP), microperimetry, and contrast sensitivity (CS), offer complementary data on early functional deficits in DR. More importantly, combining structural and functional imaging data may facilitate earlier detection of DR and targeted management strategies based on disease progression. Artificial intelligence (AI) techniques show promise for automated lesion detection, risk stratification, and biomarker discovery from various imaging data. Additionally, hematological parameters, such as neutrophil-lymphocyte ratio (NLR) and neutrophil extracellular traps (NETs), may be useful in predicting DR risk and progression. Although current methods can detect early DR, there is still a need for further research and development of reliable, cost-effective methods for large-scale screening and monitoring of individuals with DM.
Collapse
Affiliation(s)
- Zhengwei Zhang
- Department of Ophthalmology, Jiangnan University Medical Center, Wuxi 214002, China;
- Department of Ophthalmology, Wuxi No.2 People’s Hospital, Wuxi Clinical College, Nantong University, Wuxi 214002, China
- Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI 48105, USA;
| | - Callie Deng
- Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI 48105, USA;
| | - Yannis M. Paulus
- Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI 48105, USA;
- Department of Biomedical Engineering, University of Michigan, 1000 Wall Street, Ann Arbor, MI 48105, USA
| |
Collapse
|
5
|
Fleming AD, Mellor J, McGurnaghan SJ, Blackbourn LAK, Goatman KA, Styles C, Storkey AJ, McKeigue PM, Colhoun HM. Deep learning detection of diabetic retinopathy in Scotland's diabetic eye screening programme. Br J Ophthalmol 2024; 108:984-988. [PMID: 37704266 DOI: 10.1136/bjo-2023-323395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 08/17/2023] [Indexed: 09/15/2023]
Abstract
BACKGROUND/AIMS Support vector machine-based automated grading (known as iGradingM) has been shown to be safe, cost-effective and robust in the diabetic retinopathy (DR) screening (DES) programme in Scotland. It triages screening episodes as gradable with no DR versus manual grading required. The study aim was to develop a deep learning-based autograder using images and gradings from DES and to compare its performance with that of iGradingM. METHODS Retinal images, quality assurance (QA) data and routine DR grades were obtained from national datasets in 179 944 patients for years 2006-2016. QA grades were available for 744 images. We developed a deep learning-based algorithm to detect whether either eye contained ungradable images or any DR. The sensitivity and specificity were evaluated against consensus QA grades and routine grades. RESULTS Images used in QA which were ungradable or with DR were detected by deep learning with better specificity compared with manual graders (p<0.001) and with iGradingM (p<0.001) at the same sensitivities. Any DR according to the DES final grade was detected with 89.19% (270 392/303 154) sensitivity and 77.41% (500 945/647 158) specificity. Observable disease and referable disease were detected with sensitivities of 96.58% (16 613/17 201) and 98.48% (22 600/22 948), respectively. Overall, 43.84% of screening episodes would require manual grading. CONCLUSION A deep learning-based system for DR grading was evaluated in QA data and images from 11 years in 50% of people attending a national DR screening programme. The system could reduce the manual grading workload at the same sensitivity compared with the current automated grading system.
Collapse
Affiliation(s)
- Alan D Fleming
- The Institute of Genetics and Cancer, University of Edinburgh Western General Hospital, Edinburgh, UK
| | - Joseph Mellor
- Usher Institute, The University of Edinburgh, Edinburgh, UK
| | - Stuart J McGurnaghan
- The Institute of Genetics and Cancer, University of Edinburgh Western General Hospital, Edinburgh, UK
| | - Luke A K Blackbourn
- The Institute of Genetics and Cancer, University of Edinburgh Western General Hospital, Edinburgh, UK
| | | | | | - Amos J Storkey
- School of Informatics, The University of Edinburgh, Edinburgh, UK
| | | | - Helen M Colhoun
- The Institute of Genetics and Cancer, University of Edinburgh Western General Hospital, Edinburgh, UK
| |
Collapse
|
6
|
Winder AJ, Stanley EA, Fiehler J, Forkert ND. Challenges and Potential of Artificial Intelligence in Neuroradiology. Clin Neuroradiol 2024; 34:293-305. [PMID: 38285239 DOI: 10.1007/s00062-024-01382-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Accepted: 01/03/2024] [Indexed: 01/30/2024]
Abstract
PURPOSE Artificial intelligence (AI) has emerged as a transformative force in medical research and is garnering increased attention in the public consciousness. This represents a critical time period in which medical researchers, healthcare providers, insurers, regulatory agencies, and patients are all developing and shaping their beliefs and policies regarding the use of AI in the healthcare sector. The successful deployment of AI will require support from all these groups. This commentary proposes that widespread support for medical AI must be driven by clear and transparent scientific reporting, beginning at the earliest stages of scientific research. METHODS A review of relevant guidelines and literature describing how scientific reporting plays a central role at key stages in the life cycle of an AI software product was conducted. To contextualize this principle within a specific medical domain, we discuss the current state of predictive tissue outcome modeling in acute ischemic stroke and the unique challenges presented therein. RESULTS AND CONCLUSION Translating AI methods from the research to the clinical domain is complicated by challenges related to model design and validation studies, medical product regulations, and healthcare providers' reservations regarding AI's efficacy and affordability. However, each of these limitations is also an opportunity for high-impact research that will help to accelerate the clinical adoption of state-of-the-art medical AI. In all cases, establishing and adhering to appropriate reporting standards is an important responsibility that is shared by all of the parties involved in the life cycle of a prospective AI software product.
Collapse
Affiliation(s)
- Anthony J Winder
- Department of Radiology, University of Calgary, Calgary, Canada.
- Hotchkiss Brain Institute, University of Calgary, Calgary, Canada.
| | - Emma Am Stanley
- Department of Radiology, University of Calgary, Calgary, Canada
- Hotchkiss Brain Institute, University of Calgary, Calgary, Canada
- Biomedical Engineering Graduate Program, University of Calgary, Calgary, Canada
- Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Canada
| | - Jens Fiehler
- Department of Diagnostic and Interventional Neuroradiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Nils D Forkert
- Department of Radiology, University of Calgary, Calgary, Canada
- Hotchkiss Brain Institute, University of Calgary, Calgary, Canada
- Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Canada
- Department of Clinical Neuroscience, University of Calgary, Calgary, Canada
- Department of Electrical and Software Engineering, University of Calgary, Calgary, Canada
| |
Collapse
|
7
|
Wu X, Wu Y, Tu Z, Cao Z, Xu M, Xiang Y, Lin D, Jin L, Zhao L, Zhang Y, Liu Y, Yan P, Hu W, Liu J, Liu L, Wang X, Wang R, Chen J, Xiao W, Shang Y, Xie P, Wang D, Zhang X, Dongye M, Wang C, Ting DSW, Liu Y, Pan R, Lin H. Cost-effectiveness and cost-utility of a digital technology-driven hierarchical healthcare screening pattern in China. Nat Commun 2024; 15:3650. [PMID: 38688925 PMCID: PMC11061155 DOI: 10.1038/s41467-024-47211-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 03/25/2024] [Indexed: 05/02/2024] Open
Abstract
Utilization of digital technologies for cataract screening in primary care is a potential solution for addressing the dilemma between the growing aging population and unequally distributed resources. Here, we propose a digital technology-driven hierarchical screening (DH screening) pattern implemented in China to promote the equity and accessibility of healthcare. It consists of home-based mobile artificial intelligence (AI) screening, community-based AI diagnosis, and referral to hospitals. We utilize decision-analytic Markov models to evaluate the cost-effectiveness and cost-utility of different cataract screening strategies (no screening, telescreening, AI screening and DH screening). A simulated cohort of 100,000 individuals from age 50 is built through a total of 30 1-year Markov cycles. The primary outcomes are incremental cost-effectiveness ratio and incremental cost-utility ratio. The results show that DH screening dominates no screening, telescreening and AI screening in urban and rural China. Annual DH screening emerges as the most economically effective strategy with 341 (338 to 344) and 1326 (1312 to 1340) years of blindness avoided compared with telescreening, and 37 (35 to 39) and 140 (131 to 148) years compared with AI screening in urban and rural settings, respectively. The findings remain robust across all sensitivity analyses conducted. Here, we report that DH screening is cost-effective in urban and rural China, and the annual screening proves to be the most cost-effective option, providing an economic rationale for policymakers promoting public eye health in low- and middle-income countries.
Collapse
Affiliation(s)
- Xiaohang Wu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yuxuan Wu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Zhenjun Tu
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Zizheng Cao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Miaohong Xu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yifan Xiang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Duoru Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Ling Jin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Lanqin Zhao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yingzhe Zhang
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA
| | - Yu Liu
- School of Public Health and Management, Guangzhou University of Chinese Medicine, Guangzhou, Guangdong, China
| | - Pisong Yan
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Weiling Hu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Jiali Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Lixue Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Xun Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Ruixin Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Jieying Chen
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Wei Xiao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yuanjun Shang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Peichen Xie
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Dongni Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Xulin Zhang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Meimei Dongye
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Chenxinqi Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Daniel Shu Wei Ting
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-National University of Singapore Medical School, Singapore, Singapore
| | - Yizhi Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China.
| | - Rong Pan
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China.
| | - Haotian Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China.
- Hainan Eye Hospital and Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Haikou, Hainan, China.
- Center for Precision Medicine and Department of Genetics and Biomedical Informatics, Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China.
| |
Collapse
|
8
|
Coca-Serrano R, Sánchez-Tena MA, Álvarez-Peregrina C, Martínez-Pérez C, Moriche-Carretero M. [Bibliometric study and analysis of citation networks of visual screening in primary care]. Semergen 2024; 50:102225. [PMID: 38603945 DOI: 10.1016/j.semerg.2024.102225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 01/26/2024] [Accepted: 02/16/2024] [Indexed: 04/13/2024]
Abstract
AIM Screenings make it possible to detect anomalies that can be treated and identify patients who require referral to a specialist. The objective is to identify the different areas of research and determine the most cited publications on screening in primary care. METHODS An analysis of publications and visualization of citation networks has been carried out using the Citation Network Explorer software. The bibliographic search was carried out with the Web of Science (WOS) database using the search term: "screening AND (vision OR eye OR ocular OR visual)". RESULTS We analyzed 16707 publications in all fields, 23919 citation networks have been found. The number of publications has increased, with 2021 being the year with the highest number. The majority are scientific articles and the predominant language is English. The most cited article is a global meta-analysis on the prevalence of glaucoma, showing the importance of screening for its early detection since it is essential to avoid blindness. Using the clustering function we found 8 groups with a significant number of publications where we have bibliography on certain eye diseases: glaucoma, diabetic retinopathy, pediatric amblyopia, keratoconus and dry eye. CONCLUSIONS The main areas of study in relation to screening are the detection of diseases such as glaucoma, retinopathy of prematurity, keratoconus and dry eye. As well as the detection through visual analysis of childhood amblyopia and vision loss in elderly patients. It also gives importance to performing ocular motility tests in problems of acquired brain damage.
Collapse
Affiliation(s)
| | - M A Sánchez-Tena
- Departamento de Optometría y Visión, Universidad Complutense de Madrid, Madrid, España; ISEC LISBOA - Instituto Superior de Educação e Ciências, Lisboa, Portugal
| | - C Álvarez-Peregrina
- Departamento de Optometría y Visión, Universidad Complutense de Madrid, Madrid, España
| | - C Martínez-Pérez
- ISEC LISBOA - Instituto Superior de Educação e Ciências, Lisboa, Portugal
| | | |
Collapse
|
9
|
Wong CYT, O'Byrne C, Taribagil P, Liu T, Antaki F, Keane PA. Comparing code-free and bespoke deep learning approaches in ophthalmology. Graefes Arch Clin Exp Ophthalmol 2024:10.1007/s00417-024-06432-x. [PMID: 38446200 DOI: 10.1007/s00417-024-06432-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Revised: 02/13/2024] [Accepted: 02/27/2024] [Indexed: 03/07/2024] Open
Abstract
AIM Code-free deep learning (CFDL) allows clinicians without coding expertise to build high-quality artificial intelligence (AI) models without writing code. In this review, we comprehensively review the advantages that CFDL offers over bespoke expert-designed deep learning (DL). As exemplars, we use the following tasks: (1) diabetic retinopathy screening, (2) retinal multi-disease classification, (3) surgical video classification, (4) oculomics and (5) resource management. METHODS We performed a search for studies reporting CFDL applications in ophthalmology in MEDLINE (through PubMed) from inception to June 25, 2023, using the keywords 'autoML' AND 'ophthalmology'. After identifying 5 CFDL studies looking at our target tasks, we performed a subsequent search to find corresponding bespoke DL studies focused on the same tasks. Only English-written articles with full text available were included. Reviews, editorials, protocols and case reports or case series were excluded. We identified ten relevant studies for this review. RESULTS Overall, studies were optimistic towards CFDL's advantages over bespoke DL in the five ophthalmological tasks. However, much of such discussions were identified to be mono-dimensional and had wide applicability gaps. High-quality assessment of better CFDL applicability over bespoke DL warrants a context-specific, weighted assessment of clinician intent, patient acceptance and cost-effectiveness. We conclude that CFDL and bespoke DL are unique in their own assets and are irreplaceable with each other. Their benefits are differentially valued on a case-to-case basis. Future studies are warranted to perform a multidimensional analysis of both techniques and to improve limitations of suboptimal dataset quality, poor applicability implications and non-regulated study designs. CONCLUSION For clinicians without DL expertise and easy access to AI experts, CFDL allows the prototyping of novel clinical AI systems. CFDL models concert with bespoke models, depending on the task at hand. A multidimensional, weighted evaluation of the factors involved in the implementation of those models for a designated task is warranted.
Collapse
Affiliation(s)
- Carolyn Yu Tung Wong
- Institute of Ophthalmology, University College London, 11-43 Bath St, London, EC1V 9EL, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Ciara O'Byrne
- Institute of Ophthalmology, University College London, 11-43 Bath St, London, EC1V 9EL, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Priyal Taribagil
- Institute of Ophthalmology, University College London, 11-43 Bath St, London, EC1V 9EL, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Timing Liu
- Institute of Ophthalmology, University College London, 11-43 Bath St, London, EC1V 9EL, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Fares Antaki
- Institute of Ophthalmology, University College London, 11-43 Bath St, London, EC1V 9EL, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- The CHUM School of Artificial Intelligence in Healthcare, Montreal, QC, Canada
| | - Pearse Andrew Keane
- Institute of Ophthalmology, University College London, 11-43 Bath St, London, EC1V 9EL, UK.
- Moorfields Eye Hospital NHS Foundation Trust, London, UK.
- NIHR Moorfields Biomedical Research Centre, London, UK.
| |
Collapse
|
10
|
Ramoutar RR. An Economic Analysis for the Use of Artificial Intelligence in Screening for Diabetic Retinopathy in Trinidad and Tobago. Cureus 2024; 16:e55745. [PMID: 38586698 PMCID: PMC10999161 DOI: 10.7759/cureus.55745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/07/2024] [Indexed: 04/09/2024] Open
Abstract
This is a systematic review of 25 publications on the topics of the prevalence and cost of diabetic retinopathy (DR) in Trinidad and Tobago, the cost of traditional methods of screening for DR, and the use and cost of artificial intelligence (AI) in screening for DR. Analysis of these publications was used to identify and make estimates for how resources allocated to ophthalmology in public health systems in Trinidad and Tobago can be more efficiently utilized by employing AI in diagnosing treatable DR. DR screening was found to be an effective method of detecting the disease. Screening was found to be a universally cost-effective method of disease prevention and for altering the natural history of the disease in the spectrum of low-middle to high-income economies, such as Rwanda, Thailand, China, South Korea, and Singapore. AI and deep learning systems were found to be clinically superior to, or as effective as, human graders in areas where they were deployed, indicating that the systems are clinically safe. They have been shown to improve access to diabetic retinal screening, improve compliance with screening appointments, and prove to be cost-effective, especially in rural areas. Trinidad and Tobago, which is estimated to be disproportionately more affected by the burden of DR when projected out to the mid-21st century, stands to save as much as US$60 million annually from the implementation of an AI-based system to screen for DR versus conventional manual grading.
Collapse
Affiliation(s)
- Ryan R Ramoutar
- Ophthalmology, University Hospitals of Leicester NHS Trust, Leicester, GBR
| |
Collapse
|
11
|
Gu C, Wang Y, Jiang Y, Xu F, Wang S, Liu R, Yuan W, Abudureyimu N, Wang Y, Lu Y, Li X, Wu T, Dong L, Chen Y, Wang B, Zhang Y, Wei WB, Qiu Q, Zheng Z, Liu D, Chen J. Application of artificial intelligence system for screening multiple fundus diseases in Chinese primary healthcare settings: a real-world, multicentre and cross-sectional study of 4795 cases. Br J Ophthalmol 2024; 108:424-431. [PMID: 36878715 PMCID: PMC10894824 DOI: 10.1136/bjo-2022-322940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Accepted: 02/19/2023] [Indexed: 03/08/2023]
Abstract
BACKGROUND/AIMS This study evaluates the performance of the Airdoc retinal artificial intelligence system (ARAS) for detecting multiple fundus diseases in real-world scenarios in primary healthcare settings and investigates the fundus disease spectrum based on ARAS. METHODS This real-world, multicentre, cross-sectional study was conducted in Shanghai and Xinjiang, China. Six primary healthcare settings were included in this study. Colour fundus photographs were taken and graded by ARAS and retinal specialists. The performance of ARAS is described by its accuracy, sensitivity, specificity and positive and negative predictive values. The spectrum of fundus diseases in primary healthcare settings has also been investigated. RESULTS A total of 4795 participants were included. The median age was 57.0 (IQR 39.0-66.0) years, and 3175 (66.2%) participants were female. The accuracy, specificity and negative predictive value of ARAS for detecting normal fundus and 14 retinal abnormalities were high, whereas the sensitivity and positive predictive value varied in detecting different abnormalities. The proportion of retinal drusen, pathological myopia and glaucomatous optic neuropathy was significantly higher in Shanghai than in Xinjiang. Moreover, the percentages of referable diabetic retinopathy, retinal vein occlusion and macular oedema in middle-aged and elderly people in Xinjiang were significantly higher than in Shanghai. CONCLUSION This study demonstrated the dependability of ARAS for detecting multiple retinal diseases in primary healthcare settings. Implementing the AI-assisted fundus disease screening system in primary healthcare settings might be beneficial in reducing regional disparities in medical resources. However, the ARAS algorithm must be improved to achieve better performance. TRIAL REGISTRATION NUMBER NCT04592068.
Collapse
Affiliation(s)
- Chufeng Gu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine; National Clinical Research Center for Eye Diseases; Key Laboratory of Ocular Fundus Diseases; Engineering Center for Visual Science and Photomedicine; Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Yujie Wang
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine; National Clinical Research Center for Eye Diseases; Key Laboratory of Ocular Fundus Diseases; Engineering Center for Visual Science and Photomedicine; Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Yan Jiang
- Department of Ophthalmology, Shibei Hospital of Jing'an District, Shanghai, China
| | - Feiping Xu
- Department of Ophthalmology, Shibei Hospital of Jing'an District, Shanghai, China
| | - Shasha Wang
- Department of Ophthalmology, Shibei Hospital of Jing'an District, Shanghai, China
| | - Rui Liu
- Department of Ophthalmology, Shibei Hospital of Jing'an District, Shanghai, China
| | - Wen Yuan
- Department of Ophthalmology, Shibei Hospital of Jing'an District, Shanghai, China
| | - Nurbiyimu Abudureyimu
- Department of Ophthalmology, Bachu County Traditional Chinese Medicine Hospital of Kashgar, Xinjiang, China
| | - Ying Wang
- Department of Ophthalmology, Bachu Country People's Hospital of Kashgar, Xinjiang, China
| | - Yulan Lu
- Department of Ophthalmology, Linfen Community Health Service Center of Jing'an District, Shanghai, China
| | - Xiaolong Li
- Department of Ophthalmology, Pengpu New Village Community Health Service Center of Jing'an District, Shanghai, China
| | - Tao Wu
- Department of Ophthalmology, Pengpu Town Community Health Service Center of Jing'an District, Shanghai, China
| | - Li Dong
- Beijing Tongren Eye Center, Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Ophthalmology & Visual Sciences Key Lab, Medical Artificial Intelligence Research and Verification Key Laboratory of the Ministry of Industry and Information Technology, Capital Medical University, Beijing, China
| | - Yuzhong Chen
- Beijing Airdoc Technology Co., Ltd, Beijing, China
| | - Bin Wang
- Beijing Airdoc Technology Co., Ltd, Beijing, China
| | | | - Wen Bin Wei
- Beijing Tongren Eye Center, Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Ophthalmology & Visual Sciences Key Lab, Medical Artificial Intelligence Research and Verification Key Laboratory of the Ministry of Industry and Information Technology, Capital Medical University, Beijing, China
| | - Qinghua Qiu
- Department of Ophthalmology, Tong Ren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zhi Zheng
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine; National Clinical Research Center for Eye Diseases; Key Laboratory of Ocular Fundus Diseases; Engineering Center for Visual Science and Photomedicine; Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Deng Liu
- Bachu Country People's Hospital of Kashgar, Xinjiang, China
- Shanghai No. 3 Rehabilitation Hospital, Shanghai, China
| | - Jili Chen
- Department of Ophthalmology, Shibei Hospital of Jing'an District, Shanghai, China
| |
Collapse
|
12
|
Wang Y, Liu C, Hu W, Luo L, Shi D, Zhang J, Yin Q, Zhang L, Han X, He M. Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case. NPJ Digit Med 2024; 7:43. [PMID: 38383738 PMCID: PMC10881978 DOI: 10.1038/s41746-024-01032-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 02/05/2024] [Indexed: 02/23/2024] Open
Abstract
Artificial intelligence (AI) models have shown great accuracy in health screening. However, for real-world implementation, high accuracy may not guarantee cost-effectiveness. Improving AI's sensitivity finds more high-risk patients but may raise medical costs while increasing specificity reduces unnecessary referrals but may weaken detection capability. To evaluate the trade-off between AI model performance and the long-running cost-effectiveness, we conducted a cost-effectiveness analysis in a nationwide diabetic retinopathy (DR) screening program in China, comprising 251,535 participants with diabetes over 30 years. We tested a validated AI model in 1100 different diagnostic performances (presented as sensitivity/specificity pairs) and modeled annual screening scenarios. The status quo was defined as the scenario with the most accurate AI performance. The incremental cost-effectiveness ratio (ICER) was calculated for other scenarios against the status quo as cost-effectiveness metrics. Compared to the status quo (sensitivity/specificity: 93.3%/87.7%), six scenarios were cost-saving and seven were cost-effective. To achieve cost-saving or cost-effective, the AI model should reach a minimum sensitivity of 88.2% and specificity of 80.4%. The most cost-effective AI model exhibited higher sensitivity (96.3%) and lower specificity (80.4%) than the status quo. In settings with higher DR prevalence and willingness-to-pay levels, the AI needed higher sensitivity for optimal cost-effectiveness. Urban regions and younger patient groups also required higher sensitivity in AI-based screening. In real-world DR screening, the most accurate AI model may not be the most cost-effective. Cost-effectiveness should be independently evaluated, which is most likely to be affected by the AI's sensitivity.
Collapse
Affiliation(s)
- Yueye Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Chi Liu
- Faculty of Data Science, City University of Macau, Macao SAR, China
| | - Wenyi Hu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, VIC, Australia
| | - Lixia Luo
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Danli Shi
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Jian Zhang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Qiuxia Yin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Lei Zhang
- Clinical Medical Research Center, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu, 210008, China.
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia.
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia.
| | - Xiaotong Han
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
| | - Mingguang He
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong.
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, Hong Kong.
- Centre for Eye and Vision Research (CEVR), 17W Hong Kong Science Park, Shatin, Hong Kong.
| |
Collapse
|
13
|
Hu W, Joseph S, Li R, Woods E, Sun J, Shen M, Jan CL, Zhu Z, He M, Zhang L. Population impact and cost-effectiveness of artificial intelligence-based diabetic retinopathy screening in people living with diabetes in Australia: a cost effectiveness analysis. EClinicalMedicine 2024; 67:102387. [PMID: 38314061 PMCID: PMC10837545 DOI: 10.1016/j.eclinm.2023.102387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 11/29/2023] [Accepted: 12/05/2023] [Indexed: 02/06/2024] Open
Abstract
Background We aimed to evaluate the cost-effectiveness of an artificial intelligence-(AI) based diabetic retinopathy (DR) screening system in the primary care setting for both non-Indigenous and Indigenous people living with diabetes in Australia. Methods We performed a cost-effectiveness analysis between January 01, 2022 and August 01, 2023. A decision-analytic Markov model was constructed to simulate DR progression in a population of 1,197,818 non-Indigenous and 65,160 Indigenous Australians living with diabetes aged ≥20 years over 40 years. From a healthcare provider's perspective, we compared current practice to three primary care AI-based screening scenarios-(A) substitution of current manual grading, (B) scaling up to patient acceptance level, and (C) achieving universal screening. Study results were presented as incremental cost-effectiveness ratio (ICER), benefit-cost ratio (BCR), and net monetary benefits (NMB). A Willingness-to-pay (WTP) threshold of AU$50,000 per quality-adjusted life year (QALY) and a discount rate of 3.5% were adopted in this study. Findings With the status quo, the non-Indigenous diabetic population was projected to develop 96,269 blindness cases, resulting in AU$13,039.6 m spending on DR screening and treatment during 2020-2060. In comparison, all three intervention scenarios were effective and cost-saving. In particular, if a universal screening program was to be implemented (Scenario C), it would prevent 38,347 blindness cases, gain 172,090 QALYs and save AU$595.8 m, leading to a BCR of 3.96 and NMB of AU$9,200 m. Similar findings were also reported in the Indigenous population. With the status quo, 3,396 Indigenous individuals would develop blindness, which would cost the health system AU$796.0 m during 2020-2060. All three intervention scenarios were cost-saving for the Indigenous population. Notably, universal AI-based DR screening (Scenario C) would prevent 1,211 blindness cases and gain 9,800 QALYs in the Indigenous population, leading to a saving of AU$19.2 m with a BCR of 1.62 and NMB of AU$509 m. Interpretation Our findings suggest that implementing AI-based DR screening in primary care is highly effective and cost-saving in both Indigenous and non-Indigenous populations. Funding This project received grant funding from the Australian Government: the National Critical Research Infrastructure Initiative, Medical Research Future Fund (MRFAI00035) and the NHMRC Investigator Grant (APP1175405). The contents of the published material are solely the responsibility of the Administering Institution, a participating institution or individual authors and do not reflect the views of the NHMRC. This work was supported by the Global STEM Professorship Scheme (P0046113), the Fundamental Research Funds of the State Key Laboratory of Ophthalmology, Project of Investigation on Health Status of Employees in Financial Industry in Guangzhou, China (Z012014075). The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian State Government. W.H. is supported by the Melbourne Research Scholarship established by the University of Melbourne. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Collapse
Affiliation(s)
- Wenyi Hu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Sanil Joseph
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Rui Li
- Central Clinical School, Faculty of Medicine, Monash University, Melbourne, VIC, Australia
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia
- China-Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi’an Jiaotong University Health Science Center, Xi’an, Shaanxi, 710061, PR China
| | - Ekaterina Woods
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Jason Sun
- Eyetelligence Pty Ltd., Melbourne, Australia
| | - Mingwang Shen
- China-Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi’an Jiaotong University Health Science Center, Xi’an, Shaanxi, 710061, PR China
| | - Catherine Lingxue Jan
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Zhuoting Zhu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Mingguang He
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
- School of Optometry, The Hong Kong Polytechnic University, Hong Kong, China
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Lei Zhang
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Clinical Medical Research Center, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu Province 210008, China
- Central Clinical School, Faculty of Medicine, Monash University, Melbourne, VIC, Australia
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia
| |
Collapse
|
14
|
Zuhair V, Babar A, Ali R, Oduoye MO, Noor Z, Chris K, Okon II, Rehman LU. Exploring the Impact of Artificial Intelligence on Global Health and Enhancing Healthcare in Developing Nations. J Prim Care Community Health 2024; 15:21501319241245847. [PMID: 38605668 PMCID: PMC11010755 DOI: 10.1177/21501319241245847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Revised: 03/19/2024] [Accepted: 03/21/2024] [Indexed: 04/13/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI), which combines computer science with extensive datasets, seeks to mimic human-like intelligence. Subsets of AI are being applied in almost all fields of medicine and surgery. AIM This review focuses on the applications of AI in healthcare settings in developing countries, designed to underscore its significance by comprehensively outlining the advancements made thus far, the shortcomings encountered in AI applications, the present status of AI integration, persistent challenges, and innovative strategies to surmount them. METHODOLOGY Articles from PubMed, Google Scholar, and Cochrane were searched from 2000 to 2023 with keywords including AI and healthcare, focusing on multiple medical specialties. RESULTS The increasing role of AI in diagnosis, prognosis prediction, and patient management, as well as hospital management and community healthcare, has made the overall healthcare system more efficient, especially in the high patient load setups and resource-limited areas of developing countries where patient care is often compromised. However, challenges, including low adoption rates and the absence of standardized guidelines, high installation and maintenance costs of equipment, poor transportation and connectivvity issues hinder AI's full use in healthcare. CONCLUSION Despite these challenges, AI holds a promising future in healthcare. Adequate knowledge and expertise of healthcare professionals for the use of AI technology in healthcare is imperative in developing nations.
Collapse
Affiliation(s)
- Varisha Zuhair
- Jinnah Sindh Medical University, Karachi, Sindh, Pakistan
| | - Areesha Babar
- Jinnah Sindh Medical University, Karachi, Sindh, Pakistan
| | - Rabbiya Ali
- Jinnah Sindh Medical University, Karachi, Sindh, Pakistan
| | - Malik Olatunde Oduoye
- The Medical Research Circle, (MedReC), Gisenyi, Goma, Democratic Republic of the Congo
| | - Zainab Noor
- Institute of Dentistry CMH Lahore Medical College, Lahore, Punjab, Pakistan
| | - Kitumaini Chris
- The Medical Research Circle, (MedReC), Gisenyi, Goma, Democratic Republic of the Congo
- Université Libre des Pays des Grands-Lacs Goma, Noth-Kivu, Democratic Republic of the Congo
| | - Inibehe Ime Okon
- The Medical Research Circle, (MedReC), Gisenyi, Goma, Democratic Republic of the Congo
- NiMSA SCOPH, Uyo, Akwa-Ibom State, Nigeria
| | | |
Collapse
|
15
|
Sarayar R, Lestari YD, Setio AAA, Sitompul R. Accuracy of artificial intelligence model for infectious keratitis classification: a systematic review and meta-analysis. Front Public Health 2023; 11:1239231. [PMID: 38074720 PMCID: PMC10704127 DOI: 10.3389/fpubh.2023.1239231] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 11/03/2023] [Indexed: 12/18/2023] Open
Abstract
Background Infectious keratitis (IK) is a sight-threatening condition requiring immediate definite treatment. The need for prompt treatment heavily depends on timely diagnosis. The diagnosis of IK, however, is challenged by the drawbacks of the current "gold standard." The poorly differentiated clinical features, the possibility of low microbial culture yield, and the duration for culture are the culprits of delayed IK treatment. Deep learning (DL) is a recent artificial intelligence (AI) advancement that has been demonstrated to be highly promising in making automated diagnosis in IK with high accuracy. However, its exact accuracy is not yet elucidated. This article is the first systematic review and meta-analysis that aims to assess the accuracy of available DL models to correctly classify IK based on etiology compared to the current gold standards. Methods A systematic search was carried out in PubMed, Google Scholars, Proquest, ScienceDirect, Cochrane and Scopus. The used keywords are: "Keratitis," "Corneal ulcer," "Corneal diseases," "Corneal lesions," "Artificial intelligence," "Deep learning," and "Machine learning." Studies including slit lamp photography of the cornea and validity study on DL performance were considered. The primary outcomes reviewed were the accuracy and classification capability of the AI machine learning/DL algorithm. We analyzed the extracted data with the MetaXL 5.2 Software. Results A total of eleven articles from 2002 to 2022 were included with a total dataset of 34,070 images. All studies used convolutional neural networks (CNNs), with ResNet and DenseNet models being the most used models across studies. Most AI models outperform the human counterparts with a pooled area under the curve (AUC) of 0.851 and accuracy of 96.6% in differentiating IK vs. non-IK and pooled AUC 0.895 and accuracy of 64.38% for classifying bacterial keratitis (BK) vs. fungal keratitis (FK). Conclusion This study demonstrated that DL algorithms have high potential in diagnosing and classifying IK with accuracy that, if not better, is comparable to trained corneal experts. However, various factors, such as the unique architecture of DL model, the problem with overfitting, image quality of the datasets, and the complex nature of IK itself, still hamper the universal applicability of DL in daily clinical practice.
Collapse
Affiliation(s)
- Randy Sarayar
- Residency Program in Ophthalmology Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia
| | - Yeni Dwi Lestari
- Department of Ophthalmology, Faculty of Medicine Universitas Indonesia – Cipto Mangunkusumo General Hospital, Jakarta, Indonesia
| | - Arnaud A. A. Setio
- Digital Technology and Innovation, Siemens Healthineers, Erlangen, Germany
| | - Ratna Sitompul
- Department of Ophthalmology, Faculty of Medicine Universitas Indonesia – Cipto Mangunkusumo General Hospital, Jakarta, Indonesia
| |
Collapse
|
16
|
Li H, Li G, Li N, Liu C, Yuan Z, Gao Q, Hao S, Fan S, Yang J. Cost-effectiveness analysis of artificial intelligence-based diabetic retinopathy screening in rural China based on the Markov model. PLoS One 2023; 18:e0291390. [PMID: 37971984 PMCID: PMC10653408 DOI: 10.1371/journal.pone.0291390] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 08/26/2023] [Indexed: 11/19/2023] Open
Abstract
This study assessed the cost-effectiveness of different diabetic retinopathy (DR) screening strategies in rural regions in China by using a Markov model to make health economic evaluations. In this study, we determined the structure of a Markov model according to the research objectives, which required parameters collected through field investigation and literature retrieval. After perfecting the model with parameters and assumptions, we developed a Markov decision analytic model according to the natural history of DR in TreeAge Pro 2011. For this model, we performed Markov cohort and cost-effectiveness analyses to simulate the probabilistic distributions of different developments in DR and the cumulative cost-effectiveness of artificial intelligence (AI)-based screening and ophthalmologist screening for DR in the rural population with diabetes mellitus (DM) in China. Additionally, a model-based health economic evaluation was performed by using quality-adjusted life years (QALYs) and incremental cost-effectiveness ratios. Last, one-way and probabilistic sensitivity analyses were performed to assess the stability of the results. From the perspective of the health system, compared with no screening, AI-based screening cost more (the incremental cost was 37,257.76 RMB (approximately 5,211.31 US dollars)), but the effect was better (the incremental utility was 0.33). Compared with AI-based screening, the cost of ophthalmologist screening was higher (the incremental cost was 14,886.76 RMB (approximately 2,070.19 US dollars)), and the effect was worse (the incremental utility was -0.31). Compared with no screening, the incremental cost-effectiveness ratio (ICER) of AI-based DR screening was 112,146.99 RMB (15,595.47 US dollars)/QALY, which was less than the threshold for the ICER (< 3 times the per capita gross domestic product (GDP), 217,341.00 RMB (30,224.03 US dollars)). Therefore, AI-based screening was cost-effective, which meant that the increased cost for each additional quality-adjusted life year was merited. Compared with no screening and ophthalmologist screening for DR, AI-based screening was the most cost-effective, which not only saved costs but also improved the quality of life of diabetes patients. Popularizing AI-based DR screening strategies in rural areas would be economically effective and feasible and can provide a scientific basis for the further formulation of early screening programs for diabetic retinopathy.
Collapse
Affiliation(s)
- Huilin Li
- Department of Ophthalmology, Heji Hospital Affiliated to Changzhi Medical College, Changzhi, 046000, China
| | - Guanyan Li
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
- Shenzhen Longgang Otorhinolaryngology Hospital, Shenzhen, 518100, China
| | - Na Li
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Changyan Liu
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Ziyou Yuan
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Qingyue Gao
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Shaofeng Hao
- Department of Ophthalmology, Heji Hospital Affiliated to Changzhi Medical College, Changzhi, 046000, China
| | - Shengfu Fan
- Department of Foreign Languages, Changzhi Medical College, Changzhi, 046000, China
| | - Jianzhou Yang
- Department of Public Health and Preventive Medicine, Changzhi Medical College, Changzhi, 046000, China
| |
Collapse
|
17
|
Li Q, Qin Y. AI in medical education: medical student perception, curriculum recommendations and design suggestions. BMC MEDICAL EDUCATION 2023; 23:852. [PMID: 37946176 PMCID: PMC10637014 DOI: 10.1186/s12909-023-04700-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Accepted: 09/19/2023] [Indexed: 11/12/2023]
Abstract
Medical AI has transformed modern medicine and created a new environment for future doctors. However, medical education has failed to keep pace with these advances, and it is essential to provide systematic education on medical AI to current medical undergraduate and postgraduate students. To address this issue, our study utilized the Unified Theory of Acceptance and Use of Technology model to identify key factors that influence the acceptance and intention to use medical AI. We collected data from 1,243 undergraduate and postgraduate students from 13 universities and 33 hospitals, and 54.3% reported prior experience using medical AI. Our findings indicated that medical postgraduate students have a higher level of awareness in using medical AI than undergraduate students. The intention to use medical AI is positively associated with factors such as performance expectancy, habit, hedonic motivation, and trust. Therefore, future medical education should prioritize promoting students' performance in training, and courses should be designed to be both easy to learn and engaging, ensuring that students are equipped with the necessary skills to succeed in their future medical careers.
Collapse
Affiliation(s)
- Qianying Li
- Antai College of economics and management, Shanghai Jiao Tong University, Shanghai, China
| | - Yunhao Qin
- Department of Orthopedics, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
18
|
Kuklinski EJ, Henry RK, Shah M, Zarbin MA, Szirth B, Bhagat N. Screening of Diabetic Retinopathy Using Artificial Intelligence and Tele-Ophthalmology. J Diabetes Sci Technol 2023; 17:1724-1725. [PMID: 37642475 PMCID: PMC10658675 DOI: 10.1177/19322968231194041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Affiliation(s)
| | - Roger K. Henry
- Department of Ophthalmology, Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Megh Shah
- Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Marco A. Zarbin
- Department of Ophthalmology, Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Bernard Szirth
- Department of Ophthalmology, Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Neelakshi Bhagat
- Department of Ophthalmology, Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| |
Collapse
|
19
|
Rajesh AE, Davidson OQ, Lee CS, Lee AY. Artificial Intelligence and Diabetic Retinopathy: AI Framework, Prospective Studies, Head-to-head Validation, and Cost-effectiveness. Diabetes Care 2023; 46:1728-1739. [PMID: 37729502 PMCID: PMC10516248 DOI: 10.2337/dci23-0032] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 07/15/2023] [Indexed: 09/22/2023]
Abstract
Current guidelines recommend that individuals with diabetes receive yearly eye exams for detection of referable diabetic retinopathy (DR), one of the leading causes of new-onset blindness. For addressing the immense screening burden, artificial intelligence (AI) algorithms have been developed to autonomously screen for DR from fundus photography without human input. Over the last 10 years, many AI algorithms have achieved good sensitivity and specificity (>85%) for detection of referable DR compared with human graders; however, many questions still remain. In this narrative review on AI in DR screening, we discuss key concepts in AI algorithm development as a background for understanding the algorithms. We present the AI algorithms that have been prospectively validated against human graders and demonstrate the variability of reference standards and cohort demographics. We review the limited head-to-head validation studies where investigators attempt to directly compare the available algorithms. Next, we discuss the literature regarding cost-effectiveness, equity and bias, and medicolegal considerations, all of which play a role in the implementation of these AI algorithms in clinical practice. Lastly, we highlight ongoing efforts to bridge gaps in AI model data sets to pursue equitable development and delivery.
Collapse
Affiliation(s)
- Anand E. Rajesh
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Oliver Q. Davidson
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Cecilia S. Lee
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Aaron Y. Lee
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| |
Collapse
|
20
|
Nakayama LF, Zago Ribeiro L, Novaes F, Miyawaki IA, Miyawaki AE, de Oliveira JAE, Oliveira T, Malerbi FK, Regatieri CVS, Celi LA, Silva PS. Artificial intelligence for telemedicine diabetic retinopathy screening: a review. Ann Med 2023; 55:2258149. [PMID: 37734417 PMCID: PMC10515659 DOI: 10.1080/07853890.2023.2258149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Accepted: 08/31/2023] [Indexed: 09/23/2023] Open
Abstract
PURPOSE This study aims to compare artificial intelligence (AI) systems applied in diabetic retinopathy (DR) teleophthalmology screening, currently deployed systems, fairness initiatives and the challenges for implementation. METHODS The review included articles retrieved from PubMed/Medline/EMBASE literature search strategy regarding telemedicine, DR and AI. The screening criteria included human articles in English, Portuguese or Spanish and related to telemedicine and AI for DR screening. The author's affiliations and the study's population income group were classified according to the World Bank Country and Lending Groups. RESULTS The literature search yielded a total of 132 articles, and nine were included after full-text assessment. The selected articles were published between 2004 and 2020 and were grouped as telemedicine systems, algorithms, economic analysis and image quality assessment. Four telemedicine systems that perform a quality assessment, image preprocessing and pathological screening were reviewed. A data and post-deployment bias assessment are not performed in any of the algorithms, and none of the studies evaluate the social impact implementations. There is a lack of representativeness in the reviewed articles, with most authors and target populations from high-income countries and no low-income country representation. CONCLUSIONS Telemedicine and AI hold great promise for augmenting decision-making in medical care, expanding patient access and enhancing cost-effectiveness. Economic studies and social science analysis are crucial to support the implementation of AI in teleophthalmology screening programs. Promoting fairness and generalizability in automated systems combined with telemedicine screening programs is not straightforward. Improving data representativeness, reducing biases and promoting equity in deployment and post-deployment studies are all critical steps in model development.
Collapse
Affiliation(s)
- Luis Filipe Nakayama
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | - Lucas Zago Ribeiro
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | - Frederico Novaes
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | | | | | | | - Talita Oliveira
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | | | | | - Leo Anthony Celi
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biostatistics, Harvard TH Chan School of Public Health, Boston, MA, USA
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Paolo S. Silva
- Beetham Eye Institute, Joslin Diabetes Centre, Harvard Medical School, Boston, MA, USA
- Philippine Eye Research Institute, University of the Philippines, Manila, Philippines
| |
Collapse
|
21
|
Ruamviboonsuk P, Ruamviboonsuk V, Tiwari R. Recent evidence of economic evaluation of artificial intelligence in ophthalmology. Curr Opin Ophthalmol 2023; 34:449-458. [PMID: 37459289 DOI: 10.1097/icu.0000000000000987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2023]
Abstract
PURPOSE OF REVIEW Health economic evaluation (HEE) is essential for assessing value of health interventions, including artificial intelligence. Recent approaches, current challenges, and future directions of HEE of artificial intelligence in ophthalmology are reviewed. RECENT FINDINGS Majority of recent HEEs of artificial intelligence in ophthalmology were for diabetic retinopathy screening. Two models, one conducted in the rural USA (5-year period) and another in China (35-year period), found artificial intelligence to be more cost-effective than without screening for diabetic retinopathy. Two additional models, which compared artificial intelligence with human screeners in Brazil and Thailand for the lifetime of patients, found artificial intelligence to be more expensive from a healthcare system perspective. In the Thailand analysis, however, artificial intelligence was less expensive when opportunity loss from blindness was included. An artificial intelligence model for screening retinopathy of prematurity was cost-effective in the USA. A model for screening age-related macular degeneration in Japan and another for primary angle close in China did not find artificial intelligence to be cost-effective, compared with no screening. The costs of artificial intelligence varied widely in these models. SUMMARY Like other medical fields, there is limited evidence in assessing the value of artificial intelligence in ophthalmology and more appropriate HEE models are needed.
Collapse
Affiliation(s)
- Paisan Ruamviboonsuk
- Department of Ophthalmology, Rajavithi Hospital, College of Medicine, Rangsit University
| | | | | |
Collapse
|
22
|
Rizvi A, Rizvi F, Lalakia P, Hyman L, Frasso R, Sztandera L, Das AV. Is Artificial Intelligence the Cost-Saving Lens to Diabetic Retinopathy Screening in Low- and Middle-Income Countries? Cureus 2023; 15:e45539. [PMID: 37868419 PMCID: PMC10586227 DOI: 10.7759/cureus.45539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/17/2023] [Indexed: 10/24/2023] Open
Abstract
Diabetes is a rapidly growing global health crisis disproportionately affecting low- and middle-income countries (LMICs). The emergence of diabetes as a global pandemic is one of the major challenges to human health, as long-term microvascular complications such as diabetic retinopathy (DR) can lead to irreversible blindness. Leveraging artificial intelligence (AI) technology may improve the diagnostic accuracy, efficiency, and accessibility of DR screenings across LMICs. However, there is a gap between the potential of AI technology and its implementation in clinical practice. The main objective of this systematic review is to summarize the currently available literature on the health economic assessments of AI implementation for DR screening in LMICs. The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We conducted an extensive systematic search of PubMed/MEDLINE, Scopus, and the Web of Science on July 15, 2023. Our review included full-text English-language articles from any publication year. The Joanna Briggs Institute's (JBI) critical appraisal checklist for economic evaluations was used to rate the quality and rigor of the selected articles. The initial search generated 1,423 records and was narrowed to five full-text articles through comprehensive inclusion and exclusion criteria. Of the five articles included in our systematic review, two used a cost-effectiveness analysis, two used a cost-utility analysis, and one used both a cost-effectiveness analysis and a cost-utility analysis. Across the five articles, LMICs such as China, Thailand, and Brazil were represented in the economic evaluations and models. Overall, three out of the five articles concluded that AI-based DR screening was more cost-effective in comparison to standard-of-care screening methods. Our systematic review highlights the need for more primary health economic analyses that carefully evaluate the economic implications of adopting AI technology for DR screening in LMICs. We hope this systematic review will offer valuable guidance to healthcare providers, scientists, and legislators to support appropriate decision-making regarding the implementation of AI algorithms for DR screening in healthcare workflows.
Collapse
Affiliation(s)
- Anza Rizvi
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
| | - Fatima Rizvi
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
| | - Parth Lalakia
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
- Osteopathic Medicine, Rowan-Virtua School of Osteopathic Medicine, Stratford, USA
- Office of Global Affairs, Thomas Jefferson University, Philadelphia, USA
| | - Leslie Hyman
- Geriatric Medicine and Palliative Care, Department of Family Medicine, Thomas Jefferson University, Philadelphia, USA
- The Vickie and Jack Farber Vision Research Center, Wills Eye Hospital, Philadelphia, USA
| | - Rosemary Frasso
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
- Asano-Gonnella Center for Research in Medical Education and Health Care, Thomas Jefferson University, Philadelphia, USA
| | - Les Sztandera
- Kanbar College of Design, Engineering, and Commerce, Thomas Jefferson University, Philadelphia, USA
| | | |
Collapse
|
23
|
Vithlani J, Hawksworth C, Elvidge J, Ayiku L, Dawoud D. Economic evaluations of artificial intelligence-based healthcare interventions: a systematic literature review of best practices in their conduct and reporting. Front Pharmacol 2023; 14:1220950. [PMID: 37693892 PMCID: PMC10486896 DOI: 10.3389/fphar.2023.1220950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 07/25/2023] [Indexed: 09/12/2023] Open
Abstract
Objectives: Health economic evaluations (HEEs) help healthcare decision makers understand the value of new technologies. Artificial intelligence (AI) is increasingly being used in healthcare interventions. We sought to review the conduct and reporting of published HEEs for AI-based health interventions. Methods: We conducted a systematic literature review with a 15-month search window (April 2021 to June 2022) on 17th June 2022 to identify HEEs of AI health interventions and update a previous review. Records were identified from 3 databases (Medline, Embase, and Cochrane Central). Two reviewers screened papers against predefined study selection criteria. Data were extracted from included studies using prespecified data extraction tables. Included studies were quality assessed using the National Institute for Health and Care Excellence (NICE) checklist. Results were synthesized narratively. Results: A total of 21 studies were included. The most common type of AI intervention was automated image analysis (9/21, 43%) mainly used for screening or diagnosis in general medicine and oncology. Nearly all were cost-utility (10/21, 48%) or cost-effectiveness analyses (8/21, 38%) that took a healthcare system or payer perspective. Decision-analytic models were used in 16/21 (76%) studies, mostly Markov models and decision trees. Three (3/16, 19%) used a short-term decision tree followed by a longer-term Markov component. Thirteen studies (13/21, 62%) reported the AI intervention to be cost effective or dominant. Limitations tended to result from the input data, authorship conflicts of interest, and a lack of transparent reporting, especially regarding the AI nature of the intervention. Conclusion: Published HEEs of AI-based health interventions are rapidly increasing in number. Despite the potentially innovative nature of AI, most have used traditional methods like Markov models or decision trees. Most attempted to assess the impact on quality of life to present the cost per QALY gained. However, studies have not been comprehensively reported. Specific reporting standards for the economic evaluation of AI interventions would help improve transparency and promote their usefulness for decision making. This is fundamental for reimbursement decisions, which in turn will generate the necessary data to develop flexible models better suited to capturing the potentially dynamic nature of AI interventions.
Collapse
Affiliation(s)
- Jai Vithlani
- National Institute for Health and Care Excellence, London, United Kingdom
| | - Claire Hawksworth
- National Institute for Health and Care Excellence, Manchester, United Kingdom
| | - Jamie Elvidge
- National Institute for Health and Care Excellence, Manchester, United Kingdom
| | - Lynda Ayiku
- National Institute for Health and Care Excellence, Manchester, United Kingdom
| | - Dalia Dawoud
- National Institute for Health and Care Excellence, London, United Kingdom
- Faculty of Pharmacy, Cairo University, Cairo, Egypt
| |
Collapse
|
24
|
Cleland CR, Rwiza J, Evans JR, Gordon I, MacLeod D, Burton MJ, Bascaran C. Artificial intelligence for diabetic retinopathy in low-income and middle-income countries: a scoping review. BMJ Open Diabetes Res Care 2023; 11:e003424. [PMID: 37532460 PMCID: PMC10401245 DOI: 10.1136/bmjdrc-2023-003424] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 07/11/2023] [Indexed: 08/04/2023] Open
Abstract
Diabetic retinopathy (DR) is a leading cause of blindness globally. There is growing evidence to support the use of artificial intelligence (AI) in diabetic eye care, particularly for screening populations at risk of sight loss from DR in low-income and middle-income countries (LMICs) where resources are most stretched. However, implementation into clinical practice remains limited. We conducted a scoping review to identify what AI tools have been used for DR in LMICs and to report their performance and relevant characteristics. 81 articles were included. The reported sensitivities and specificities were generally high providing evidence to support use in clinical practice. However, the majority of studies focused on sensitivity and specificity only and there was limited information on cost, regulatory approvals and whether the use of AI improved health outcomes. Further research that goes beyond reporting sensitivities and specificities is needed prior to wider implementation.
Collapse
Affiliation(s)
- Charles R Cleland
- International Centre for Eye Health, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK
- Eye Department, Kilimanjaro Christian Medical Centre, Moshi, United Republic of Tanzania
| | - Justus Rwiza
- Eye Department, Kilimanjaro Christian Medical Centre, Moshi, United Republic of Tanzania
| | - Jennifer R Evans
- International Centre for Eye Health, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK
| | - Iris Gordon
- International Centre for Eye Health, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK
| | - David MacLeod
- Tropical Epidemiology Group, Department of Infectious Disease Epidemiology, London School of Hygiene & Tropical Medicine, London, UK
| | - Matthew J Burton
- International Centre for Eye Health, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK
- National Institute for Health Research Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Covadonga Bascaran
- International Centre for Eye Health, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK
| |
Collapse
|
25
|
Srisubat A, Kittrongsiri K, Sangroongruangsri S, Khemvaranan C, Shreibati JB, Ching J, Hernandez J, Tiwari R, Hersch F, Liu Y, Hanutsaha P, Ruamviboonsuk V, Turongkaravee S, Raman R, Ruamviboonsuk P. Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program. Ophthalmol Ther 2023; 12:1339-1357. [PMID: 36841895 PMCID: PMC10011252 DOI: 10.1007/s40123-023-00688-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Accepted: 02/10/2023] [Indexed: 02/27/2023] Open
Abstract
INTRODUCTION Deep learning (DL) for screening diabetic retinopathy (DR) has the potential to address limited healthcare resources by enabling expanded access to healthcare. However, there is still limited health economic evaluation, particularly in low- and middle-income countries, on this subject to aid decision-making for DL adoption. METHODS In the context of a middle-income country (MIC), using Thailand as a model, we constructed a decision tree-Markov hybrid model to estimate lifetime costs and outcomes of Thailand's national DR screening program via DL and trained human graders (HG). We calculated the incremental cost-effectiveness ratio (ICER) between the two strategies. Sensitivity analyses were performed to probe the influence of modeling parameters. RESULTS From a societal perspective, screening with DL was associated with a reduction in costs of ~ US$ 2.70, similar quality-adjusted life-years (QALY) of + 0.0043, and an incremental net monetary benefit of ~ US$ 24.10 in the base case. In sensitivity analysis, DL remained cost-effective even with a price increase from US$ 1.00 to US$ 4.00 per patient at a Thai willingness-to-pay threshold of ~ US$ 4.997 per QALY gained. When further incorporating recent findings suggesting improved compliance to treatment referral with DL, our analysis models effectiveness benefits of ~ US$ 20 to US$ 50 depending on compliance. CONCLUSION DR screening using DL in an MIC using Thailand as a model may result in societal cost-savings and similar health outcomes compared with HG. This study may provide an economic rationale to expand DL-based DR screening in MICs as an alternative solution for limited availability of skilled human resources for primary screening, particularly in MICs with similar prevalence of diabetes and low compliance to referrals for treatment.
Collapse
Affiliation(s)
- Attasit Srisubat
- Department of Medical Services, Ministry of Public Health, Nonthaburi, Thailand
| | - Kankamon Kittrongsiri
- Social, Economic and Administrative Pharmacy (SEAP) Graduate Program, Faculty of Pharmacy, Mahidol University, Bangkok, Thailand
| | - Sermsiri Sangroongruangsri
- Social and Administrative Pharmacy Division, Department of Pharmacy, Faculty of Pharmacy, Mahidol University, Bangkok, Thailand.
| | - Chalida Khemvaranan
- Department of Research and Technology Assessment, Lerdsin Hospital, Bangkok, Thailand
| | | | | | | | | | | | - Yun Liu
- Google LLC, Mountain View, CA, USA
| | - Prut Hanutsaha
- Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand
| | | | - Saowalak Turongkaravee
- Social and Administrative Pharmacy Division, Department of Pharmacy, Faculty of Pharmacy, Mahidol University, Bangkok, Thailand
| | - Rajiv Raman
- Sri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Chennai, Tamil Nadu, India
| | - Paisan Ruamviboonsuk
- Department of Ophthalmology, College of Medicine, Rajavithi Hospital, Rangsit University, Bangkok, Thailand.
| |
Collapse
|
26
|
Ferrier C, Bendifallah S, Suisse S, Dabi Y, Touboul C, Puchar A, Zarca K, Durand Zaleski I. Saliva microRNA signature to diagnose endometriosis: A cost-effectiveness evaluation of the Endotest®. BJOG 2023; 130:396-406. [PMID: 36424910 DOI: 10.1111/1471-0528.17348] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Revised: 11/03/2022] [Accepted: 11/05/2022] [Indexed: 11/26/2022]
Abstract
OBJECTIVE To evaluate a saliva diagnostic test (Endotest®) for endometriosis compared with the conventional algorithm. DESIGN A cost-effectiveness analysis with a decision-tree model based on literature data. SETTING France. POPULATION Women with chronic pelvic pain. METHODS Strategy I is the French algorithm, representing the comparator. For strategy II, all patients have an Endotest®. For strategy III, patients undergo ultrasonography to detect endometrioma and patients with no endometrioma detected have an Endotest®. For strategy IV, patients with no endometrioma detected on ultrasonography undergo pelvic magnetic resonance imaging (MRI) to detect endometrioma and/or deep endometriosis. An Endotest® is then performed for patients with a negative result on MRI. MAIN OUTCOMES MEASURES Costs and accuracy rates and incremental cost-effectiveness ratios (ICERs). Three analyses were performed with an Endotest® priced at €500, €750, and €1000. Probabilistic sensitivity analysis was conducted with Monte Carlo simulations. RESULTS With an Endotest® priced at €750, the cost per correctly diagnosed case was €1542, €990, €919 and €1000, respectively, for strategies I, II, III and IV. Strategy I was dominated by all other strategies. Strategies IV, III and II were, respectively, preferred for a willingness-to-pay threshold below €473, between €473 and €4670, and beyond €4670 per correctly diagnosed case. At a price of €500 per Endotest®, strategy I was dominated by all other strategies. At €1000, the ICERs of strategies II and III were €724 and €387 per correctly diagnosed case, respectively, compared with strategy I. CONCLUSION The present study demonstrates the value of the Endotest® from an economic perspective.
Collapse
Affiliation(s)
- Clement Ferrier
- Department of Obstetrics and Reproductive Medicine, Hôpital Tenon, Paris, France.,Clinical Research Group (GRC) Paris 6: Centre Expert Endométriose (C3E), Sorbonne University (GRC6 C3E SU), Paris, France
| | - Sofiane Bendifallah
- Department of Obstetrics and Reproductive Medicine, Hôpital Tenon, Paris, France.,Clinical Research Group (GRC) Paris 6: Centre Expert Endométriose (C3E), Sorbonne University (GRC6 C3E SU), Paris, France
| | | | - Yohann Dabi
- Department of Obstetrics and Reproductive Medicine, Hôpital Tenon, Paris, France.,Clinical Research Group (GRC) Paris 6: Centre Expert Endométriose (C3E), Sorbonne University (GRC6 C3E SU), Paris, France
| | - Cyril Touboul
- Department of Obstetrics and Reproductive Medicine, Hôpital Tenon, Paris, France.,Clinical Research Group (GRC) Paris 6: Centre Expert Endométriose (C3E), Sorbonne University (GRC6 C3E SU), Paris, France
| | - Anne Puchar
- Department of Obstetrics and Reproductive Medicine, Hôpital Tenon, Paris, France.,Clinical Research Group (GRC) Paris 6: Centre Expert Endométriose (C3E), Sorbonne University (GRC6 C3E SU), Paris, France
| | - Kevin Zarca
- Assistance Publique-Hôpitaux de Paris, DRCI-URC Eco Ile-de-France, Paris, France.,Université de Paris, Research Centre of Research Epidemiology and Statistics (CRESS-UMR1153), Inserm, Paris, France
| | - Isabelle Durand Zaleski
- Assistance Publique-Hôpitaux de Paris, DRCI-URC Eco Ile-de-France, Paris, France.,Université de Paris, Research Centre of Research Epidemiology and Statistics (CRESS-UMR1153), Inserm, Paris, France
| |
Collapse
|
27
|
Vodanović M, Subašić M, Milošević D, Savić Pavičin I. Artificial Intelligence in Medicine and Dentistry. Acta Stomatol Croat 2023; 57:70-84. [PMID: 37288152 PMCID: PMC10243707 DOI: 10.15644/asc57/1/8] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Accepted: 03/01/2023] [Indexed: 09/14/2023] Open
Abstract
INTRODUCTION Artificial intelligence has been applied in various fields throughout history, but its integration into daily life is more recent. The first applications of AI were primarily in academia and government research institutions, but as technology has advanced, AI has also been applied in industry, commerce, medicine and dentistry. OBJECTIVE Considering that the possibilities of applying artificial intelligence are developing rapidly and that this field is one of the areas with the greatest increase in the number of newly published articles, the aim of this paper was to provide an overview of the literature and to give an insight into the possibilities of applying artificial intelligence in medicine and dentistry. In addition, the aim was to discuss its advantages and disadvantages. CONCLUSION The possibilities of applying artificial intelligence to medicine and dentistry are just being discovered. Artificial intelligence will greatly contribute to developments in medicine and dentistry, as it is a tool that enables development and progress, especially in terms of personalized healthcare that will lead to much better treatment outcomes.
Collapse
Affiliation(s)
- Marin Vodanović
- Department of Dental Anthropology, School of Dental Medicine, University of Zagreb, Croatia
- University Hospital Centre Zagreb, Croatia
| | - Marko Subašić
- Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia
| | - Denis Milošević
- Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia
| | - Ivana Savić Pavičin
- Department of Dental Anthropology, School of Dental Medicine, University of Zagreb, Croatia
- University Hospital Centre Zagreb, Croatia
| |
Collapse
|
28
|
Yang Y, Tan J, He Y, Huang H, Wang T, Gong J, Liu Y, Zhang Q, Xu X. Predictive model for diabetic retinopathy under limited medical resources: A multicenter diagnostic study. Front Endocrinol (Lausanne) 2023; 13:1099302. [PMID: 36686423 PMCID: PMC9849672 DOI: 10.3389/fendo.2022.1099302] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Accepted: 12/19/2022] [Indexed: 01/06/2023] Open
Abstract
Background Comprehensive eye examinations for diabetic retinopathy is poorly implemented in medically underserved areas. There is a critical need for a widely available and economical tool to aid patient selection for priority retinal screening. We investigated the possibility of a predictive model for retinopathy identification using simple parameters. Methods Clinical data were retrospectively collected from 4, 159 patients with diabetes admitted to five tertiary hospitals. Independent predictors were identified by univariate analysis and least absolute shrinkage and selection operator (LASSO) regression, and a nomogram was developed based on a multivariate logistic regression model. The validity and clinical practicality of this nomogram were assessed using concordance index (C-index), area under the receiver operating characteristic curve (AUROC), calibration curves, decision curve analysis (DCA), and clinical impact curves (CIC). Results The predictive factors in the multivariate model included the duration of diabetes, history of hypertension, and cardiovascular disease. The three-variable model displayed medium prediction ability with an AUROC of 0.722 (95%CI 0.696-0.748) in the training set, 0.715 (95%CI 0.670-0.754) in the internal set, and 0.703 (95%CI 0.552-0.853) in the external dataset. DCA showed that the threshold probability of DR in diabetic patients was 17-55% according to the nomogram, and CIC also showed that the nomogram could be applied clinically if the risk threshold exceeded 30%. An operation interface on a webpage (https://cqmuxss.shinyapps.io/dr_tjj/) was built to improve the clinical utility of the nomogram. Conclusions The predictive model developed based on a minimal amount of clinical data available to diabetic patients with restricted medical resources could help primary healthcare practitioners promptly identify potential retinopathy.
Collapse
Affiliation(s)
- Yanzhi Yang
- Department of Endocrinology and Metabolism, Chengdu First People’s Hospital, Chengdu, China
| | - Juntao Tan
- Operation Management Office, Affiliated Banan Hospital of Chongqing Medical University, Chongqing, China
| | - Yuxin He
- Department of Medical Administration, Affiliated Banan Hospital of Chongqing Medical University, Chongqing, China
| | - Huanhuan Huang
- Department of Nursing, the First Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Tingting Wang
- College of Medical Informatics, Chongqing Medical University, Chongqing, China
| | - Jun Gong
- Department of Information Center, The University Town Hospital of Chongqing Medical University, Chongqing, China
| | - Yunyu Liu
- Medical Records Department, the Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Qin Zhang
- Department of Endocrinology and Metabolism, Chengdu First People’s Hospital, Chengdu, China
| | - Xiaomei Xu
- Department of Infectious Diseases, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China
- Department of Gastroenterology, Chengdu Fifth People’s hospital, Chengdu, China
| |
Collapse
|