1
|
Kubin AM, Huhtinen P, Ohtonen P, Keskitalo A, Wirkkala J, Hautala N. Comparison of 21 artificial intelligence algorithms in automated diabetic retinopathy screening using handheld fundus camera. Ann Med 2024; 56:2352018. [PMID: 38738798 PMCID: PMC11095279 DOI: 10.1080/07853890.2024.2352018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 04/21/2024] [Indexed: 05/14/2024] Open
Abstract
BACKGROUND Diabetic retinopathy (DR) is a common complication of diabetes and may lead to irreversible visual loss. Efficient screening and improved treatment of both diabetes and DR have amended visual prognosis for DR. The number of patients with diabetes is increasing and telemedicine, mobile handheld devices and automated solutions may alleviate the burden for healthcare. We compared the performance of 21 artificial intelligence (AI) algorithms for referable DR screening in datasets taken by handheld Optomed Aurora fundus camera in a real-world setting. PATIENTS AND METHODS Prospective study of 156 patients (312 eyes) attending DR screening and follow-up. Both papilla- and macula-centred 50° fundus images were taken from each eye. DR was graded by experienced ophthalmologists and 21 AI algorithms. RESULTS Most eyes, 183 out of 312 (58.7%), had no DR and mild NPDR was noted in 21 (6.7%) of the eyes. Moderate NPDR was detected in 66 (21.2%) of the eyes, severe NPDR in 1 (0.3%), and PDR in 41 (13.1%) composing a group of 34.6% of eyes with referable DR. The AI algorithms achieved a mean agreement of 79.4% for referable DR, but the results varied from 49.4% to 92.3%. The mean sensitivity for referable DR was 77.5% (95% CI 69.1-85.8) and specificity 80.6% (95% CI 72.1-89.2). The rate for images ungradable by AI varied from 0% to 28.2% (mean 1.9%). Nineteen out of 21 (90.5%) AI algorithms resulted in grading for DR at least in 98% of the images. CONCLUSIONS Fundus images captured with Optomed Aurora were suitable for DR screening. The performance of the AI algorithms varied considerably emphasizing the need for external validation of screening algorithms in real-world settings before their clinical application.
Collapse
Affiliation(s)
- Anna-Maria Kubin
- Department of Ophthalmology, Oulu University Hospital, Oulu, Finland
- Research Unit of Clinical Medicine, Oulu, Finland
- Medical Research Center, University of Oulu, Oulu, Finland
| | | | - Pasi Ohtonen
- Research Service Unit, Oulu, Finland
- The Research Unit of Surgery, Anesthesia and Intensive Care, Oulu University Hospital and University of Oulu, Oulu, Finland
| | - Antti Keskitalo
- Department of Ophthalmology, Oulu University Hospital, Oulu, Finland
| | - Joonas Wirkkala
- Department of Ophthalmology, Oulu University Hospital, Oulu, Finland
- Research Unit of Clinical Medicine, Oulu, Finland
- Medical Research Center, University of Oulu, Oulu, Finland
| | - Nina Hautala
- Department of Ophthalmology, Oulu University Hospital, Oulu, Finland
- Research Unit of Clinical Medicine, Oulu, Finland
- Medical Research Center, University of Oulu, Oulu, Finland
| |
Collapse
|
2
|
Larsen TJ, Pettersen MB, Nygaard Jensen H, Lynge Pedersen M, Lund-Andersen H, Jørgensen ME, Byberg S. The use of artificial intelligence to assess diabetic eye disease among the Greenlandic population. Int J Circumpolar Health 2024; 83:2314802. [PMID: 38359160 PMCID: PMC10877649 DOI: 10.1080/22423982.2024.2314802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 02/01/2024] [Indexed: 02/17/2024] Open
Abstract
Background: Retina fundus images conducted in Greenland are telemedically assessed for diabetic retinopathy by ophthalmological nurses in Denmark. Applying an AI grading solution, in a Greenlandic setting, could potentially improve the efficiency and cost-effectiveness of DR screening.Method: We developed an AI model using retina fundus photos, performed on persons registered with diabetes in Greenland and Denmark, using Optos® ultra wide-field scanning laser ophthalmoscope, graded according to ICDR.Using the ResNet50 network we compared the model's ability to distinguish between different images of ICDR severity levels in a confusion matrix.Results: Comparing images with ICDR level 0 to images of ICDR level 4 resulted in an accuracy of 0.9655, AUC of 0.9905, sensitivity and specificity of 96.6%.Comparing ICDR levels 0,1,2 with ICDR levels 3,4, we achieved a performance with an accuracy of 0.8077, an AUC of 0.8728, a sensitivity of 84.6% and a specificity of 78.8%. For the other comparisons, we achieved a modest performance.Conclusion: We developed an AI model using Greenlandic data, to automatically detect DR on Optos retina fundus images. The sensitivity and specificity were too low for our model to be applied directly in a clinical setting, thus optimising the model should be prioritised.
Collapse
Affiliation(s)
- Trine Jul Larsen
- Greenland Center of Health Research, Institute of Nursing and Health Science, University of Greenland, Nuuk, Greenland
| | | | | | - Michael Lynge Pedersen
- Greenland Center of Health Research, Institute of Nursing and Health Science, University of Greenland, Nuuk, Greenland
- Rigshospitalet-Glostrup University Hospital, Glostrup, Denmark
| | - Henrik Lund-Andersen
- Clinical Epidemiology, Steno Diabetes Center Copenhagen, Copenhagen, Denmark
- Rigshospitalet-Glostrup University Hospital, Glostrup, Denmark
| | | | - Stine Byberg
- Clinical Epidemiology, Steno Diabetes Center Copenhagen, Copenhagen, Denmark
| |
Collapse
|
3
|
Wu H, Jin K, Yip CC, Koh V, Ye J. A systematic review of economic evaluation of artificial intelligence-based screening for eye diseases: From possibility to reality. Surv Ophthalmol 2024; 69:499-507. [PMID: 38492584 DOI: 10.1016/j.survophthal.2024.03.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 03/04/2024] [Accepted: 03/12/2024] [Indexed: 03/18/2024]
Abstract
Artificial Intelligence (AI) has become a focus of research in the rapidly evolving field of ophthalmology. Nevertheless, there is a lack of systematic studies on the health economics of AI in this field. We examine studies from the PubMed, Google Scholar, and Web of Science databases that employed quantitative analysis, retrieved up to July 2023. Most of the studies indicate that AI leads to cost savings and improved efficiency in ophthalmology. On the other hand, some studies suggest that using AI in healthcare may raise costs for patients, especially when taking into account factors such as labor costs, infrastructure, and patient adherence. Future research should cover a wider range of ophthalmic diseases beyond common eye conditions. Moreover, conducting extensive health economic research, designed to collect data relevant to its own context, is imperative.
Collapse
Affiliation(s)
- Hongkang Wu
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Kai Jin
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Chee Chew Yip
- Department of Ophthalmology & Visual Sciences, Khoo Teck Puat Hospital, Singapore, Singapore
| | - Victor Koh
- Department of Ophthalmology, National University Hospital, National University of Singapore, Singapore
| | - Juan Ye
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang, China.
| |
Collapse
|
4
|
Malerbi FK, Nakayama LF, Melo GB, Stuchi JA, Lencione D, Prado PV, Ribeiro LZ, Dib SA, Regatieri CV. Automated Identification of Different Severity Levels of Diabetic Retinopathy Using a Handheld Fundus Camera and Single-Image Protocol. OPHTHALMOLOGY SCIENCE 2024; 4:100481. [PMID: 38694494 PMCID: PMC11060947 DOI: 10.1016/j.xops.2024.100481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 01/20/2024] [Accepted: 01/25/2024] [Indexed: 05/04/2024]
Abstract
Purpose To evaluate the performance of artificial intelligence (AI) systems embedded in a mobile, handheld retinal camera, with a single retinal image protocol, in detecting both diabetic retinopathy (DR) and more-than-mild diabetic retinopathy (mtmDR). Design Multicenter cross-sectional diagnostic study, conducted at 3 diabetes care and eye care facilities. Participants A total of 327 individuals with diabetes mellitus (type 1 or type 2) underwent a retinal imaging protocol enabling expert reading and automated analysis. Methods Participants underwent fundus photographs using a portable retinal camera (Phelcom Eyer). The captured images were automatically analyzed by deep learning algorithms retinal alteration score (RAS) and diabetic retinopathy alteration score (DRAS), consisting of convolutional neural networks trained on EyePACS data sets and fine-tuned using data sets of portable device fundus images. The ground truth was the classification of DR corresponding to adjudicated expert reading, performed by 3 certified ophthalmologists. Main Outcome Measures Primary outcome measures included the sensitivity and specificity of the AI system in detecting DR and/or mtmDR using a single-field, macula-centered fundus photograph for each eye, compared with a rigorous clinical reference standard comprising the reading center grading of 2-field imaging protocol using the International Classification of Diabetic Retinopathy severity scale. Results Of 327 analyzed patients (mean age, 57.0 ± 16.8 years; mean diabetes duration, 16.3 ± 9.7 years), 307 completed the study protocol. Sensitivity and specificity of the AI system were high in detecting any DR with DRAS (sensitivity, 90.48% [95% confidence interval (CI), 84.99%-94.46%]; specificity, 90.65% [95% CI, 84.54%-94.93%]) and mtmDR with the combination of RAS and DRAS (sensitivity, 90.23% [95% CI, 83.87%-94.69%]; specificity, 85.06% [95% CI, 78.88%-90.00%]). The area under the receiver operating characteristic curve was 0.95 for any DR and 0.89 for mtmDR. Conclusions This study showed a high accuracy for the detection of DR in different levels of severity with a single retinal photo per eye in an all-in-one solution, composed of a portable retinal camera powered by AI. Such a strategy holds great potential for increasing coverage rates of screening programs, contributing to prevention of avoidable blindness. Financial Disclosures F.K.M. is a medical consultant for Phelcom Technologies. J.A.S. is Chief Executive Officer and proprietary of Phelcom Technologies. D.L. is Chief Technology Officer and proprietary of Phelcom Technologies. P.V.P. is an employee at Phelcom Technologies.
Collapse
|
5
|
Zhang Z, Deng C, Paulus YM. Advances in Structural and Functional Retinal Imaging and Biomarkers for Early Detection of Diabetic Retinopathy. Biomedicines 2024; 12:1405. [PMID: 39061979 PMCID: PMC11274328 DOI: 10.3390/biomedicines12071405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 05/27/2024] [Accepted: 06/10/2024] [Indexed: 07/28/2024] Open
Abstract
Diabetic retinopathy (DR), a vision-threatening microvascular complication of diabetes mellitus (DM), is a leading cause of blindness worldwide that requires early detection and intervention. However, diagnosing DR early remains challenging due to the subtle nature of initial pathological changes. This review explores developments in multimodal imaging and functional tests for early DR detection. Where conventional color fundus photography is limited in the field of view and resolution, advanced quantitative analysis of retinal vessel traits such as retinal microvascular caliber, tortuosity, and fractal dimension (FD) can provide additional prognostic value. Optical coherence tomography (OCT) has also emerged as a reliable structural imaging tool for assessing retinal and choroidal neurodegenerative changes, which show potential as early DR biomarkers. Optical coherence tomography angiography (OCTA) enables the evaluation of vascular perfusion and the contours of the foveal avascular zone (FAZ), providing valuable insights into early retinal and choroidal vascular changes. Functional tests, including multifocal electroretinography (mfERG), visual evoked potential (VEP), multifocal pupillographic objective perimetry (mfPOP), microperimetry, and contrast sensitivity (CS), offer complementary data on early functional deficits in DR. More importantly, combining structural and functional imaging data may facilitate earlier detection of DR and targeted management strategies based on disease progression. Artificial intelligence (AI) techniques show promise for automated lesion detection, risk stratification, and biomarker discovery from various imaging data. Additionally, hematological parameters, such as neutrophil-lymphocyte ratio (NLR) and neutrophil extracellular traps (NETs), may be useful in predicting DR risk and progression. Although current methods can detect early DR, there is still a need for further research and development of reliable, cost-effective methods for large-scale screening and monitoring of individuals with DM.
Collapse
Affiliation(s)
- Zhengwei Zhang
- Department of Ophthalmology, Jiangnan University Medical Center, Wuxi 214002, China;
- Department of Ophthalmology, Wuxi No.2 People’s Hospital, Wuxi Clinical College, Nantong University, Wuxi 214002, China
- Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI 48105, USA;
| | - Callie Deng
- Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI 48105, USA;
| | - Yannis M. Paulus
- Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI 48105, USA;
- Department of Biomedical Engineering, University of Michigan, 1000 Wall Street, Ann Arbor, MI 48105, USA
| |
Collapse
|
6
|
Yao J, Lim J, Lim GYS, Ong JCL, Ke Y, Tan TF, Tan TE, Vujosevic S, Ting DSW. Novel artificial intelligence algorithms for diabetic retinopathy and diabetic macular edema. EYE AND VISION (LONDON, ENGLAND) 2024; 11:23. [PMID: 38880890 PMCID: PMC11181581 DOI: 10.1186/s40662-024-00389-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Accepted: 05/09/2024] [Indexed: 06/18/2024]
Abstract
BACKGROUND Diabetic retinopathy (DR) and diabetic macular edema (DME) are major causes of visual impairment that challenge global vision health. New strategies are needed to tackle these growing global health problems, and the integration of artificial intelligence (AI) into ophthalmology has the potential to revolutionize DR and DME management to meet these challenges. MAIN TEXT This review discusses the latest AI-driven methodologies in the context of DR and DME in terms of disease identification, patient-specific disease profiling, and short-term and long-term management. This includes current screening and diagnostic systems and their real-world implementation, lesion detection and analysis, disease progression prediction, and treatment response models. It also highlights the technical advancements that have been made in these areas. Despite these advancements, there are obstacles to the widespread adoption of these technologies in clinical settings, including regulatory and privacy concerns, the need for extensive validation, and integration with existing healthcare systems. We also explore the disparity between the potential of AI models and their actual effectiveness in real-world applications. CONCLUSION AI has the potential to revolutionize the management of DR and DME, offering more efficient and precise tools for healthcare professionals. However, overcoming challenges in deployment, regulatory compliance, and patient privacy is essential for these technologies to realize their full potential. Future research should aim to bridge the gap between technological innovation and clinical application, ensuring AI tools integrate seamlessly into healthcare workflows to enhance patient outcomes.
Collapse
Affiliation(s)
- Jie Yao
- Singapore National Eye Centre, Singapore Eye Research Institute, 11 Third Hospital Avenue, Singapore, 168751, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Joshua Lim
- Singapore National Eye Centre, Singapore Eye Research Institute, 11 Third Hospital Avenue, Singapore, 168751, Singapore
| | - Gilbert Yong San Lim
- Duke-NUS Medical School, Singapore, Singapore
- SingHealth AI Health Program, Singapore, Singapore
| | - Jasmine Chiat Ling Ong
- Duke-NUS Medical School, Singapore, Singapore
- Division of Pharmacy, Singapore General Hospital, Singapore, Singapore
| | - Yuhe Ke
- Department of Anesthesiology and Perioperative Science, Singapore General Hospital, Singapore, Singapore
| | - Ting Fang Tan
- Singapore National Eye Centre, Singapore Eye Research Institute, 11 Third Hospital Avenue, Singapore, 168751, Singapore
| | - Tien-En Tan
- Singapore National Eye Centre, Singapore Eye Research Institute, 11 Third Hospital Avenue, Singapore, 168751, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Stela Vujosevic
- Department of Biomedical, Surgical and Dental Sciences, University of Milan, Milan, Italy
- Eye Clinic, IRCCS MultiMedica, Milan, Italy
| | - Daniel Shu Wei Ting
- Singapore National Eye Centre, Singapore Eye Research Institute, 11 Third Hospital Avenue, Singapore, 168751, Singapore.
- Duke-NUS Medical School, Singapore, Singapore.
- SingHealth AI Health Program, Singapore, Singapore.
| |
Collapse
|
7
|
Mahesh N, Devishamani CS, Raghu K, Mahalingam M, Bysani P, Chakravarthy AV, Raman R. Advancing healthcare: the role and impact of AI and foundation models. Am J Transl Res 2024; 16:2166-2179. [PMID: 39006256 PMCID: PMC11236664 DOI: 10.62347/wqwv9220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2024] [Accepted: 05/06/2024] [Indexed: 07/16/2024]
Abstract
BACKGROUND The integration of artificial intelligence (AI) into the healthcare domain is a monumental shift with profound implications for diagnostics, medical interventions, and the overall structure of healthcare systems. PURPOSE This study explores the transformative journey of foundation AI models in healthcare, shedding light on the challenges, ethical considerations, and vast potential they hold for improving patient outcome and system efficiency. Notably, in this investigation we observe a relatively slow adoption of AI within the public sector of healthcare. The evolution of AI in healthcare is un-paralleled, especially its prowess in revolutionizing diagnostic processes. RESULTS This research showcases how these foundational models can unravel hidden patterns within complex medical datasets. The impact of AI reverberates through medical interventions, encompassing pathology, imaging, genomics, and personalized healthcare, positioning AI as a cornerstone in the quest for precision medicine. The paper delves into the applications of generative AI models in critical facets of healthcare, including decision support, medical imaging, and the prediction of protein structures. The study meticulously evaluates various AI models, such as transfer learning, RNN, autoencoders, and their roles in the healthcare landscape. A pioneering concept introduced in this exploration is that of General Medical AI (GMAI), advocating for the development of reusable and flexible AI models. CONCLUSION The review article discusses how AI can revolutionize healthcare by stressing the significance of transparency, fairness and accountability, in AI applications regarding patient data privacy and biases. By tackling these issues and suggesting a governance structure the article adds to the conversation about AI integration in healthcare environments.
Collapse
Affiliation(s)
- Nandhini Mahesh
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Medical Research Foundation Chennai, Tamil Nadu, India
| | - Chitralekha S Devishamani
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Medical Research Foundation Chennai, Tamil Nadu, India
| | - Keerthana Raghu
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Medical Research Foundation Chennai, Tamil Nadu, India
| | - Maanasi Mahalingam
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Medical Research Foundation Chennai, Tamil Nadu, India
| | - Pragathi Bysani
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Medical Research Foundation Chennai, Tamil Nadu, India
| | | | - Rajiv Raman
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Medical Research Foundation Chennai, Tamil Nadu, India
| |
Collapse
|
8
|
Kong M, Song SJ. Artificial Intelligence Applications in Diabetic Retinopathy: What We Have Now and What to Expect in the Future. Endocrinol Metab (Seoul) 2024; 39:416-424. [PMID: 38853435 PMCID: PMC11220221 DOI: 10.3803/enm.2023.1913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 03/11/2024] [Accepted: 04/03/2024] [Indexed: 06/11/2024] Open
Abstract
Diabetic retinopathy (DR) is a major complication of diabetes mellitus and is a leading cause of vision loss globally. A prompt and accurate diagnosis is crucial for ensuring favorable visual outcomes, highlighting the need for increased access to medical care. The recent remarkable advancements in artificial intelligence (AI) have raised high expectations for its role in disease diagnosis and prognosis prediction across various medical fields. In addition to achieving high precision comparable to that of ophthalmologists, AI-based diagnosis of DR has the potential to improve medical accessibility, especially through telemedicine. In this review paper, we aim to examine the current role of AI in the diagnosis of DR and explore future directions.
Collapse
Affiliation(s)
- Mingui Kong
- Department of Ophthalmology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Korea
| | - Su Jeong Song
- Department of Ophthalmology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Korea
- Biomedical Institute for Convergence (BICS), Sungkyunkwan University, Suwon, Korea
| |
Collapse
|
9
|
Teixeira PF, Battelino T, Carlsson A, Gudbjörnsdottir S, Hannelius U, von Herrath M, Knip M, Korsgren O, Elding Larsson H, Lindqvist A, Ludvigsson J, Lundgren M, Nowak C, Pettersson P, Pociot F, Sundberg F, Åkesson K, Lernmark Å, Forsander G. Assisting the implementation of screening for type 1 diabetes by using artificial intelligence on publicly available data. Diabetologia 2024; 67:985-994. [PMID: 38353727 PMCID: PMC11058797 DOI: 10.1007/s00125-024-06089-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 12/06/2023] [Indexed: 04/30/2024]
Abstract
The type 1 diabetes community is coalescing around the benefits and advantages of early screening for disease risk. To be accepted by healthcare providers, regulatory authorities and payers, screening programmes need to show that the testing variables allow accurate risk prediction and that individualised risk-informed monitoring plans are established, as well as operational feasibility, cost-effectiveness and acceptance at population level. Artificial intelligence (AI) has the potential to contribute to solving these issues, starting with the identification and stratification of at-risk individuals. ASSET (AI for Sustainable Prevention of Autoimmunity in the Society; www.asset.healthcare ) is a public/private consortium that was established to contribute to research around screening for type 1 diabetes and particularly to how AI can drive the implementation of a precision medicine approach to disease prevention. ASSET will additionally focus on issues pertaining to operational implementation of screening. The authors of this article, researchers and clinicians active in the field of type 1 diabetes, met in an open forum to independently debate key issues around screening for type 1 diabetes and to advise ASSET. The potential use of AI in the analysis of longitudinal data from observational cohort studies to inform the design of improved, more individualised screening programmes was also discussed. A key issue was whether AI would allow the research community and industry to capitalise on large publicly available data repositories to design screening programmes that allow the early detection of individuals at high risk and enable clinical evaluation of preventive therapies. Overall, AI has the potential to revolutionise type 1 diabetes screening, in particular to help identify individuals who are at increased risk of disease and aid in the design of appropriate follow-up plans. We hope that this initiative will stimulate further research on this very timely topic.
Collapse
Affiliation(s)
| | - Tadej Battelino
- University Medical Center Ljubljana, University of Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Anneli Carlsson
- Department of Clinical Sciences, Lund University/CRC, Skåne University Hospital, Malmö, Sweden
| | - Soffia Gudbjörnsdottir
- Swedish National Diabetes Register, Centre of Registers, Gothenburg, Sweden
- Department of Molecular and Clinical Medicine, Institute of Medicine, University of Gothenburg, Gothenburg, Sweden
| | | | - Matthias von Herrath
- Global Chief Medical Office, Novo Nordisk, A/S, Søborg, Denmark
- Diabetes Research Institute, University of Miami, Miami, FL, USA
| | - Mikael Knip
- Research Program for Clinical and Molecular Metabolism, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Center for Child Health Research, Tampere University Hospital, Tampere, Finland
| | - Olle Korsgren
- Department of Immunology, Genetics and Pathology, Uppsala University, Uppsala, Sweden
- Department of Clinical Chemistry and Transfusion Medicine, Institute of Biomedicine, University of Gothenburg, Gothenburg, Sweden
| | - Helena Elding Larsson
- Department of Clinical Sciences, Lund University/CRC, Skåne University Hospital, Malmö, Sweden
- Department of Pediatrics, Skåne University Hospital, Malmö, Sweden
| | | | - Johnny Ludvigsson
- Crown Princess Victoria Children's Hospital and Division of Pediatrics, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
| | - Markus Lundgren
- Department of Clinical Sciences Malmö, Lund University, Malmö, Sweden
- Department of Paediatrics, Kristianstad Hospital, Kristianstad, Sweden
| | | | - Paul Pettersson
- Division of Networked and Embedded Systems, Mälardalen University, Västerås, Sweden
- MainlyAI AB, Stockholm, Sweden
| | - Flemming Pociot
- Steno Diabetes Center Copenhagen, Herlev, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Frida Sundberg
- Department of Paediatrics, Institute for Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Queen Silvia Children's Hospital, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Karin Åkesson
- Department of Clinical and Experimental Medicine, Division of Pediatrics and Diabetes Research Center, Linköping University, Linköping, Sweden
- Department of Pediatrics, Ryhov County Hospital, Jönköping, Sweden
| | - Åke Lernmark
- Department of Clinical Sciences, Lund University/CRC, Skåne University Hospital, Malmö, Sweden.
| | - Gun Forsander
- Department of Paediatrics, Institute for Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
- Queen Silvia Children's Hospital, Sahlgrenska University Hospital, Gothenburg, Sweden.
| |
Collapse
|
10
|
Karabeg M, Petrovski G, Hertzberg SN, Erke MG, Fosmark DS, Russell G, Moe MC, Volke V, Raudonis V, Verkauskiene R, Sokolovska J, Haugen IBK, Petrovski BE. A pilot cost-analysis study comparing AI-based EyeArt® and ophthalmologist assessment of diabetic retinopathy in minority women in Oslo, Norway. Int J Retina Vitreous 2024; 10:40. [PMID: 38783384 PMCID: PMC11112837 DOI: 10.1186/s40942-024-00547-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2024] [Accepted: 03/17/2024] [Indexed: 05/25/2024] Open
Abstract
BACKGROUND Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed. PURPOSE To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images. METHODS On Minority Women's Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods. RESULTS 33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P < 0.01), between the human and AI-based EyeArt grading system for detecting DR. The prevalence of DR was 18.6% (95% CI: 11.4-25.8%), and the sensitivity and specificity were 100% (95% CI: 100-100% and 95% CI: 100-100%), respectively. The cost difference for AI screening compared to human screening was $143 lower per patient (cost-saving) in favour of AI. CONCLUSION Our results indicate that The EyeArt AI system is both a reliable, cost-saving, and useful tool for DR grading in clinical practice.
Collapse
Affiliation(s)
- Mia Karabeg
- Center for Eye Research and Innovative Diagnostics, Department of Ophthalmology, Institute for Clinical Medicine, University of Oslo, Kirkeveien 166, 0450, Oslo, Norway
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway
| | - Goran Petrovski
- Center for Eye Research and Innovative Diagnostics, Department of Ophthalmology, Institute for Clinical Medicine, University of Oslo, Kirkeveien 166, 0450, Oslo, Norway
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway
- Department of Ophthalmology, University of Split School of Medicine and University Hospital Centre, 21000, Split, Croatia
- UKLONetwork, University St. Kliment Ohridski-Bitola, 7000, Bitola, Macedonia
| | - Silvia Nw Hertzberg
- Center for Eye Research and Innovative Diagnostics, Department of Ophthalmology, Institute for Clinical Medicine, University of Oslo, Kirkeveien 166, 0450, Oslo, Norway
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway
| | - Maja Gran Erke
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway
| | - Dag Sigurd Fosmark
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway
| | - Greg Russell
- Clinical Development, Eyenuk Inc, Woodland Hills, CA, USA
| | - Morten C Moe
- Center for Eye Research and Innovative Diagnostics, Department of Ophthalmology, Institute for Clinical Medicine, University of Oslo, Kirkeveien 166, 0450, Oslo, Norway
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway
| | - Vallo Volke
- Faculty of Medicine, Tartu University, 50411, Tartu, Estonia
| | - Vidas Raudonis
- Automation Department, Kaunas University of Technology, 51368, Kaunas, Lithuania
| | - Rasa Verkauskiene
- Institute of Endocrinology, Lithuanian University of Health Sciences, 50161, Kaunas, Lithuania
| | | | | | - Beata Eva Petrovski
- Center for Eye Research and Innovative Diagnostics, Department of Ophthalmology, Institute for Clinical Medicine, University of Oslo, Kirkeveien 166, 0450, Oslo, Norway.
- Department of Ophthalmology, Oslo University Hospital, Kirkeveien 166, 0450, Oslo, Norway.
- Institute of Endocrinology, Lithuanian University of Health Sciences, 50161, Kaunas, Lithuania.
| |
Collapse
|
11
|
Elvidge J, Hawksworth C, Avşar TS, Zemplenyi A, Chalkidou A, Petrou S, Petykó Z, Srivastava D, Chandra G, Delaye J, Denniston A, Gomes M, Knies S, Nousios P, Siirtola P, Wang J, Dawoud D. Consolidated Health Economic Evaluation Reporting Standards for Interventions That Use Artificial Intelligence (CHEERS-AI). VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024:S1098-3015(24)02366-0. [PMID: 38795956 DOI: 10.1016/j.jval.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Revised: 04/22/2024] [Accepted: 05/04/2024] [Indexed: 05/28/2024]
Abstract
OBJECTIVES Economic evaluations (EEs) are commonly used by decision makers to understand the value of health interventions. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022) provide reporting guidelines for EEs. Healthcare systems will increasingly see new interventions that use artificial intelligence (AI) to perform their function. We developed Consolidated Health Economic Evaluation Reporting Standards for Interventions that use AI (CHEERS-AI) to ensure EEs of AI-based health interventions are reported in a transparent and reproducible manner. METHODS Potential CHEERS-AI reporting items were informed by 2 published systematic literature reviews of EEs and a contemporary update. A Delphi study was conducted using 3 survey rounds to elicit multidisciplinary expert views on 26 potential items, through a 9-point Likert rating scale and qualitative comments. An online consensus meeting was held to finalize outstanding reporting items. A digital health patient group reviewed the final checklist from a patient perspective. RESULTS A total of 58 participants responded to survey round 1, 42, and 31 of whom responded to rounds 2 and 3, respectively. Nine participants joined the consensus meeting. Ultimately, 38 reporting items were included in CHEERS-AI. They comprised the 28 original CHEERS 2022 items, plus 10 new AI-specific reporting items. Additionally, 8 of the original CHEERS 2022 items were elaborated on to ensure AI-specific nuance is reported. CONCLUSIONS CHEERS-AI should be used when reporting an EE of an intervention that uses AI to perform its function. CHEERS-AI will help decision makers and reviewers to understand important AI-specific details of an intervention, and any implications for the EE methods used and cost-effectiveness conclusions.
Collapse
Affiliation(s)
- Jamie Elvidge
- National Institute for Health and Care Excellence (NICE), England, UK.
| | - Claire Hawksworth
- National Institute for Health and Care Excellence (NICE), England, UK
| | - Tuba Saygın Avşar
- National Institute for Health and Care Excellence (NICE), England, UK
| | - Antal Zemplenyi
- Center for Health Technology Assessment and Pharmacoeconomic Research, University of Pécs, Pécs, Hungary; University of Colorado Anschutz Medical Campus, Denver, CO, USA; Syreon Research Institute, Budapest, Hungary
| | | | - Stavros Petrou
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, England, UK
| | | | - Divya Srivastava
- Department of Health Policy, London School of Economics and Political Science, London, England, UK
| | - Gunjan Chandra
- Biomimetics and Intelligent Systems Group, University of Oulu, Oulu, Finland
| | | | - Alastair Denniston
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, England, UK
| | - Manuel Gomes
- Department of Primary Care and Population Health, University College London, England, UK
| | - Saskia Knies
- National Healthcare Institute (ZIN), Diemen, The Netherlands
| | - Petros Nousios
- Dental and Pharmaceutical Benefits Agency (TLV), Stockholm, Sweden
| | - Pekka Siirtola
- Biomimetics and Intelligent Systems Group, University of Oulu, Oulu, Finland
| | - Junfeng Wang
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University, Utrecht, The Netherlands
| | - Dalia Dawoud
- National Institute for Health and Care Excellence (NICE), England, UK
| |
Collapse
|
12
|
Wu X, Wu Y, Tu Z, Cao Z, Xu M, Xiang Y, Lin D, Jin L, Zhao L, Zhang Y, Liu Y, Yan P, Hu W, Liu J, Liu L, Wang X, Wang R, Chen J, Xiao W, Shang Y, Xie P, Wang D, Zhang X, Dongye M, Wang C, Ting DSW, Liu Y, Pan R, Lin H. Cost-effectiveness and cost-utility of a digital technology-driven hierarchical healthcare screening pattern in China. Nat Commun 2024; 15:3650. [PMID: 38688925 PMCID: PMC11061155 DOI: 10.1038/s41467-024-47211-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 03/25/2024] [Indexed: 05/02/2024] Open
Abstract
Utilization of digital technologies for cataract screening in primary care is a potential solution for addressing the dilemma between the growing aging population and unequally distributed resources. Here, we propose a digital technology-driven hierarchical screening (DH screening) pattern implemented in China to promote the equity and accessibility of healthcare. It consists of home-based mobile artificial intelligence (AI) screening, community-based AI diagnosis, and referral to hospitals. We utilize decision-analytic Markov models to evaluate the cost-effectiveness and cost-utility of different cataract screening strategies (no screening, telescreening, AI screening and DH screening). A simulated cohort of 100,000 individuals from age 50 is built through a total of 30 1-year Markov cycles. The primary outcomes are incremental cost-effectiveness ratio and incremental cost-utility ratio. The results show that DH screening dominates no screening, telescreening and AI screening in urban and rural China. Annual DH screening emerges as the most economically effective strategy with 341 (338 to 344) and 1326 (1312 to 1340) years of blindness avoided compared with telescreening, and 37 (35 to 39) and 140 (131 to 148) years compared with AI screening in urban and rural settings, respectively. The findings remain robust across all sensitivity analyses conducted. Here, we report that DH screening is cost-effective in urban and rural China, and the annual screening proves to be the most cost-effective option, providing an economic rationale for policymakers promoting public eye health in low- and middle-income countries.
Collapse
Affiliation(s)
- Xiaohang Wu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yuxuan Wu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Zhenjun Tu
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Zizheng Cao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Miaohong Xu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yifan Xiang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Duoru Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Ling Jin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Lanqin Zhao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yingzhe Zhang
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA
| | - Yu Liu
- School of Public Health and Management, Guangzhou University of Chinese Medicine, Guangzhou, Guangdong, China
| | - Pisong Yan
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Weiling Hu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Jiali Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Lixue Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Xun Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Ruixin Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Jieying Chen
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Wei Xiao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Yuanjun Shang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Peichen Xie
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Dongni Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Xulin Zhang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Meimei Dongye
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Chenxinqi Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China
| | - Daniel Shu Wei Ting
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-National University of Singapore Medical School, Singapore, Singapore
| | - Yizhi Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China.
| | - Rong Pan
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China.
| | - Haotian Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China.
- Hainan Eye Hospital and Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Haikou, Hainan, China.
- Center for Precision Medicine and Department of Genetics and Biomedical Informatics, Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China.
| |
Collapse
|
13
|
Zhang J, Lin S, Cheng T, Xu Y, Lu L, He J, Yu T, Peng Y, Zhang Y, Zou H, Ma Y. RETFound-enhanced community-based fundus disease screening: real-world evidence and decision curve analysis. NPJ Digit Med 2024; 7:108. [PMID: 38693205 PMCID: PMC11063045 DOI: 10.1038/s41746-024-01109-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2023] [Accepted: 04/12/2024] [Indexed: 05/03/2024] Open
Abstract
Visual impairments and blindness are major public health concerns globally. Effective eye disease screening aided by artificial intelligence (AI) is a promising countermeasure, although it is challenged by practical constraints such as poor image quality in community screening. The recently developed ophthalmic foundation model RETFound has shown higher accuracy in retinal image recognition tasks. This study developed an RETFound-enhanced deep learning (DL) model for multiple-eye disease screening using real-world images from community screenings. Our results revealed that our DL model improved the sensitivity and specificity by over 15% compared with commercial models. Our model also shows better generalisation ability than AI models developed using traditional processes. Additionally, decision curve analysis underscores the higher net benefit of employing our model in both urban and rural settings in China. These findings indicate that the RETFound-enhanced DL model can achieve a higher net benefit in community-based screening, advocating its adoption in low- and middle-income countries to address global eye health challenges.
Collapse
Affiliation(s)
- Juzhao Zhang
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Disease, Shanghai, China
- Shanghai Engineering Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Senlin Lin
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Disease, Shanghai, China
- Shanghai Engineering Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Tianhao Cheng
- School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China
| | - Yi Xu
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Disease, Shanghai, China
- Shanghai Engineering Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Lina Lu
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Disease, Shanghai, China
- Shanghai Engineering Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Jiangnan He
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Tao Yu
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yajun Peng
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yuejie Zhang
- School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China.
| | - Haidong Zou
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China.
- National Clinical Research Center for Eye Disease, Shanghai, China.
- Shanghai Engineering Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China.
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Yingyan Ma
- Shanghai Eye Disease Prevention & Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China.
- National Clinical Research Center for Eye Disease, Shanghai, China.
- Shanghai Engineering Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China.
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| |
Collapse
|
14
|
Lin S, Ma Y, Jiang Y, Li W, Peng Y, Yu T, Xu Y, Zhu J, Lu L, Zou H. Service Quality and Residents' Preferences for Facilitated Self-Service Fundus Disease Screening: Cross-Sectional Study. J Med Internet Res 2024; 26:e45545. [PMID: 38630535 PMCID: PMC11063888 DOI: 10.2196/45545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Revised: 10/15/2023] [Accepted: 03/12/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Fundus photography is the most important examination in eye disease screening. A facilitated self-service eye screening pattern based on the fully automatic fundus camera was developed in 2022 in Shanghai, China; it may help solve the problem of insufficient human resources in primary health care institutions. However, the service quality and residents' preference for this new pattern are unclear. OBJECTIVE This study aimed to compare the service quality and residents' preferences between facilitated self-service eye screening and traditional manual screening and to explore the relationships between the screening service's quality and residents' preferences. METHODS We conducted a cross-sectional study in Shanghai, China. Residents who underwent facilitated self-service fundus disease screening at one of the screening sites were assigned to the exposure group; those who were screened with a traditional fundus camera operated by an optometrist at an adjacent site comprised the control group. The primary outcome was the screening service quality, including effectiveness (image quality and screening efficiency), physiological discomfort, safety, convenience, and trustworthiness. The secondary outcome was the participants' preferences. Differences in service quality and the participants' preferences between the 2 groups were compared using chi-square tests separately. Subgroup analyses for exploring the relationships between the screening service's quality and residents' preference were conducted using generalized logit models. RESULTS A total of 358 residents enrolled; among them, 176 (49.16%) were included in the exposure group and the remaining 182 (50.84%) in the control group. Residents' basic characteristics were balanced between the 2 groups. There was no significant difference in service quality between the 2 groups (image quality pass rate: P=.79; average screening time: P=.57; no physiological discomfort rate: P=.92; safety rate: P=.78; convenience rate: P=.95; trustworthiness rate: P=.20). However, the proportion of participants who were willing to use the same technology for their next screening was significantly lower in the exposure group than in the control group (P<.001). Subgroup analyses suggest that distrust in the facilitated self-service eye screening might increase the probability of refusal to undergo screening (P=.02). CONCLUSIONS This study confirms that the facilitated self-service fundus disease screening pattern could achieve good service quality. However, it was difficult to reverse residents' preferences for manual screening in a short period, especially when the original manual service was already excellent. Therefore, the digital transformation of health care must be cautious. We suggest that attention be paid to the residents' individual needs. More efficient man-machine collaboration and personalized health management solutions based on large language models are both needed.
Collapse
Affiliation(s)
- Senlin Lin
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Yingyan Ma
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
- Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Yanwei Jiang
- Shanghai Hongkou Center for Disease Control and Prevention, Shanghai, China
| | - Wenwen Li
- School of Management, Fudan University, Shanghai, China
| | - Yajun Peng
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Tao Yu
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Yi Xu
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Jianfeng Zhu
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Lina Lu
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
| | - Haidong Zou
- Shanghai Eye Diseases Prevention &Treatment Center/ Shanghai Eye Hospital, School of Medicine, Tongji University, Shanghai, China
- National Clinical Research Center for Eye Diseases, Shanghai, China
- Shanghai Engineering Research Center of Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China
- Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
15
|
Aravindhan A, Fenwick EK, Chan AWD, Man REK, Tan NC, Wong WT, Soo WF, Lim SW, Wee SYM, Sabanayagam C, Finkelstein E, Tan G, Hamzah H, Chakraborty B, Acharyya S, Shyong TE, Scanlon P, Wong TY, Lamoureux EL. Extending the diabetic retinopathy screening intervals in Singapore: methodology and preliminary findings of a cohort study. BMC Public Health 2024; 24:786. [PMID: 38481239 PMCID: PMC10935797 DOI: 10.1186/s12889-024-18287-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 03/05/2024] [Indexed: 03/17/2024] Open
Abstract
BACKGROUND The Diabetic Retinopathy Extended Screening Study (DRESS) aims to develop and validate a new DR/diabetic macular edema (DME) risk stratification model in patients with Type 2 diabetes (DM) to identify low-risk groups who can be safely assigned to biennial or triennial screening intervals. We describe the study methodology, participants' baseline characteristics, and preliminary DR progression rates at the first annual follow-up. METHODS DRESS is a 3-year ongoing longitudinal study of patients with T2DM and no or mild non-proliferative DR (NPDR, non-referable) who underwent teleophthalmic screening under the Singapore integrated Diabetic Retinopathy Programme (SiDRP) at four SingHealth Polyclinics. Patients with referable DR/DME (> mild NPDR) or ungradable fundus images were excluded. Sociodemographic, lifestyle, medical and clinical information was obtained from medical records and interviewer-administered questionnaires at baseline. These data are extracted from medical records at 12, 24 and 36 months post-enrollment. Baseline descriptive characteristics stratified by DR severity at baseline and rates of progression to referable DR at 12-month follow-up were calculated. RESULTS Of 5,840 eligible patients, 78.3% (n = 4,570, median [interquartile range [IQR] age 61.0 [55-67] years; 54.7% male; 68.0% Chinese) completed the baseline assessment. At baseline, 97.4% and 2.6% had none and mild NPDR (worse eye), respectively. Most participants had hypertension (79.2%) and dyslipidemia (92.8%); and almost half were obese (43.4%, BMI ≥ 27.5 kg/m2). Participants without DR (vs mild DR) reported shorter DM duration, and had lower haemoglobin A1c, triglycerides and urine albumin/creatinine ratio (all p < 0.05). To date, we have extracted 41.8% (n = 1909) of the 12-month follow-up data. Of these, 99.7% (n = 1,904) did not progress to referable DR. Those who progressed to referable DR status (0.3%) had no DR at baseline. CONCLUSIONS In our prospective study of patients with T2DM and non-referable DR attending polyclinics, we found extremely low annual DR progression rates. These preliminary results suggest that extending screening intervals beyond 12 months may be viable and safe for most participants, although our 3-year follow up data are needed to substantiate this claim and develop the risk stratification model to identify low-risk patients with T2DM who can be assigned biennial or triennial screening intervals.
Collapse
Affiliation(s)
- Amudha Aravindhan
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Eva K Fenwick
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Aurora Wing Dan Chan
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
| | - Ryan Eyn Kidd Man
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | | | | | | | | | | | - Charumathi Sabanayagam
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | | | - Gavin Tan
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Haslina Hamzah
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
| | | | | | - Tai E Shyong
- Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore, Singapore
| | - Peter Scanlon
- Gloucestershire Hospitals NHS Foundation Trust, Gloucester, UK
| | | | - Ecosse L Lamoureux
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore.
- Duke-NUS Medical School, Singapore, Singapore.
- The University of Melbourne, Melbourne, Australia.
| |
Collapse
|
16
|
Ramoutar RR. An Economic Analysis for the Use of Artificial Intelligence in Screening for Diabetic Retinopathy in Trinidad and Tobago. Cureus 2024; 16:e55745. [PMID: 38586698 PMCID: PMC10999161 DOI: 10.7759/cureus.55745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/07/2024] [Indexed: 04/09/2024] Open
Abstract
This is a systematic review of 25 publications on the topics of the prevalence and cost of diabetic retinopathy (DR) in Trinidad and Tobago, the cost of traditional methods of screening for DR, and the use and cost of artificial intelligence (AI) in screening for DR. Analysis of these publications was used to identify and make estimates for how resources allocated to ophthalmology in public health systems in Trinidad and Tobago can be more efficiently utilized by employing AI in diagnosing treatable DR. DR screening was found to be an effective method of detecting the disease. Screening was found to be a universally cost-effective method of disease prevention and for altering the natural history of the disease in the spectrum of low-middle to high-income economies, such as Rwanda, Thailand, China, South Korea, and Singapore. AI and deep learning systems were found to be clinically superior to, or as effective as, human graders in areas where they were deployed, indicating that the systems are clinically safe. They have been shown to improve access to diabetic retinal screening, improve compliance with screening appointments, and prove to be cost-effective, especially in rural areas. Trinidad and Tobago, which is estimated to be disproportionately more affected by the burden of DR when projected out to the mid-21st century, stands to save as much as US$60 million annually from the implementation of an AI-based system to screen for DR versus conventional manual grading.
Collapse
Affiliation(s)
- Ryan R Ramoutar
- Ophthalmology, University Hospitals of Leicester NHS Trust, Leicester, GBR
| |
Collapse
|
17
|
Posner KM, Bakus C, Basralian G, Chester G, Zeiman M, O'Malley GR, Klein GR. Evaluating ChatGPT's Capabilities on Orthopedic Training Examinations: An Analysis of New Image Processing Features. Cureus 2024; 16:e55945. [PMID: 38601421 PMCID: PMC11005479 DOI: 10.7759/cureus.55945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/11/2024] [Indexed: 04/12/2024] Open
Abstract
Introduction The efficacy of integrating artificial intelligence (AI) models like ChatGPT into the medical field, specifically orthopedic surgery, has yet to be fully determined. The most recent adaptation of ChatGPT that has yet to be explored is its image analysis capabilities. This study assesses ChatGPT's performance in answering Orthopedic In-Training Examination (OITE) questions, including those that require image analysis. Methods Questions from the 2014, 2015, 2021, and 2022 AAOS OITE were screened for inclusion. All questions without images were entered into ChatGPT 3.5 and 4.0 twice. Questions that necessitated the use of images were only entered into ChatGPT 4.0 twice, as this is the only version of the system that can analyze images. The responses were recorded and compared to AAOS's correct answers, evaluating the AI's accuracy and precision. Results A total of 940 questions were included in the final analysis (457 questions with images and 483 questions without images). ChatGPT 4.0 performed significantly better on questions that did not require image analysis (67.81% vs 47.59%, p<0.001). Discussion While the use of AI in orthopedics is an intriguing possibility, this evaluation demonstrates how, even with the addition of image processing capabilities, ChatGPT still falls short in terms of its accuracy. As AI technology evolves, ongoing research is vital to harness AI's potential effectively, ensuring it complements rather than attempts to replace the nuanced skills of orthopedic surgeons.
Collapse
Affiliation(s)
- Kevin M Posner
- Department of Orthopedic Surgery, Hackensack Meridian School of Medicine, Nutley, USA
| | - Cassandra Bakus
- Department of Orthopedic Surgery, Hackensack Meridian School of Medicine, Nutley, USA
| | - Grace Basralian
- Department of Orthopedic Surgery, Hackensack Meridian School of Medicine, Nutley, USA
| | - Grace Chester
- Department of Orthopedic Surgery, Hackensack Meridian School of Medicine, Nutley, USA
| | - Mallery Zeiman
- Department of Orthopedic Surgery, Hackensack Meridian School of Medicine, Nutley, USA
| | - Geoffrey R O'Malley
- Department of Orthopedic Surgery, Hackensack University Medical Center, Hackensack, USA
| | - Gregg R Klein
- Department of Orthopedic Surgery, Hackensack University Medical Center, Hackensack, USA
| |
Collapse
|
18
|
Wang Y, Liu C, Hu W, Luo L, Shi D, Zhang J, Yin Q, Zhang L, Han X, He M. Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case. NPJ Digit Med 2024; 7:43. [PMID: 38383738 PMCID: PMC10881978 DOI: 10.1038/s41746-024-01032-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 02/05/2024] [Indexed: 02/23/2024] Open
Abstract
Artificial intelligence (AI) models have shown great accuracy in health screening. However, for real-world implementation, high accuracy may not guarantee cost-effectiveness. Improving AI's sensitivity finds more high-risk patients but may raise medical costs while increasing specificity reduces unnecessary referrals but may weaken detection capability. To evaluate the trade-off between AI model performance and the long-running cost-effectiveness, we conducted a cost-effectiveness analysis in a nationwide diabetic retinopathy (DR) screening program in China, comprising 251,535 participants with diabetes over 30 years. We tested a validated AI model in 1100 different diagnostic performances (presented as sensitivity/specificity pairs) and modeled annual screening scenarios. The status quo was defined as the scenario with the most accurate AI performance. The incremental cost-effectiveness ratio (ICER) was calculated for other scenarios against the status quo as cost-effectiveness metrics. Compared to the status quo (sensitivity/specificity: 93.3%/87.7%), six scenarios were cost-saving and seven were cost-effective. To achieve cost-saving or cost-effective, the AI model should reach a minimum sensitivity of 88.2% and specificity of 80.4%. The most cost-effective AI model exhibited higher sensitivity (96.3%) and lower specificity (80.4%) than the status quo. In settings with higher DR prevalence and willingness-to-pay levels, the AI needed higher sensitivity for optimal cost-effectiveness. Urban regions and younger patient groups also required higher sensitivity in AI-based screening. In real-world DR screening, the most accurate AI model may not be the most cost-effective. Cost-effectiveness should be independently evaluated, which is most likely to be affected by the AI's sensitivity.
Collapse
Affiliation(s)
- Yueye Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Chi Liu
- Faculty of Data Science, City University of Macau, Macao SAR, China
| | - Wenyi Hu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, VIC, Australia
| | - Lixia Luo
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Danli Shi
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Jian Zhang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Qiuxia Yin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Lei Zhang
- Clinical Medical Research Center, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu, 210008, China.
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia.
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia.
| | - Xiaotong Han
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
| | - Mingguang He
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, Hong Kong.
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, Hong Kong.
- Centre for Eye and Vision Research (CEVR), 17W Hong Kong Science Park, Shatin, Hong Kong.
| |
Collapse
|
19
|
Chia MA, Hersch F, Sayres R, Bavishi P, Tiwari R, Keane PA, Turner AW. Validation of a deep learning system for the detection of diabetic retinopathy in Indigenous Australians. Br J Ophthalmol 2024; 108:268-273. [PMID: 36746615 PMCID: PMC10850716 DOI: 10.1136/bjo-2022-322237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 12/31/2022] [Indexed: 02/08/2023]
Abstract
BACKGROUND/AIMS Deep learning systems (DLSs) for diabetic retinopathy (DR) detection show promising results but can underperform in racial and ethnic minority groups, therefore external validation within these populations is critical for health equity. This study evaluates the performance of a DLS for DR detection among Indigenous Australians, an understudied ethnic group who suffer disproportionately from DR-related blindness. METHODS We performed a retrospective external validation study comparing the performance of a DLS against a retinal specialist for the detection of more-than-mild DR (mtmDR), vision-threatening DR (vtDR) and all-cause referable DR. The validation set consisted of 1682 consecutive, single-field, macula-centred retinal photographs from 864 patients with diabetes (mean age 54.9 years, 52.4% women) at an Indigenous primary care service in Perth, Australia. Three-person adjudication by a panel of specialists served as the reference standard. RESULTS For mtmDR detection, sensitivity of the DLS was superior to the retina specialist (98.0% (95% CI, 96.5 to 99.4) vs 87.1% (95% CI, 83.6 to 90.6), McNemar's test p<0.001) with a small reduction in specificity (95.1% (95% CI, 93.6 to 96.4) vs 97.0% (95% CI, 95.9 to 98.0), p=0.006). For vtDR, the DLS's sensitivity was again superior to the human grader (96.2% (95% CI, 93.4 to 98.6) vs 84.4% (95% CI, 79.7 to 89.2), p<0.001) with a slight drop in specificity (95.8% (95% CI, 94.6 to 96.9) vs 97.8% (95% CI, 96.9 to 98.6), p=0.002). For all-cause referable DR, there was a substantial increase in sensitivity (93.7% (95% CI, 91.8 to 95.5) vs 74.4% (95% CI, 71.1 to 77.5), p<0.001) and a smaller reduction in specificity (91.7% (95% CI, 90.0 to 93.3) vs 96.3% (95% CI, 95.2 to 97.4), p<0.001). CONCLUSION The DLS showed improved sensitivity and similar specificity compared with a retina specialist for DR detection. This demonstrates its potential to support DR screening among Indigenous Australians, an underserved population with a high burden of diabetic eye disease.
Collapse
Affiliation(s)
- Mark A Chia
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Lions Outback Vision, Lions Eye Institute, Nedlands, Western Australia, Australia
| | | | | | | | | | - Pearse A Keane
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Angus W Turner
- Lions Outback Vision, Lions Eye Institute, Nedlands, Western Australia, Australia
- Centre for Ophthalmology and Visual Science, The University of Western Australia, Nedlands, Western Australia, Australia
| |
Collapse
|
20
|
Ong JCL, Seng BJJ, Law JZF, Low LL, Kwa ALH, Giacomini KM, Ting DSW. Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions. Cell Rep Med 2024; 5:101356. [PMID: 38232690 PMCID: PMC10829781 DOI: 10.1016/j.xcrm.2023.101356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 10/12/2023] [Accepted: 12/10/2023] [Indexed: 01/19/2024]
Abstract
This perspective highlights the importance of addressing social determinants of health (SDOH) in patient health outcomes and health inequity, a global problem exacerbated by the COVID-19 pandemic. We provide a broad discussion on current developments in digital health and artificial intelligence (AI), including large language models (LLMs), as transformative tools in addressing SDOH factors, offering new capabilities for disease surveillance and patient care. Simultaneously, we bring attention to challenges, such as data standardization, infrastructure limitations, digital literacy, and algorithmic bias, that could hinder equitable access to AI benefits. For LLMs, we highlight potential unique challenges and risks including environmental impact, unfair labor practices, inadvertent disinformation or "hallucinations," proliferation of bias, and infringement of copyrights. We propose the need for a multitiered approach to digital inclusion as an SDOH and the development of ethical and responsible AI practice frameworks globally and provide suggestions on bridging the gap from development to implementation of equitable AI technologies.
Collapse
Affiliation(s)
- Jasmine Chiat Ling Ong
- Division of Pharmacy, Singapore General Hospital, Singapore, Singapore; SingHealth Duke-NUS Medicine Academic Clinical Programme, Singapore, Singapore
| | - Benjamin Jun Jie Seng
- MOHH Holdings (Singapore) Pte., Ltd., Singapore, Singapore; SingHealth Duke-NUS Family Medicine Academic Clinical Programme, Singapore, Singapore
| | | | - Lian Leng Low
- SingHealth Duke-NUS Family Medicine Academic Clinical Programme, Singapore, Singapore; Population Health and Integrated Care Office, Singapore General Hospital, Singapore, Singapore; Centre for Population Health Research and Implementation, SingHealth Regional Health System, Singapore, Singapore; Outram Community Hospital, SingHealth Community Hospitals, Singapore, Singapore
| | - Andrea Lay Hoon Kwa
- Division of Pharmacy, Singapore General Hospital, Singapore, Singapore; SingHealth Duke-NUS Medicine Academic Clinical Programme, Singapore, Singapore; Emerging Infectious Diseases, Duke-NUS Medical School, Singapore, Singapore
| | - Kathleen M Giacomini
- Department of Bioengineering and Therapeutic Sciences, Schools of Pharmacy and Medicine, University of California, San Francisco, San Francisco, CA, USA
| | - Daniel Shu Wei Ting
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research, Singapore, Singapore; Duke-NUS Medical School, National University of Singapore, Singapore, Singapore; Byers Eye Institute, Stanford University, Stanford, CA, USA.
| |
Collapse
|
21
|
Hu W, Joseph S, Li R, Woods E, Sun J, Shen M, Jan CL, Zhu Z, He M, Zhang L. Population impact and cost-effectiveness of artificial intelligence-based diabetic retinopathy screening in people living with diabetes in Australia: a cost effectiveness analysis. EClinicalMedicine 2024; 67:102387. [PMID: 38314061 PMCID: PMC10837545 DOI: 10.1016/j.eclinm.2023.102387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 11/29/2023] [Accepted: 12/05/2023] [Indexed: 02/06/2024] Open
Abstract
Background We aimed to evaluate the cost-effectiveness of an artificial intelligence-(AI) based diabetic retinopathy (DR) screening system in the primary care setting for both non-Indigenous and Indigenous people living with diabetes in Australia. Methods We performed a cost-effectiveness analysis between January 01, 2022 and August 01, 2023. A decision-analytic Markov model was constructed to simulate DR progression in a population of 1,197,818 non-Indigenous and 65,160 Indigenous Australians living with diabetes aged ≥20 years over 40 years. From a healthcare provider's perspective, we compared current practice to three primary care AI-based screening scenarios-(A) substitution of current manual grading, (B) scaling up to patient acceptance level, and (C) achieving universal screening. Study results were presented as incremental cost-effectiveness ratio (ICER), benefit-cost ratio (BCR), and net monetary benefits (NMB). A Willingness-to-pay (WTP) threshold of AU$50,000 per quality-adjusted life year (QALY) and a discount rate of 3.5% were adopted in this study. Findings With the status quo, the non-Indigenous diabetic population was projected to develop 96,269 blindness cases, resulting in AU$13,039.6 m spending on DR screening and treatment during 2020-2060. In comparison, all three intervention scenarios were effective and cost-saving. In particular, if a universal screening program was to be implemented (Scenario C), it would prevent 38,347 blindness cases, gain 172,090 QALYs and save AU$595.8 m, leading to a BCR of 3.96 and NMB of AU$9,200 m. Similar findings were also reported in the Indigenous population. With the status quo, 3,396 Indigenous individuals would develop blindness, which would cost the health system AU$796.0 m during 2020-2060. All three intervention scenarios were cost-saving for the Indigenous population. Notably, universal AI-based DR screening (Scenario C) would prevent 1,211 blindness cases and gain 9,800 QALYs in the Indigenous population, leading to a saving of AU$19.2 m with a BCR of 1.62 and NMB of AU$509 m. Interpretation Our findings suggest that implementing AI-based DR screening in primary care is highly effective and cost-saving in both Indigenous and non-Indigenous populations. Funding This project received grant funding from the Australian Government: the National Critical Research Infrastructure Initiative, Medical Research Future Fund (MRFAI00035) and the NHMRC Investigator Grant (APP1175405). The contents of the published material are solely the responsibility of the Administering Institution, a participating institution or individual authors and do not reflect the views of the NHMRC. This work was supported by the Global STEM Professorship Scheme (P0046113), the Fundamental Research Funds of the State Key Laboratory of Ophthalmology, Project of Investigation on Health Status of Employees in Financial Industry in Guangzhou, China (Z012014075). The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian State Government. W.H. is supported by the Melbourne Research Scholarship established by the University of Melbourne. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Collapse
Affiliation(s)
- Wenyi Hu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Sanil Joseph
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Rui Li
- Central Clinical School, Faculty of Medicine, Monash University, Melbourne, VIC, Australia
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia
- China-Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi’an Jiaotong University Health Science Center, Xi’an, Shaanxi, 710061, PR China
| | - Ekaterina Woods
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Jason Sun
- Eyetelligence Pty Ltd., Melbourne, Australia
| | - Mingwang Shen
- China-Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi’an Jiaotong University Health Science Center, Xi’an, Shaanxi, 710061, PR China
| | - Catherine Lingxue Jan
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Zhuoting Zhu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Mingguang He
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
- School of Optometry, The Hong Kong Polytechnic University, Hong Kong, China
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Lei Zhang
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Clinical Medical Research Center, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu Province 210008, China
- Central Clinical School, Faculty of Medicine, Monash University, Melbourne, VIC, Australia
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia
| |
Collapse
|
22
|
Wong TY, Tan TE. The Diabetic Retinopathy "Pandemic" and Evolving Global Strategies: The 2023 Friedenwald Lecture. Invest Ophthalmol Vis Sci 2023; 64:47. [PMID: 38153754 PMCID: PMC10756246 DOI: 10.1167/iovs.64.15.47] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 07/30/2023] [Indexed: 12/29/2023] Open
Affiliation(s)
- Tien Yin Wong
- Singapore Eye Research Institute, Singapore, Singapore National Eye Centre, Singapore
- Duke-National University of Singapore, Singapore
- Tsinghua Medicine, Tsinghua University, Beijing, China
| | - Tien-En Tan
- Singapore Eye Research Institute, Singapore, Singapore National Eye Centre, Singapore
- Duke-National University of Singapore, Singapore
| |
Collapse
|
23
|
Dow ER, Khan NC, Chen KM, Mishra K, Perera C, Narala R, Basina M, Dang J, Kim M, Levine M, Phadke A, Tan M, Weng K, Do DV, Moshfeghi DM, Mahajan VB, Mruthyunjaya P, Leng T, Myung D. AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy. OPHTHALMOLOGY SCIENCE 2023; 3:100330. [PMID: 37449051 PMCID: PMC10336195 DOI: 10.1016/j.xops.2023.100330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Revised: 05/04/2023] [Accepted: 05/08/2023] [Indexed: 07/18/2023]
Abstract
Objective Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases. Design Prospective cohort study and retrospective analysis. Participants Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic. Methods Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist. Main Outcome Measures Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists. Results The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%-100%) but lower specificity (60.3% specificity; 95% CI, 47.7%-72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%-88.3%; 96.9% specificity; 95% CI, 93.5%-100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%-100%) and a specificity of 98.2% (95% CI, 94.6%-100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters. Conclusions Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy. Financial Disclosures Proprietary or commercial disclosure may be found after the references.
Collapse
Affiliation(s)
- Eliot R. Dow
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
- Veterans Affairs Palo Alto Health Care System, Palo Alto, California
| | - Nergis C. Khan
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Karen M. Chen
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Kapil Mishra
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Chandrashan Perera
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Ramsudha Narala
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Marina Basina
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Jimmy Dang
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Michael Kim
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Marcie Levine
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Anuradha Phadke
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Marilyn Tan
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Kirsti Weng
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Diana V. Do
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Darius M. Moshfeghi
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Vinit B. Mahajan
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
- Veterans Affairs Palo Alto Health Care System, Palo Alto, California
| | - Prithvi Mruthyunjaya
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Theodore Leng
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - David Myung
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| |
Collapse
|
24
|
Zang B, Rong S, Li D, Ding X, Zang D, Wang F, Liang Y, Zhai G, Feng K, Zhou Z, Wang Y. Undiagnosed diabetic retinopathy in Northeast China: prevalence and determinants. Front Endocrinol (Lausanne) 2023; 14:1263508. [PMID: 38093961 PMCID: PMC10716530 DOI: 10.3389/fendo.2023.1263508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Accepted: 11/13/2023] [Indexed: 12/18/2023] Open
Abstract
Objective To report the prevalence and contributing factors of undiagnosed diabetic retinopathy (DR) in a population from Northeastern China. Subjects/Methods A total of 800 subjects from the Fushun Diabetic Retinopathy Cohort Study were enrolled. A questionnaire assessing incentives and barriers to diagnosis of DR was administered. Logistic regression was used to identify clinical and sociodemographic factors associated with undiagnosed DR. In a prespecified subgroup analysis, we divided patients into vision-threatening diabetic retinopathy (VTDR) and non-VTDR (NVTDR) subgroups. Results Among 800 participants with DR, 712 (89.0%) were undiagnosed. Among 601 with NVTDR, 566 (94.2%) were undiagnosed. Among 199 with VTDR, 146 (73.4%) were undiagnosed. The risk factors affecting the timely diagnosis of NVTDR and VTDR exhibit significant disparities. In multivariate models, factors associated with undiagnosed VTDR were age over 60 years (OR = 2.966; 95% CI = 1.205-7.299; P = 0.018), duration of diabetes over 10 years (OR = 0.299; 95% CI = 0.118-0753; P = 0.010), visual impairment or blindness (OR = 0.310; 95% CI = 0.117-0.820; P = 0.018), receiving a reminder to schedule an eye examination (OR = 0.380; 95% CI = 0.163-0.883; P = 0.025), and the belief that "people with diabetes are unlikely to develop an eye disease" (OR = 4.691; 95% CI = 1.116-19.724; P = 0.035). However, none of the factors were associated with undiagnosed NVTDR (all P ≥ 0.145). Conclusion Our research has uncovered a disconcerting trend of underdiagnosis in cases of DR within our population. Addressing determinants of undiagnosed DR may facilitate early detection.
Collapse
Affiliation(s)
- Bo Zang
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| | - Shisong Rong
- Mass Eye and Ear, Mass General Brigham, Harvard Medical School, Boston, MA, United States
| | - Dong Li
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| | - Xiaoxia Ding
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| | - Dongxiao Zang
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| | - Fenghua Wang
- Department of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing Ophthalmology and Visual Science Key Laboratory, Beijing, China
| | - Yuanbo Liang
- Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Gang Zhai
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| | - Kemi Feng
- Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Zixi Zhou
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| | - Yu Wang
- Department of Ophthalmology, Fushun Eye Hospital, Fushun, Liaoning, China
| |
Collapse
|
25
|
Li H, Li G, Li N, Liu C, Yuan Z, Gao Q, Hao S, Fan S, Yang J. Cost-effectiveness analysis of artificial intelligence-based diabetic retinopathy screening in rural China based on the Markov model. PLoS One 2023; 18:e0291390. [PMID: 37971984 PMCID: PMC10653408 DOI: 10.1371/journal.pone.0291390] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 08/26/2023] [Indexed: 11/19/2023] Open
Abstract
This study assessed the cost-effectiveness of different diabetic retinopathy (DR) screening strategies in rural regions in China by using a Markov model to make health economic evaluations. In this study, we determined the structure of a Markov model according to the research objectives, which required parameters collected through field investigation and literature retrieval. After perfecting the model with parameters and assumptions, we developed a Markov decision analytic model according to the natural history of DR in TreeAge Pro 2011. For this model, we performed Markov cohort and cost-effectiveness analyses to simulate the probabilistic distributions of different developments in DR and the cumulative cost-effectiveness of artificial intelligence (AI)-based screening and ophthalmologist screening for DR in the rural population with diabetes mellitus (DM) in China. Additionally, a model-based health economic evaluation was performed by using quality-adjusted life years (QALYs) and incremental cost-effectiveness ratios. Last, one-way and probabilistic sensitivity analyses were performed to assess the stability of the results. From the perspective of the health system, compared with no screening, AI-based screening cost more (the incremental cost was 37,257.76 RMB (approximately 5,211.31 US dollars)), but the effect was better (the incremental utility was 0.33). Compared with AI-based screening, the cost of ophthalmologist screening was higher (the incremental cost was 14,886.76 RMB (approximately 2,070.19 US dollars)), and the effect was worse (the incremental utility was -0.31). Compared with no screening, the incremental cost-effectiveness ratio (ICER) of AI-based DR screening was 112,146.99 RMB (15,595.47 US dollars)/QALY, which was less than the threshold for the ICER (< 3 times the per capita gross domestic product (GDP), 217,341.00 RMB (30,224.03 US dollars)). Therefore, AI-based screening was cost-effective, which meant that the increased cost for each additional quality-adjusted life year was merited. Compared with no screening and ophthalmologist screening for DR, AI-based screening was the most cost-effective, which not only saved costs but also improved the quality of life of diabetes patients. Popularizing AI-based DR screening strategies in rural areas would be economically effective and feasible and can provide a scientific basis for the further formulation of early screening programs for diabetic retinopathy.
Collapse
Affiliation(s)
- Huilin Li
- Department of Ophthalmology, Heji Hospital Affiliated to Changzhi Medical College, Changzhi, 046000, China
| | - Guanyan Li
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
- Shenzhen Longgang Otorhinolaryngology Hospital, Shenzhen, 518100, China
| | - Na Li
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Changyan Liu
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Ziyou Yuan
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Qingyue Gao
- Postgraduate Department, Changzhi Medical College, Changzhi, 046000, China
| | - Shaofeng Hao
- Department of Ophthalmology, Heji Hospital Affiliated to Changzhi Medical College, Changzhi, 046000, China
| | - Shengfu Fan
- Department of Foreign Languages, Changzhi Medical College, Changzhi, 046000, China
| | - Jianzhou Yang
- Department of Public Health and Preventive Medicine, Changzhi Medical College, Changzhi, 046000, China
| |
Collapse
|
26
|
Daich Varela M, Sen S, De Guimaraes TAC, Kabiri N, Pontikos N, Balaskas K, Michaelides M. Artificial intelligence in retinal disease: clinical application, challenges, and future directions. Graefes Arch Clin Exp Ophthalmol 2023; 261:3283-3297. [PMID: 37160501 PMCID: PMC10169139 DOI: 10.1007/s00417-023-06052-x] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 03/20/2023] [Accepted: 03/24/2023] [Indexed: 05/11/2023] Open
Abstract
Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.
Collapse
Affiliation(s)
- Malena Daich Varela
- UCL Institute of Ophthalmology, London, UK
- Moorfields Eye Hospital, London, UK
| | | | | | | | - Nikolas Pontikos
- UCL Institute of Ophthalmology, London, UK
- Moorfields Eye Hospital, London, UK
| | | | - Michel Michaelides
- UCL Institute of Ophthalmology, London, UK.
- Moorfields Eye Hospital, London, UK.
| |
Collapse
|
27
|
Wang M, Lin T, Wang L, Lin A, Zou K, Xu X, Zhou Y, Peng Y, Meng Q, Qian Y, Deng G, Wu Z, Chen J, Lin J, Zhang M, Zhu W, Zhang C, Zhang D, Goh RSM, Liu Y, Pang CP, Chen X, Chen H, Fu H. Uncertainty-inspired open set learning for retinal anomaly identification. Nat Commun 2023; 14:6757. [PMID: 37875484 PMCID: PMC10598011 DOI: 10.1038/s41467-023-42444-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 10/11/2023] [Indexed: 10/26/2023] Open
Abstract
Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies.
Collapse
Affiliation(s)
- Meng Wang
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Tian Lin
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
| | - Lianyu Wang
- College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, 211100, Nanjing, Jiangsu, China
- Laboratory of Brain-Machine Intelligence Technology, Ministry of Education Nanjing University of Aeronautics and Astronautics, 211106, Nanjing, Jiangsu, China
| | - Aidi Lin
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
| | - Ke Zou
- National Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science, Sichuan University, 610065, Chengdu, Sichuan, China
| | - Xinxing Xu
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Yi Zhou
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China
| | - Yuanyuan Peng
- School of Biomedical Engineering, Anhui Medical University, 230032, Hefei, Anhui, China
| | - Qingquan Meng
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China
| | - Yiming Qian
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Guoyao Deng
- National Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science, Sichuan University, 610065, Chengdu, Sichuan, China
| | - Zhiqun Wu
- Longchuan People's Hospital, 517300, Heyuan, Guangdong, China
| | - Junhong Chen
- Puning People's Hospital, 515300, Jieyang, Guangdong, China
| | - Jianhong Lin
- Haifeng PengPai Memory Hospital, 516400, Shanwei, Guangdong, China
| | - Mingzhi Zhang
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
| | - Weifang Zhu
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China
| | - Changqing Zhang
- College of Intelligence and Computing, Tianjin University, 300350, Tianjin, China
| | - Daoqiang Zhang
- College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, 211100, Nanjing, Jiangsu, China
- Laboratory of Brain-Machine Intelligence Technology, Ministry of Education Nanjing University of Aeronautics and Astronautics, 211106, Nanjing, Jiangsu, China
| | - Rick Siow Mong Goh
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Yong Liu
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Chi Pui Pang
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, 999077, Hong Kong, China
| | - Xinjian Chen
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China.
- State Key Laboratory of Radiation Medicine and Protection, Soochow University, 215006, Suzhou, China.
| | - Haoyu Chen
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China.
| | - Huazhu Fu
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore.
| |
Collapse
|
28
|
Cobo M, Menéndez Fernández-Miranda P, Bastarrika G, Lloret Iglesias L. Enhancing radiomics and Deep Learning systems through the standardization of medical imaging workflows. Sci Data 2023; 10:732. [PMID: 37865635 PMCID: PMC10590396 DOI: 10.1038/s41597-023-02641-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Accepted: 10/12/2023] [Indexed: 10/23/2023] Open
Affiliation(s)
- Miriam Cobo
- Advanced Computing and e-Science Group, Institute of Physics of Cantabria (IFCA), CSIC - UC, Santander, Spain.
| | | | - Gorka Bastarrika
- Clínica Universidad de Navarra, Department of Radiology, Pamplona, Spain
| | - Lara Lloret Iglesias
- Advanced Computing and e-Science Group, Institute of Physics of Cantabria (IFCA), CSIC - UC, Santander, Spain
| |
Collapse
|
29
|
Guan Z, Li H, Liu R, Cai C, Liu Y, Li J, Wang X, Huang S, Wu L, Liu D, Yu S, Wang Z, Shu J, Hou X, Yang X, Jia W, Sheng B. Artificial intelligence in diabetes management: Advancements, opportunities, and challenges. Cell Rep Med 2023; 4:101213. [PMID: 37788667 PMCID: PMC10591058 DOI: 10.1016/j.xcrm.2023.101213] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Revised: 08/07/2023] [Accepted: 09/08/2023] [Indexed: 10/05/2023]
Abstract
The increasing prevalence of diabetes, high avoidable morbidity and mortality due to diabetes and diabetic complications, and related substantial economic burden make diabetes a significant health challenge worldwide. A shortage of diabetes specialists, uneven distribution of medical resources, low adherence to medications, and improper self-management contribute to poor glycemic control in patients with diabetes. Recent advancements in digital health technologies, especially artificial intelligence (AI), provide a significant opportunity to achieve better efficiency in diabetes care, which may diminish the increase in diabetes-related health-care expenditures. Here, we review the recent progress in the application of AI in the management of diabetes and then discuss the opportunities and challenges of AI application in clinical practice. Furthermore, we explore the possibility of combining and expanding upon existing digital health technologies to develop an AI-assisted digital health-care ecosystem that includes the prevention and management of diabetes.
Collapse
Affiliation(s)
- Zhouyu Guan
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Huating Li
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Ruhan Liu
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China; MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China; National Engineering Research Center of Personalized Diagnostic and Therapeutic Technology, Furong Laboratory, Changsha, Hunan 41000, China
| | - Chun Cai
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Yuexing Liu
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Jiajia Li
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China; MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Xiangning Wang
- Department of Ophthalmology, Shanghai Jiao Tong University Affiliated Sixth People's Hospital, Shanghai 200233, China
| | - Shan Huang
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China; MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Liang Wu
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Dan Liu
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Shujie Yu
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Zheyuan Wang
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China; MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Jia Shu
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China; MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Xuhong Hou
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China
| | - Xiaokang Yang
- MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Weiping Jia
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China.
| | - Bin Sheng
- Shanghai International Joint Laboratory of Intelligent Prevention and Treatment for Metabolic Diseases, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Department of Endocrinology and Metabolism, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Diabetes Institute, Shanghai Clinical Center for Diabetes, Shanghai 200240, China; MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China.
| |
Collapse
|
30
|
Thirunavukarasu AJ, Elangovan K, Gutierrez L, Li Y, Tan I, Keane PA, Korot E, Ting DSW. Democratizing Artificial Intelligence Imaging Analysis With Automated Machine Learning: Tutorial. J Med Internet Res 2023; 25:e49949. [PMID: 37824185 PMCID: PMC10603560 DOI: 10.2196/49949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 08/21/2023] [Accepted: 09/13/2023] [Indexed: 10/13/2023] Open
Abstract
Deep learning-based clinical imaging analysis underlies diagnostic artificial intelligence (AI) models, which can match or even exceed the performance of clinical experts, having the potential to revolutionize clinical practice. A wide variety of automated machine learning (autoML) platforms lower the technical barrier to entry to deep learning, extending AI capabilities to clinicians with limited technical expertise, and even autonomous foundation models such as multimodal large language models. Here, we provide a technical overview of autoML with descriptions of how autoML may be applied in education, research, and clinical practice. Each stage of the process of conducting an autoML project is outlined, with an emphasis on ethical and technical best practices. Specifically, data acquisition, data partitioning, model training, model validation, analysis, and model deployment are considered. The strengths and limitations of available code-free, code-minimal, and code-intensive autoML platforms are considered. AutoML has great potential to democratize AI in medicine, improving AI literacy by enabling "hands-on" education. AutoML may serve as a useful adjunct in research by facilitating rapid testing and benchmarking before significant computational resources are committed. AutoML may also be applied in clinical contexts, provided regulatory requirements are met. The abstraction by autoML of arduous aspects of AI engineering promotes prioritization of data set curation, supporting the transition from conventional model-driven approaches to data-centric development. To fulfill its potential, clinicians must be educated on how to apply these technologies ethically, rigorously, and effectively; this tutorial represents a comprehensive summary of relevant considerations.
Collapse
Affiliation(s)
- Arun James Thirunavukarasu
- University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore, Singapore
| | - Kabilan Elangovan
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore, Singapore
| | - Laura Gutierrez
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore, Singapore
| | - Yong Li
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore, Singapore
| | - Iris Tan
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore, Singapore
| | - Pearse A Keane
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
| | - Edward Korot
- Byers Eye Institute, Stanford University, Palo Alto, CA, United States
- Retina Specialists of Michigan, Grand Rapids, MI, United States
| | - Daniel Shu Wei Ting
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore, Singapore
- Byers Eye Institute, Stanford University, Palo Alto, CA, United States
- Singapore National Eye Centre, Singapore, Singapore
| |
Collapse
|
31
|
Rajesh AE, Davidson OQ, Lee CS, Lee AY. Artificial Intelligence and Diabetic Retinopathy: AI Framework, Prospective Studies, Head-to-head Validation, and Cost-effectiveness. Diabetes Care 2023; 46:1728-1739. [PMID: 37729502 PMCID: PMC10516248 DOI: 10.2337/dci23-0032] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 07/15/2023] [Indexed: 09/22/2023]
Abstract
Current guidelines recommend that individuals with diabetes receive yearly eye exams for detection of referable diabetic retinopathy (DR), one of the leading causes of new-onset blindness. For addressing the immense screening burden, artificial intelligence (AI) algorithms have been developed to autonomously screen for DR from fundus photography without human input. Over the last 10 years, many AI algorithms have achieved good sensitivity and specificity (>85%) for detection of referable DR compared with human graders; however, many questions still remain. In this narrative review on AI in DR screening, we discuss key concepts in AI algorithm development as a background for understanding the algorithms. We present the AI algorithms that have been prospectively validated against human graders and demonstrate the variability of reference standards and cohort demographics. We review the limited head-to-head validation studies where investigators attempt to directly compare the available algorithms. Next, we discuss the literature regarding cost-effectiveness, equity and bias, and medicolegal considerations, all of which play a role in the implementation of these AI algorithms in clinical practice. Lastly, we highlight ongoing efforts to bridge gaps in AI model data sets to pursue equitable development and delivery.
Collapse
Affiliation(s)
- Anand E. Rajesh
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Oliver Q. Davidson
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Cecilia S. Lee
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Aaron Y. Lee
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| |
Collapse
|
32
|
Sung YC, Sheu SJ, Hsu SL. Feasibility Analysis of Teleophthalmology Diagnosis and Referral in Remote Areas of Taiwan. Telemed J E Health 2023; 29:1523-1529. [PMID: 37022780 DOI: 10.1089/tmj.2022.0440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/07/2023] Open
Abstract
Background: Remote areas of Taiwan lack routine and specialized ophthalmology services. This study aimed to analyze feasibility of teleophthalmology service for diseases diagnosis and referral in remote areas of Taiwan. Methods: A retrospective study of medical records from 11 remote teleophthalmology clinics in the Taitung area of Taiwan was conducted from May 2020 to December 2021. Vision and intraocular pressure were checked. Ophthalmic imaging was performed by local trained nurses using a hand-held ophthalmoscope and slit lamp biomicroscope. The images were transmitted by telemedicine system to a medical center. Consultation was conducted via face-to-face real-time video calls. Ophthalmologists in the medical center provided diagnosis and treatment advice based on the real-time images and interactive history taking via the telemedicine system. All the images and data were collected and well-reviewed by ophthalmologists in the medical center, and disease prevalence and referral were analyzed for the program. A small-scale satisfaction questionnaire survey was conducted for efficacy evaluation of the program. Results: A total of 1,401 medical records from 1,094 patients were collected and screened. Patients' ages ranged from 9 months to 94 years, with a mean age of 57.27 (standard deviation ±20.47) years. The most frequent ophthalmologic diagnosis was dry eye disease (20.2%), followed by conjunctivitis (12.4%). Among 322 patients with underlying diseases of diabetes mellitus, 59 patients (18.3%) were diagnosed with diabetic retinopathy. Major diagnosis was made in 102 patients (7.3%) and referral to hospital for further management was suggested. This program had high overall satisfaction score of 89% (mean 4.43 ± 0.52 points) in satisfaction questionnaire survey. Conclusion: Teleophthalmology provides an alternative tool for ocular disease diagnosis and screening for patients in remote areas, especially during the COVID-19 pandemic. This service helps to detect major but undiagnosed diseases and promotes health care accessibility and availability in remote areas that lack specialists.
Collapse
Affiliation(s)
- Yu-Chi Sung
- Department of Ophthalmology, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
- Department of Ophthalmology, School of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
| | - Shwu-Jiuan Sheu
- Department of Ophthalmology, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
- Department of Ophthalmology, School of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
| | - Shiuh-Liang Hsu
- Department of Ophthalmology, Kaohsiung Medical University Hospital, Kaohsiung, Taiwan
- Department of Ophthalmology, School of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
| |
Collapse
|
33
|
Nakayama LF, Zago Ribeiro L, Novaes F, Miyawaki IA, Miyawaki AE, de Oliveira JAE, Oliveira T, Malerbi FK, Regatieri CVS, Celi LA, Silva PS. Artificial intelligence for telemedicine diabetic retinopathy screening: a review. Ann Med 2023; 55:2258149. [PMID: 37734417 PMCID: PMC10515659 DOI: 10.1080/07853890.2023.2258149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Accepted: 08/31/2023] [Indexed: 09/23/2023] Open
Abstract
PURPOSE This study aims to compare artificial intelligence (AI) systems applied in diabetic retinopathy (DR) teleophthalmology screening, currently deployed systems, fairness initiatives and the challenges for implementation. METHODS The review included articles retrieved from PubMed/Medline/EMBASE literature search strategy regarding telemedicine, DR and AI. The screening criteria included human articles in English, Portuguese or Spanish and related to telemedicine and AI for DR screening. The author's affiliations and the study's population income group were classified according to the World Bank Country and Lending Groups. RESULTS The literature search yielded a total of 132 articles, and nine were included after full-text assessment. The selected articles were published between 2004 and 2020 and were grouped as telemedicine systems, algorithms, economic analysis and image quality assessment. Four telemedicine systems that perform a quality assessment, image preprocessing and pathological screening were reviewed. A data and post-deployment bias assessment are not performed in any of the algorithms, and none of the studies evaluate the social impact implementations. There is a lack of representativeness in the reviewed articles, with most authors and target populations from high-income countries and no low-income country representation. CONCLUSIONS Telemedicine and AI hold great promise for augmenting decision-making in medical care, expanding patient access and enhancing cost-effectiveness. Economic studies and social science analysis are crucial to support the implementation of AI in teleophthalmology screening programs. Promoting fairness and generalizability in automated systems combined with telemedicine screening programs is not straightforward. Improving data representativeness, reducing biases and promoting equity in deployment and post-deployment studies are all critical steps in model development.
Collapse
Affiliation(s)
- Luis Filipe Nakayama
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | - Lucas Zago Ribeiro
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | - Frederico Novaes
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | | | | | | | - Talita Oliveira
- Department of Ophthalmology, São Paulo Federal University, Sao Paulo, Brazil
| | | | | | - Leo Anthony Celi
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biostatistics, Harvard TH Chan School of Public Health, Boston, MA, USA
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Paolo S. Silva
- Beetham Eye Institute, Joslin Diabetes Centre, Harvard Medical School, Boston, MA, USA
- Philippine Eye Research Institute, University of the Philippines, Manila, Philippines
| |
Collapse
|
34
|
Wolf RM, Channa R, Lehmann HP, Abramoff MD, Liu TA. Clinical Implementation of Autonomous Artificial Intelligence Systems for Diabetic Eye Exams: Considerations for Success. Clin Diabetes 2023; 42:142-149. [PMID: 38230333 PMCID: PMC10788651 DOI: 10.2337/cd23-0019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/18/2024]
Affiliation(s)
- Risa M. Wolf
- Department of Pediatric Endocrinology and Diabetes, Johns Hopkins University School of Medicine, Baltimore, MD
| | - Roomasa Channa
- Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, WI
| | - Harold P. Lehmann
- Section on Biomedical Informatics and Data Science, Johns Hopkins University, Baltimore, MD
| | - Michael D. Abramoff
- Department of Ophthalmology and Visual Sciences, The University of Iowa, Iowa City, IA
- Digital Diagnostics, Coralville, IA
| | - T.Y. Alvin Liu
- Wilmer Eye Institute at the Johns Hopkins University School of Medicine, Baltimore, MD
| |
Collapse
|
35
|
Tan TF, Thirunavukarasu AJ, Jin L, Lim J, Poh S, Teo ZL, Ang M, Chan RVP, Ong J, Turner A, Karlström J, Wong TY, Stern J, Ting DSW. Artificial intelligence and digital health in global eye health: opportunities and challenges. Lancet Glob Health 2023; 11:e1432-e1443. [PMID: 37591589 DOI: 10.1016/s2214-109x(23)00323-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2023] [Revised: 06/26/2023] [Accepted: 07/04/2023] [Indexed: 08/19/2023]
Abstract
Global eye health is defined as the degree to which vision, ocular health, and function are maximised worldwide, thereby optimising overall wellbeing and quality of life. Improving eye health is a global priority as a key to unlocking human potential by reducing the morbidity burden of disease, increasing productivity, and supporting access to education. Although extraordinary progress fuelled by global eye health initiatives has been made over the last decade, there remain substantial challenges impeding further progress. The accelerated development of digital health and artificial intelligence (AI) applications provides an opportunity to transform eye health, from facilitating and increasing access to eye care to supporting clinical decision making with an objective, data-driven approach. Here, we explore the opportunities and challenges presented by digital health and AI in global eye health and describe how these technologies could be leveraged to improve global eye health. AI, telehealth, and emerging technologies have great potential, but require specific work to overcome barriers to implementation. We suggest that a global digital eye health task force could facilitate coordination of funding, infrastructural development, and democratisation of AI and digital health to drive progress forwards in this domain.
Collapse
Affiliation(s)
- Ting Fang Tan
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Singapore National Eye Centre, Singapore General Hospital, Singapore
| | - Arun J Thirunavukarasu
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Corpus Christi College, University of Cambridge, Cambridge, UK; School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Liyuan Jin
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Duke-NUS Medical School, National University of Singapore, Singapore
| | - Joshua Lim
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Singapore National Eye Centre, Singapore General Hospital, Singapore
| | - Stanley Poh
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Singapore National Eye Centre, Singapore General Hospital, Singapore
| | - Zhen Ling Teo
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Singapore National Eye Centre, Singapore General Hospital, Singapore
| | - Marcus Ang
- Singapore National Eye Centre, Singapore General Hospital, Singapore; Duke-NUS Medical School, National University of Singapore, Singapore
| | - R V Paul Chan
- Illinois Eye and Ear Infirmary, University of Illinois College of Medicine, Urbana-Champaign, IL, USA
| | - Jasmine Ong
- Pharmacy Department, Singapore General Hospital, Singapore
| | - Angus Turner
- Lions Eye Institute, University of Western Australia, Nedlands, WA, Australia
| | - Jonas Karlström
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Tien Yin Wong
- Singapore National Eye Centre, Singapore General Hospital, Singapore; Tsinghua Medicine, Tsinghua University, Beijing, China
| | - Jude Stern
- The International Agency for the Prevention of Blindness, London, UK
| | - Daniel Shu-Wei Ting
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research Institute, Singapore; Singapore National Eye Centre, Singapore General Hospital, Singapore; Duke-NUS Medical School, National University of Singapore, Singapore.
| |
Collapse
|
36
|
Ruamviboonsuk P, Ruamviboonsuk V, Tiwari R. Recent evidence of economic evaluation of artificial intelligence in ophthalmology. Curr Opin Ophthalmol 2023; 34:449-458. [PMID: 37459289 DOI: 10.1097/icu.0000000000000987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2023]
Abstract
PURPOSE OF REVIEW Health economic evaluation (HEE) is essential for assessing value of health interventions, including artificial intelligence. Recent approaches, current challenges, and future directions of HEE of artificial intelligence in ophthalmology are reviewed. RECENT FINDINGS Majority of recent HEEs of artificial intelligence in ophthalmology were for diabetic retinopathy screening. Two models, one conducted in the rural USA (5-year period) and another in China (35-year period), found artificial intelligence to be more cost-effective than without screening for diabetic retinopathy. Two additional models, which compared artificial intelligence with human screeners in Brazil and Thailand for the lifetime of patients, found artificial intelligence to be more expensive from a healthcare system perspective. In the Thailand analysis, however, artificial intelligence was less expensive when opportunity loss from blindness was included. An artificial intelligence model for screening retinopathy of prematurity was cost-effective in the USA. A model for screening age-related macular degeneration in Japan and another for primary angle close in China did not find artificial intelligence to be cost-effective, compared with no screening. The costs of artificial intelligence varied widely in these models. SUMMARY Like other medical fields, there is limited evidence in assessing the value of artificial intelligence in ophthalmology and more appropriate HEE models are needed.
Collapse
Affiliation(s)
- Paisan Ruamviboonsuk
- Department of Ophthalmology, Rajavithi Hospital, College of Medicine, Rangsit University
| | | | | |
Collapse
|
37
|
Rizvi A, Rizvi F, Lalakia P, Hyman L, Frasso R, Sztandera L, Das AV. Is Artificial Intelligence the Cost-Saving Lens to Diabetic Retinopathy Screening in Low- and Middle-Income Countries? Cureus 2023; 15:e45539. [PMID: 37868419 PMCID: PMC10586227 DOI: 10.7759/cureus.45539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/17/2023] [Indexed: 10/24/2023] Open
Abstract
Diabetes is a rapidly growing global health crisis disproportionately affecting low- and middle-income countries (LMICs). The emergence of diabetes as a global pandemic is one of the major challenges to human health, as long-term microvascular complications such as diabetic retinopathy (DR) can lead to irreversible blindness. Leveraging artificial intelligence (AI) technology may improve the diagnostic accuracy, efficiency, and accessibility of DR screenings across LMICs. However, there is a gap between the potential of AI technology and its implementation in clinical practice. The main objective of this systematic review is to summarize the currently available literature on the health economic assessments of AI implementation for DR screening in LMICs. The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We conducted an extensive systematic search of PubMed/MEDLINE, Scopus, and the Web of Science on July 15, 2023. Our review included full-text English-language articles from any publication year. The Joanna Briggs Institute's (JBI) critical appraisal checklist for economic evaluations was used to rate the quality and rigor of the selected articles. The initial search generated 1,423 records and was narrowed to five full-text articles through comprehensive inclusion and exclusion criteria. Of the five articles included in our systematic review, two used a cost-effectiveness analysis, two used a cost-utility analysis, and one used both a cost-effectiveness analysis and a cost-utility analysis. Across the five articles, LMICs such as China, Thailand, and Brazil were represented in the economic evaluations and models. Overall, three out of the five articles concluded that AI-based DR screening was more cost-effective in comparison to standard-of-care screening methods. Our systematic review highlights the need for more primary health economic analyses that carefully evaluate the economic implications of adopting AI technology for DR screening in LMICs. We hope this systematic review will offer valuable guidance to healthcare providers, scientists, and legislators to support appropriate decision-making regarding the implementation of AI algorithms for DR screening in healthcare workflows.
Collapse
Affiliation(s)
- Anza Rizvi
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
| | - Fatima Rizvi
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
| | - Parth Lalakia
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
- Osteopathic Medicine, Rowan-Virtua School of Osteopathic Medicine, Stratford, USA
- Office of Global Affairs, Thomas Jefferson University, Philadelphia, USA
| | - Leslie Hyman
- Geriatric Medicine and Palliative Care, Department of Family Medicine, Thomas Jefferson University, Philadelphia, USA
- The Vickie and Jack Farber Vision Research Center, Wills Eye Hospital, Philadelphia, USA
| | - Rosemary Frasso
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
- College of Population Health, Thomas Jefferson University, Philadelphia, USA
- Asano-Gonnella Center for Research in Medical Education and Health Care, Thomas Jefferson University, Philadelphia, USA
| | - Les Sztandera
- Kanbar College of Design, Engineering, and Commerce, Thomas Jefferson University, Philadelphia, USA
| | | |
Collapse
|
38
|
Penha FM, Priotto BM, Hennig F, Przysiezny B, Wiethorn BA, Orsi J, Nagel IBF, Wiggers B, Stuchi JA, Lencione D, de Souza Prado PV, Yamanaka F, Lojudice F, Malerbi FK. Single retinal image for diabetic retinopathy screening: performance of a handheld device with embedded artificial intelligence. Int J Retina Vitreous 2023; 9:41. [PMID: 37430345 DOI: 10.1186/s40942-023-00477-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 06/23/2023] [Indexed: 07/12/2023] Open
Abstract
BACKGROUND Diabetic retinopathy (DR) is a leading cause of blindness. Our objective was to evaluate the performance of an artificial intelligence (AI) system integrated into a handheld smartphone-based retinal camera for DR screening using a single retinal image per eye. METHODS Images were obtained from individuals with diabetes during a mass screening program for DR in Blumenau, Southern Brazil, conducted by trained operators. Automatic analysis was conducted using an AI system (EyerMaps™, Phelcom Technologies LLC, Boston, USA) with one macula-centered, 45-degree field of view retinal image per eye. The results were compared to the assessment by a retinal specialist, considered as the ground truth, using two images per eye. Patients with ungradable images were excluded from the analysis. RESULTS A total of 686 individuals (average age 59.2 ± 13.3 years, 56.7% women, diabetes duration 12.1 ± 9.4 years) were included in the analysis. The rates of insulin use, daily glycemic monitoring, and systemic hypertension treatment were 68.4%, 70.2%, and 70.2%, respectively. Although 97.3% of patients were aware of the risk of blindness associated with diabetes, more than half of them underwent their first retinal examination during the event. The majority (82.5%) relied exclusively on the public health system. Approximately 43.4% of individuals were either illiterate or had not completed elementary school. DR classification based on the ground truth was as follows: absent or nonproliferative mild DR 86.9%, more than mild (mtm) DR 13.1%. The AI system achieved sensitivity, specificity, positive predictive value, and negative predictive value percentages (95% CI) for mtmDR as follows: 93.6% (87.8-97.2), 71.7% (67.8-75.4), 42.7% (39.3-46.2), and 98.0% (96.2-98.9), respectively. The area under the ROC curve was 86.4%. CONCLUSION The portable retinal camera combined with AI demonstrated high sensitivity for DR screening using only one image per eye, offering a simpler protocol compared to the traditional approach of two images per eye. Simplifying the DR screening process could enhance adherence rates and overall program coverage.
Collapse
Affiliation(s)
- Fernando Marcondes Penha
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil.
- Botelho Hospital da Visão, Rua 2 de Setembro, 2958, Blumenau, 89052-504, SC, Brazil.
| | - Bruna Milene Priotto
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Francini Hennig
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Bernardo Przysiezny
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Bruno Antunes Wiethorn
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Julia Orsi
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | | | - Brenda Wiggers
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | | | | | | | | | - Fernando Lojudice
- Bayer Healthcare - Brazil, São Paulo, SP, Brazil
- Cell and Molecular Theraphy Center (NUCEL), School of Medicine, University of São Paulo, São Paulo, SP, Brazil
| | - Fernando Korn Malerbi
- Department of Ophthalmology, Federal University of São Paulo (UNIFESP), São Paulo, SP, Brazil
| |
Collapse
|
39
|
Farah L, Davaze-Schneider J, Martin T, Nguyen P, Borget I, Martelli N. Are current clinical studies on artificial intelligence-based medical devices comprehensive enough to support a full health technology assessment? A systematic review. Artif Intell Med 2023; 140:102547. [PMID: 37210155 DOI: 10.1016/j.artmed.2023.102547] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 03/28/2023] [Accepted: 04/04/2023] [Indexed: 05/22/2023]
Abstract
INTRODUCTION Artificial Intelligence-based Medical Devices (AI-based MDs) are experiencing exponential growth in healthcare. This study aimed to investigate whether current studies assessing AI contain the information required for health technology assessment (HTA) by HTA bodies. METHODS We conducted a systematic literature review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology to extract articles published between 2016 and 2021 related to the assessment of AI-based MDs. Data extraction focused on study characteristics, technology, algorithms, comparators, and results. AI quality assessment and HTA scores were calculated to evaluate whether the items present in the included studies were concordant with the HTA requirements. We performed a linear regression for the HTA and AI scores with the explanatory variables of the impact factor, publication date, and medical specialty. We conducted a univariate analysis of the HTA score and a multivariate analysis of the AI score with an alpha risk of 5 %. RESULTS Of 5578 retrieved records, 56 were included. The mean AI quality assessment score was 67 %; 32 % of articles had an AI quality score ≥ 70 %, 50 % had a score between 50 % and 70 %, and 18 % had a score under 50 %. The highest quality scores were observed for the study design (82 %) and optimisation (69 %) categories, whereas the scores were lowest in the clinical practice category (23 %). The mean HTA score was 52 % for all seven domains. 100 % of the studies assessed clinical effectiveness, whereas only 9 % evaluated safety, and 20 % evaluated economic issues. There was a statistically significant relationship between the impact factor and the HTA and AI scores (both p = 0.046). DISCUSSION Clinical studies on AI-based MDs have limitations and often lack adapted, robust, and complete evidence. High-quality datasets are also required because the output data can only be trusted if the inputs are reliable. The existing assessment frameworks are not specifically designed to assess AI-based MDs. From the perspective of regulatory authorities, we suggest that these frameworks should be adapted to assess the interpretability, explainability, cybersecurity, and safety of ongoing updates. From the perspective of HTA agencies, we highlight that transparency, professional and patient acceptance, ethical issues, and organizational changes are required for the implementation of these devices. Economic assessments of AI should rely on a robust methodology (business impact or health economic models) to provide decision-makers with more reliable evidence. CONCLUSION Currently, AI studies are insufficient to cover HTA prerequisites. HTA processes also need to be adapted because they do not consider the important specificities of AI-based MDs. Specific HTA workflows and accurate assessment tools should be designed to standardise evaluations, generate reliable evidence, and create confidence.
Collapse
Affiliation(s)
- Line Farah
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé (GRADES) Department, University Paris-Saclay, Orsay, France; Innovation Center for Medical Devices, Foch Hospital, 40 Rue Worth, 92150 Suresnes, France.
| | - Julie Davaze-Schneider
- Pharmacy Department, Georges Pompidou European Hospital, AP-HP, 20 Rue Leblanc, 75015 Paris, France
| | - Tess Martin
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé (GRADES) Department, University Paris-Saclay, Orsay, France; Pharmacy Department, Georges Pompidou European Hospital, AP-HP, 20 Rue Leblanc, 75015 Paris, France
| | - Pierre Nguyen
- Pharmacy Department, Georges Pompidou European Hospital, AP-HP, 20 Rue Leblanc, 75015 Paris, France
| | - Isabelle Borget
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé (GRADES) Department, University Paris-Saclay, Orsay, France; Department of Biostatistics and Epidemiology, Gustave Roussy, University Paris-Saclay, 94805 Villejuif, France; Oncostat U1018, Inserm, University Paris-Saclay, Équipe Labellisée Ligue Contre le Cancer, Villejuif, France
| | - Nicolas Martelli
- Groupe de Recherche et d'accueil en Droit et Economie de la Santé (GRADES) Department, University Paris-Saclay, Orsay, France; Pharmacy Department, Georges Pompidou European Hospital, AP-HP, 20 Rue Leblanc, 75015 Paris, France
| |
Collapse
|
40
|
Vaghefi E, Yang S, Xie L, Han D, Yap A, Schmeidel O, Marshall J, Squirrell D. A multi-centre prospective evaluation of THEIA™ to detect diabetic retinopathy (DR) and diabetic macular oedema (DMO) in the New Zealand screening program. Eye (Lond) 2023; 37:1683-1689. [PMID: 36057664 PMCID: PMC10219993 DOI: 10.1038/s41433-022-02217-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 07/09/2022] [Accepted: 08/12/2022] [Indexed: 12/23/2022] Open
Abstract
PURPOSE To validate the potential application of THEIA™ as clinical decision making assistant in a national screening program. METHODS A total of 900 patients were recruited from either an urban large eye hospital, or a semi-rural optometrist led screening provider, as they were attending their appointment as part of New Zealand Diabetic Eye Screening Programme. The de-identified images were independently graded by three senior specialists, and final results were aggregated using New Zealand grading scheme, which was then converted to referable/non-referable and Healthy/mild/more than mild/sight threatening categories. RESULTS THEIA™ managed to grade all images obtained during the study. Comparing the adjudicated images from the specialist grading team, "ground truth", with the grading by the AI platform in detecting "sight threatening" disease, at the patient level THEIA™ achieved 100% imageability, 100% [98.49-100.00%] sensitivity and [97.02-99.16%] specificity, and negative predictive value of 100%. In other words, THEIA™ did not miss any patients with "more than mild" or "sight threatening" disease. The level of agreement between the clinicians and the aggregated results was (k value: 0.9881, 0.9557, and 0.9175), and the level of agreement between THEIA™ and the aggregated labels was (k value: 0.9515). CONCLUSION This multi-centre prospective trial showed that THEIA™ did not miss referable disease when screening for diabetic retinopathy and maculopathy. It also had a very high level of granularity in reporting the disease level. As THEIA™ has been tested on a variety of cameras, operating in a range of clinics (rural/urban, ophthalmologist-led\optometrist-led), we believe that it will be a suitable addition to a public diabetic screening program.
Collapse
Affiliation(s)
- Ehsan Vaghefi
- Toku Eyes®, Auckland, New Zealand.
- School of Optometry and Vision Science, The University of Auckland, Auckland, New Zealand.
| | | | - Li Xie
- Toku Eyes®, Auckland, New Zealand
| | | | - Aaron Yap
- Department of Ophthalmology, The University of Auckland, Auckland, New Zealand
| | - Ole Schmeidel
- Department of Diabetes, Auckland District Health Board, Auckland, New Zealand
| | - John Marshall
- Institute of Ophthalmology, University College of London, London, UK
| | - David Squirrell
- Toku Eyes®, Auckland, New Zealand
- Department of Ophthalmology, The University of Auckland, Auckland, New Zealand
| |
Collapse
|
41
|
Teo ZL, Kwee A, Lim JC, Lam CS, Ho D, Maurer-Stroh S, Su Y, Chesterman S, Chen T, Tan CC, Wong TY, Ngiam KY, Tan CH, Soon D, Choong ML, Chua R, Wong S, Lim C, Cheong WY, Ting DS. Artificial intelligence innovation in healthcare: Relevance of reporting guidelines for clinical translation from bench to bedside. ANNALS OF THE ACADEMY OF MEDICINE, SINGAPORE 2023; 52:199-212. [PMID: 38904533 DOI: 10.47102/annals-acadmedsg.2022452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/22/2024]
Abstract
Artificial intelligence (AI) and digital innovation are transforming healthcare. Technologies such as machine learning in image analysis, natural language processing in medical chatbots and electronic medical record extraction have the potential to improve screening, diagnostics and prognostication, leading to precision medicine and preventive health. However, it is crucial to ensure that AI research is conducted with scientific rigour to facilitate clinical implementation. Therefore, reporting guidelines have been developed to standardise and streamline the development and validation of AI technologies in health. This commentary proposes a structured approach to utilise these reporting guidelines for the translation of promising AI techniques from research and development into clinical translation, and eventual widespread implementation from bench to bedside.
Collapse
Affiliation(s)
- Zhen Ling Teo
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
| | - Ann Kwee
- Department of Endocrinology, Singapore General Hospital, Singapore
| | - John Cw Lim
- Centre of Regulatory Excellence, Duke-NUS Medical School, National University of Singapore, Singapore
| | - Carolyn Sp Lam
- Department of Cardiology, National Heart Centre Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Dean Ho
- Department of Biomedical Engineering, Institute of Digital Medicine, N.1 Institute of Health and Department of Pharmacology, National University of Singapore, Singapore
| | - Sebastian Maurer-Stroh
- Bioinformatics Institute and Infectious Diseases Labs, Agency for Science, Technology and Research, Singapore
- Yong Loo Lin School of Medicine and Department of Biological Sciences, National University of Singapore, Singapore
| | - Yi Su
- Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore
| | - Simon Chesterman
- Faculty of Law, National University of Singapore, Singapore
- AI Singapore, Singapore
| | - Tsuhan Chen
- AI Singapore, Singapore
- School of Computing, National University of Singapore, Singapore
| | - Chorh Chuan Tan
- Chief Health Scientist Office, Ministry of Health, Singapore
| | - Tien Yin Wong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
- Tsinghua Medicine, Tsinghua University, Beijing, China
| | - Kee Yuan Ngiam
- Group Technology Office, National University Health System, Singapore
| | - Cher Heng Tan
- Centre for Health Innovation, National Healthcare Group, Singapore
| | - Danny Soon
- Consortium for Clinical Research and Innovation, Singapore, Singapore
| | | | - Raymond Chua
- Director of Medical Services Office (Health Regulation Group), Ministry of Health, Singapore
| | - Sutowo Wong
- Data Analytics, Ministry of Health, Singapore
| | - Colin Lim
- Technology, Ministry of Health, Singapore
| | | | - Daniel Sw Ting
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore
- Artificial Intelligence Office, Singapore Health Services, Singapore
| |
Collapse
|
42
|
Srisubat A, Kittrongsiri K, Sangroongruangsri S, Khemvaranan C, Shreibati JB, Ching J, Hernandez J, Tiwari R, Hersch F, Liu Y, Hanutsaha P, Ruamviboonsuk V, Turongkaravee S, Raman R, Ruamviboonsuk P. Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program. Ophthalmol Ther 2023; 12:1339-1357. [PMID: 36841895 PMCID: PMC10011252 DOI: 10.1007/s40123-023-00688-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Accepted: 02/10/2023] [Indexed: 02/27/2023] Open
Abstract
INTRODUCTION Deep learning (DL) for screening diabetic retinopathy (DR) has the potential to address limited healthcare resources by enabling expanded access to healthcare. However, there is still limited health economic evaluation, particularly in low- and middle-income countries, on this subject to aid decision-making for DL adoption. METHODS In the context of a middle-income country (MIC), using Thailand as a model, we constructed a decision tree-Markov hybrid model to estimate lifetime costs and outcomes of Thailand's national DR screening program via DL and trained human graders (HG). We calculated the incremental cost-effectiveness ratio (ICER) between the two strategies. Sensitivity analyses were performed to probe the influence of modeling parameters. RESULTS From a societal perspective, screening with DL was associated with a reduction in costs of ~ US$ 2.70, similar quality-adjusted life-years (QALY) of + 0.0043, and an incremental net monetary benefit of ~ US$ 24.10 in the base case. In sensitivity analysis, DL remained cost-effective even with a price increase from US$ 1.00 to US$ 4.00 per patient at a Thai willingness-to-pay threshold of ~ US$ 4.997 per QALY gained. When further incorporating recent findings suggesting improved compliance to treatment referral with DL, our analysis models effectiveness benefits of ~ US$ 20 to US$ 50 depending on compliance. CONCLUSION DR screening using DL in an MIC using Thailand as a model may result in societal cost-savings and similar health outcomes compared with HG. This study may provide an economic rationale to expand DL-based DR screening in MICs as an alternative solution for limited availability of skilled human resources for primary screening, particularly in MICs with similar prevalence of diabetes and low compliance to referrals for treatment.
Collapse
Affiliation(s)
- Attasit Srisubat
- Department of Medical Services, Ministry of Public Health, Nonthaburi, Thailand
| | - Kankamon Kittrongsiri
- Social, Economic and Administrative Pharmacy (SEAP) Graduate Program, Faculty of Pharmacy, Mahidol University, Bangkok, Thailand
| | - Sermsiri Sangroongruangsri
- Social and Administrative Pharmacy Division, Department of Pharmacy, Faculty of Pharmacy, Mahidol University, Bangkok, Thailand.
| | - Chalida Khemvaranan
- Department of Research and Technology Assessment, Lerdsin Hospital, Bangkok, Thailand
| | | | | | | | | | | | - Yun Liu
- Google LLC, Mountain View, CA, USA
| | - Prut Hanutsaha
- Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand
| | | | - Saowalak Turongkaravee
- Social and Administrative Pharmacy Division, Department of Pharmacy, Faculty of Pharmacy, Mahidol University, Bangkok, Thailand
| | - Rajiv Raman
- Sri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Chennai, Tamil Nadu, India
| | - Paisan Ruamviboonsuk
- Department of Ophthalmology, College of Medicine, Rajavithi Hospital, Rangsit University, Bangkok, Thailand.
| |
Collapse
|
43
|
Channa R, Wolf RM, Abràmoff MD, Lehmann HP. Effectiveness of artificial intelligence screening in preventing vision loss from diabetes: a policy model. NPJ Digit Med 2023; 6:53. [PMID: 36973403 PMCID: PMC10042864 DOI: 10.1038/s41746-023-00785-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Accepted: 02/24/2023] [Indexed: 03/29/2023] Open
Abstract
The effectiveness of using artificial intelligence (AI) systems to perform diabetic retinal exams ('screening') on preventing vision loss is not known. We designed the Care Process for Preventing Vision Loss from Diabetes (CAREVL), as a Markov model to compare the effectiveness of point-of-care autonomous AI-based screening with in-office clinical exam by an eye care provider (ECP), on preventing vision loss among patients with diabetes. The estimated incidence of vision loss at 5 years was 1535 per 100,000 in the AI-screened group compared to 1625 per 100,000 in the ECP group, leading to a modelled risk difference of 90 per 100,000. The base-case CAREVL model estimated that an autonomous AI-based screening strategy would result in 27,000 fewer Americans with vision loss at 5 years compared with ECP. Vision loss at 5 years remained lower in the AI-screened group compared to the ECP group, in a wide range of parameters including optimistic estimates biased toward ECP. Real-world modifiable factors associated with processes of care could further increase its effectiveness. Of these factors, increased adherence with treatment was estimated to have the greatest impact.
Collapse
Affiliation(s)
- Roomasa Channa
- Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, WI, USA.
| | - Risa M Wolf
- Department of Pediatrics, Division of Endocrinology, Johns Hopkins Medicine, Baltimore, MD, USA
| | - Michael D Abràmoff
- Department of Ophthalmology and Visual Sciences, University of Iowa, Iowa City, IA, USA
| | - Harold P Lehmann
- Department of Medicine, Section on Biomedical Informatics and Data Science, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
44
|
Wen J, Liu D, Wu Q, Zhao L, Iao WC, Lin H. Retinal image‐based artificial intelligence in detecting and predicting kidney diseases: Current advances and future perspectives. VIEW 2023. [DOI: 10.1002/viw.20220070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/22/2023] Open
Affiliation(s)
- Jingyi Wen
- State Key Laboratory of OphthalmologyZhongshan Ophthalmic CenterSun Yat‐sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Disease GuangzhouChina
| | - Dong Liu
- State Key Laboratory of OphthalmologyZhongshan Ophthalmic CenterSun Yat‐sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Disease GuangzhouChina
| | - Qianni Wu
- State Key Laboratory of OphthalmologyZhongshan Ophthalmic CenterSun Yat‐sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Disease GuangzhouChina
| | - Lanqin Zhao
- State Key Laboratory of OphthalmologyZhongshan Ophthalmic CenterSun Yat‐sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Disease GuangzhouChina
| | - Wai Cheng Iao
- State Key Laboratory of OphthalmologyZhongshan Ophthalmic CenterSun Yat‐sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Disease GuangzhouChina
| | - Haotian Lin
- State Key Laboratory of OphthalmologyZhongshan Ophthalmic CenterSun Yat‐sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Disease GuangzhouChina
- Center for Precision Medicine and Department of Genetics and Biomedical Informatics Zhongshan School of Medicine Sun Yat‐sen University Guangzhou China
| |
Collapse
|
45
|
Yang Y, Xu F, Chen J, Tao C, Li Y, Chen Q, Tang S, Lee HK, Shen W. Artificial intelligence-assisted smartphone-based sensing for bioanalytical applications: A review. Biosens Bioelectron 2023; 229:115233. [PMID: 36965381 DOI: 10.1016/j.bios.2023.115233] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 02/23/2023] [Accepted: 03/13/2023] [Indexed: 03/18/2023]
Abstract
Artificial intelligence (AI) has received great attention since the concept was proposed, and it has developed rapidly in recent years with applications in many fields. Meanwhile, newer iterations of smartphone hardware technologies which have excellent data processing capabilities have leveraged on AI capabilities. Based on the desirability for portable detection, researchers have been investigating intelligent analysis by combining smartphones with AI algorithms. Various examples of the application of AI algorithm-based smartphone detection and analysis have been developed. In this review, we give an overview of this field, with a particular focus on bioanalytical detection applications. The applications are presented in terms of hardware design, software algorithms, and specific application areas. We also discuss the existing limitations of AI-based smartphone detection and analytical approaches, and their future prospects. The take-home message of our review is that the application of AI in the field of detection analysis is restricted by the limitations of the smartphone's hardware as well as the model building of AI for detection targets with insufficient data. Nevertheless, at this juncture, while bioanalytical diagnostics and health monitoring have set the pace for AI-based smartphone applicability, the future should see the technology making greater inroads into other fields. In relation to the latter, it is likely that the ordinary or average person will play a greater participatory role.
Collapse
Affiliation(s)
- Yizhuo Yang
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China
| | - Fang Xu
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China
| | - Jisen Chen
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China
| | - Chunxu Tao
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China
| | - Yunxin Li
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China
| | - Quansheng Chen
- College of Ocean Food and Biological Engineering, Jimei University, Xiamen, 361021, Fujian Province, China
| | - Sheng Tang
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China.
| | - Hian Kee Lee
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China; Department of Chemistry, National University of Singapore, 3 Science Drive 3, Singapore, 117543, Singapore.
| | - Wei Shen
- School of Environmental and Chemical Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, Jiangsu Province, China.
| |
Collapse
|
46
|
Artificial Intelligence for Diabetic Retinopathy Screening Using Color Retinal Photographs: From Development to Deployment. Ophthalmol Ther 2023; 12:1419-1437. [PMID: 36862308 PMCID: PMC10164194 DOI: 10.1007/s40123-023-00691-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 02/14/2023] [Indexed: 03/03/2023] Open
Abstract
Diabetic retinopathy (DR), a leading cause of preventable blindness, is expected to remain a growing health burden worldwide. Screening to detect early sight-threatening lesions of DR can reduce the burden of vision loss; nevertheless, the process requires intensive manual labor and extensive resources to accommodate the increasing number of patients with diabetes. Artificial intelligence (AI) has been shown to be an effective tool which can potentially lower the burden of screening DR and vision loss. In this article, we review the use of AI for DR screening on color retinal photographs in different phases of application, ranging from development to deployment. Early studies of machine learning (ML)-based algorithms using feature extraction to detect DR achieved a high sensitivity but relatively lower specificity. Robust sensitivity and specificity were achieved with the application of deep learning (DL), although ML is still used in some tasks. Public datasets were utilized in retrospective validations of the developmental phases in most algorithms, which require a large number of photographs. Large prospective clinical validation studies led to the approval of DL for autonomous screening of DR although the semi-autonomous approach may be preferable in some real-world settings. There have been few reports on real-world implementations of DL for DR screening. It is possible that AI may improve some real-world indicators for eye care in DR, such as increased screening uptake and referral adherence, but this has not been proven. The challenges in deployment may include workflow issues, such as mydriasis to lower ungradable cases; technical issues, such as integration into electronic health record systems and integration into existing camera systems; ethical issues, such as data privacy and security; acceptance of personnel and patients; and health-economic issues, such as the need to conduct health economic evaluations of using AI in the context of the country. The deployment of AI for DR screening should follow the governance model for AI in healthcare which outlines four main components: fairness, transparency, trustworthiness, and accountability.
Collapse
|
47
|
Liu H, Li R, Zhang Y, Zhang K, Yusufu M, Liu Y, Mou D, Chen X, Tian J, Li H, Fan S, Tang J, Wang N. Economic evaluation of combined population-based screening for multiple blindness-causing eye diseases in China: a cost-effectiveness analysis. Lancet Glob Health 2023; 11:e456-e465. [PMID: 36702141 DOI: 10.1016/s2214-109x(22)00554-x] [Citation(s) in RCA: 16] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 11/24/2022] [Accepted: 12/14/2022] [Indexed: 01/25/2023]
Abstract
BACKGROUND More than 90% of vision impairment is avoidable. However, in China, a routine screening programme is currently unavailable in primary health care. With the dearth of economic evidence on screening programmes for multiple blindness-causing eye diseases, delivery options, and screening frequencies, we aimed to evaluate the costs and benefits of a population-based screening programme for multiple eye diseases in China. METHODS We developed a decision-analytic Markov model for a cohort of individuals aged 50 years and older with a total of 30 1-year cycles. We calculated the cost-effectiveness and cost-utility of screening programmes for multiple major blindness-causing eye diseases in China, including age-related macular degeneration, glaucoma, diabetic retinopathy, cataracts, and pathological myopia, from a societal perspective (including direct and indirect costs). We analysed rural and urban settings separately by different screening delivery options (non-telemedicine [ie, face-to-face] screening, artificial intelligence [AI] telemedicine screening, and non-AI telemedicine screening) and frequencies. We calculated incremental cost-utility ratios (ICURs) using quality-adjusted life-years and incremental cost-effectiveness ratios (ICERs) in terms of the cost per blindness year avoided. One-way deterministic and simulated probabilistic sensitivity analyses were used to assess the robustness of the main outcomes. FINDINGS Compared with no screening, non-telemedicine combined screening of multiple eye diseases satisfied the criterion for a highly cost-effective health intervention, with an ICUR of US$2494 (95% CI 1130 to 2716) and an ICER of $12 487 (8773 to 18 791) in rural settings. In urban areas, the ICUR was $624 (395 to 907), and the ICER was $7251 (4238 to 13 501). Non-AI telemedicine screening could result in fewer costs and greater gains in health benefits (ICUR $2326 [1064 to 2538] and ICER $11 766 [8200 to 18 000] in rural settings; ICUR $581 [368 to 864] and ICER $6920 [3926 to 13 231] in urban settings). AI telemedicine screening dominated no screening in rural settings, and in urban settings the ICUR was $244 (-315 to 1073) and the ICER was $2567 (-4111 to 15 389). Sensitivity analyses showed all results to be robust. By further comparison, annual AI telemedicine screening was the most cost-effective strategy in both rural and urban areas. INTERPRETATION Combined screening of multiple eye diseases is cost-effective in both rural and urban China. AI coupled with teleophthalmology presents an opportunity to promote equity in eye health. FUNDING National Natural Science Foundation of China.
Collapse
Affiliation(s)
- Hanruo Liu
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China; School of Medical Technology, Beijing Institute of Technology, Beijing, China; National Institutes of Health Data Science at Peking University, Beijing, China.
| | - Ruyue Li
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Yue Zhang
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Kaiwen Zhang
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Mayinuer Yusufu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, VIC, Australia
| | - Yanting Liu
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Dapeng Mou
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Xiaoniao Chen
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Jiaxin Tian
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Huiqi Li
- School of Medical Technology, Beijing Institute of Technology, Beijing, China
| | - Sujie Fan
- Handan City Eye Hospital, Handan, China
| | - Jianjun Tang
- School of Agricultural Economics and Rural Development, Renmin University of China, Beijing, China.
| | - Ningli Wang
- Beijing Institute of Ophthalmology, Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China; School of Medical Technology, Beijing Institute of Technology, Beijing, China; National Institutes of Health Data Science at Peking University, Beijing, China.
| |
Collapse
|
48
|
Zicarelli F, Agarwal A, Rizzi C, Oldani M, Staurenghi G, Invernizzi A. Correlation Between Retinal Vessel Diameters and Uveitis Activity. Invest Ophthalmol Vis Sci 2023; 64:13. [PMID: 36877516 PMCID: PMC10007899 DOI: 10.1167/iovs.64.3.13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2023] Open
Abstract
Purpose To assess retinal vein and artery diameters during active and inactive intraocular inflammation in eyes with uveitis. Methods Color fundus photographs and clinical data of eyes with uveitis collected during two visits (active disease [i.e., T0] and inactive stage [i.e., T1]) were reviewed. The images were semi-automatically analyzed to obtain the central retina vein equivalent (CRVE) and central retina artery equivalent (CRAE). Changes of CRVE and CRAE from T0 to T1 were calculated, and their possible correlation with clinical data, including age, gender, ethnicity, uveitis etiology, and visual acuity, were investigated. Results Eighty-nine eyes were enrolled in the study. Both CRVE and CRAE reduced from T0 to T1 (P < 0.0001 and P = 0.01, respectively), with active inflammation being able to influence the CRVE and CRAE (P < 0.0001 and P = 0.0004, respectively) after accounting for all other variables. The degree of venular (∆V) and arteriolar (∆A) dilation was influenced only by time (P = 0.03 and P = 0.04, respectively). Best-corrected visual acuity was influenced by time and ethnicity (P = 0.003 and P = 0.0006). Conclusions CRVE and CRAE are increased in eyes with active intraocular inflammation regardless of the type of uveitis, and they decrease when the inflammation wears off.
Collapse
Affiliation(s)
- Federico Zicarelli
- Eye Clinic, Department of Biomedical and Clinical Science, Luigi Sacco Hospital, University of Milan, Milan, Italy
| | - Aniruddha Agarwal
- Eye Institute, Cleveland Clinic Abu Dhabi, Al Maryah Island, Abu Dhabi, United Arab Emirates.,Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, Ohio, United States
| | - Chiara Rizzi
- Eye Clinic, Department of Biomedical and Clinical Science, Luigi Sacco Hospital, University of Milan, Milan, Italy
| | - Marta Oldani
- Eye Clinic, Department of Biomedical and Clinical Science, Luigi Sacco Hospital, University of Milan, Milan, Italy
| | - Giovanni Staurenghi
- Eye Clinic, Department of Biomedical and Clinical Science, Luigi Sacco Hospital, University of Milan, Milan, Italy
| | - Alessandro Invernizzi
- Eye Clinic, Department of Biomedical and Clinical Science, Luigi Sacco Hospital, University of Milan, Milan, Italy.,The University of Sydney, Save Sight Institute, Discipline of Ophthalmology, Sydney Medical School, Sydney, New South Wales, Australia
| |
Collapse
|
49
|
Lin S, Ma Y, Xu Y, Lu L, He J, Zhu J, Peng Y, Yu T, Congdon N, Zou H. Artificial Intelligence in Community-Based Diabetic Retinopathy Telemedicine Screening in Urban China: Cost-effectiveness and Cost-Utility Analyses With Real-world Data. JMIR Public Health Surveill 2023; 9:e41624. [PMID: 36821353 PMCID: PMC9999255 DOI: 10.2196/41624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 11/13/2022] [Accepted: 01/12/2023] [Indexed: 02/24/2023] Open
Abstract
BACKGROUND Community-based telemedicine screening for diabetic retinopathy (DR) has been highly recommended worldwide. However, evidence from low- and middle-income countries (LMICs) on the choice between artificial intelligence (AI)-based and manual grading-based telemedicine screening is inadequate for policy making. OBJECTIVE The aim of this study was to test whether the AI model is more worthwhile than manual grading in community-based telemedicine screening for DR in the context of labor costs in urban China. METHODS We conducted cost-effectiveness and cost-utility analyses by using decision-analytic Markov models with 30 one-year cycles from a societal perspective to compare the cost, effectiveness, and utility of 2 scenarios in telemedicine screening for DR: manual grading and an AI model. Sensitivity analyses were performed. Real-world data were obtained mainly from the Shanghai Digital Eye Disease Screening Program. The main outcomes were the incremental cost-effectiveness ratio (ICER) and the incremental cost-utility ratio (ICUR). The ICUR thresholds were set as 1 and 3 times the local gross domestic product per capita. RESULTS The total expected costs for a 65-year-old resident were US $3182.50 and US $3265.40, while the total expected years without blindness were 9.80 years and 9.83 years, and the utilities were 6.748 quality-adjusted life years (QALYs) and 6.753 QALYs in the AI model and manual grading, respectively. The ICER for the AI-assisted model was US $2553.39 per year without blindness, and the ICUR was US $15,216.96 per QALY, which indicated that AI-assisted model was not cost-effective. The sensitivity analysis suggested that if there is an increase in compliance with referrals after the adoption of AI by 7.5%, an increase in on-site screening costs in manual grading by 50%, or a decrease in on-site screening costs in the AI model by 50%, then the AI model could be the dominant strategy. CONCLUSIONS Our study may provide a reference for policy making in planning community-based telemedicine screening for DR in LMICs. Our findings indicate that unless the referral compliance of patients with suspected DR increases, the adoption of the AI model may not improve the value of telemedicine screening compared to that of manual grading in LMICs. The main reason is that in the context of the low labor costs in LMICs, the direct health care costs saved by replacing manual grading with AI are less, and the screening effectiveness (QALYs and years without blindness) decreases. Our study suggests that the magnitude of the value generated by this technology replacement depends primarily on 2 aspects. The first is the extent of direct health care costs reduced by AI, and the second is the change in health care service utilization caused by AI. Therefore, our research can also provide analytical ideas for other health care sectors in their decision to use AI.
Collapse
Affiliation(s)
- Senlin Lin
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Yingyan Ma
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Yi Xu
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Lina Lu
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Jiangnan He
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Jianfeng Zhu
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Yajun Peng
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Tao Yu
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| | - Nathan Congdon
- Centre for Public Health, Queen's University Belfast, Belfast, United Kingdom.,Orbis International, New York, NY, United States.,Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Haidong Zou
- Department of Eye Disease Prevention and Control, Shanghai Eye Disease Prevention and Treatment Center/Shanghai Eye Hospital, Shanghai, China.,Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai General Hospital, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, China
| |
Collapse
|
50
|
Peeters F, Rommes S, Elen B, Gerrits N, Stalmans I, Jacob J, De Boever P. Artificial Intelligence Software for Diabetic Eye Screening: Diagnostic Performance and Impact of Stratification. J Clin Med 2023; 12:jcm12041408. [PMID: 36835942 PMCID: PMC9967595 DOI: 10.3390/jcm12041408] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Revised: 01/31/2023] [Accepted: 02/07/2023] [Indexed: 02/12/2023] Open
Abstract
AIM To evaluate the MONA.health artificial intelligence screening software for detecting referable diabetic retinopathy (DR) and diabetic macular edema (DME), including subgroup analysis. METHODS The algorithm's threshold value was fixed at the 90% sensitivity operating point on the receiver operating curve to perform the disease classification. Diagnostic performance was appraised on a private test set and publicly available datasets. Stratification analysis was executed on the private test set considering age, ethnicity, sex, insulin dependency, year of examination, camera type, image quality, and dilatation status. RESULTS The software displayed an area under the curve (AUC) of 97.28% for DR and 98.08% for DME on the private test set. The specificity and sensitivity for combined DR and DME predictions were 94.24 and 90.91%, respectively. The AUC ranged from 96.91 to 97.99% on the publicly available datasets for DR. AUC values were above 95% in all subgroups, with lower predictive values found for individuals above the age of 65 (82.51% sensitivity) and Caucasians (84.03% sensitivity). CONCLUSION We report good overall performance of the MONA.health screening software for DR and DME. The software performance remains stable with no significant deterioration of the deep learning models in any studied strata.
Collapse
Affiliation(s)
- Freya Peeters
- Department of Ophthalmology, University Hospitals Leuven, 3000 Leuven, Belgium
- Biomedical Sciences Group, Research Group Ophthalmology, Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium
- Correspondence:
| | - Stef Rommes
- MONA.health, 3060 Bertem, Belgium
- Flemish Institute for Technological Research (VITO), 2400 Mol, Belgium
| | - Bart Elen
- Flemish Institute for Technological Research (VITO), 2400 Mol, Belgium
| | - Nele Gerrits
- Flemish Institute for Technological Research (VITO), 2400 Mol, Belgium
| | - Ingeborg Stalmans
- Department of Ophthalmology, University Hospitals Leuven, 3000 Leuven, Belgium
- Biomedical Sciences Group, Research Group Ophthalmology, Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium
| | - Julie Jacob
- Department of Ophthalmology, University Hospitals Leuven, 3000 Leuven, Belgium
- Biomedical Sciences Group, Research Group Ophthalmology, Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium
| | - Patrick De Boever
- Flemish Institute for Technological Research (VITO), 2400 Mol, Belgium
- Centre for Environmental Sciences, Hasselt University, Diepenbeek, 3500 Hasselt, Belgium
| |
Collapse
|