1
|
Vidal-Alaball J, Panadés Zafra R, Escalé-Besa A, Martinez-Millana A. The artificial intelligence revolution in primary care: Challenges, dilemmas and opportunities. Aten Primaria 2024; 56:102820. [PMID: 38056048 PMCID: PMC10714322 DOI: 10.1016/j.aprim.2023.102820] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 11/08/2023] [Indexed: 12/08/2023] Open
Abstract
Artificial intelligence (AI) can be a valuable tool for primary care (PC), as, among other things, it can help healthcare professionals improve diagnostic accuracy, chronic disease management and the overall efficiency of the care they provide. It is important to emphasise that AI should not be seen as a replacement tool, but as an aid to PC professionals. Although AI is capable of processing large amounts of data and generating accurate predictions, it cannot replace the skill and expertise of professionals in clinical decision making. AI still requires the interpretation and clinical judgement of a trained healthcare professional and cannot provide the empathy and emotional support often required in healthcare.
Collapse
Affiliation(s)
- Josep Vidal-Alaball
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Sant Fruitós de Bages, Barcelona, Spain; Grup de Recerca Promoció de la Salut en l'Àmbit Rural, Gerència d'Atenció Primària i a la Comunitat de Catalunya Central, Institut Català de la Salut, Sant Fruitós de Bages, Barcelona, Spain; Facultat de Medicina, Universitat de Vic-Universitat Central de Catalunya, Vic, Barcelona, Spain; Grup de Salut Digital CAMFIC, Barcelona, Spain
| | - Robert Panadés Zafra
- Grup de Recerca Promoció de la Salut en l'Àmbit Rural, Gerència d'Atenció Primària i a la Comunitat de Catalunya Central, Institut Català de la Salut, Sant Fruitós de Bages, Barcelona, Spain; Grup de Salut Digital CAMFIC, Barcelona, Spain; Equip d'Atenció Primària d'Anoia Rural, Gerència d'Atenció Primària i a la Comunitat de Catalunya Central, Institut Català de la Salut, Jorba i Copons, Barcelona, Spain
| | - Anna Escalé-Besa
- Grup de Recerca Promoció de la Salut en l'Àmbit Rural, Gerència d'Atenció Primària i a la Comunitat de Catalunya Central, Institut Català de la Salut, Sant Fruitós de Bages, Barcelona, Spain; Grup de Salut Digital CAMFIC, Barcelona, Spain; Equip d'Atenció Primària Navàs-Balsareny, Gerència d'Atenció Primària i a la Comunitat de Catalunya Central, Institut Català de la Salut, Navàs, Barcelona, Spain.
| | - Antonio Martinez-Millana
- Grup de Salut Digital CAMFIC, Barcelona, Spain; Instituto Universitario de Investigación de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas, Universitat Politècnica de València, Valencia, Spain
| |
Collapse
|
2
|
Zhelev Z, Peters J, Rogers M, Allen M, Kijauskaite G, Seedat F, Wilkinson E, Hyde C. Test accuracy of artificial intelligence-based grading of fundus images in diabetic retinopathy screening: A systematic review. J Med Screen 2023; 30:97-112. [PMID: 36617971 PMCID: PMC10399100 DOI: 10.1177/09691413221144382] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 11/14/2022] [Accepted: 11/18/2022] [Indexed: 01/10/2023]
Abstract
OBJECTIVES To systematically review the accuracy of artificial intelligence (AI)-based systems for grading of fundus images in diabetic retinopathy (DR) screening. METHODS We searched MEDLINE, EMBASE, the Cochrane Library and the ClinicalTrials.gov from 1st January 2000 to 27th August 2021. Accuracy studies published in English were included if they met the pre-specified inclusion criteria. Selection of studies for inclusion, data extraction and quality assessment were conducted by one author with a second reviewer independently screening and checking 20% of titles. Results were analysed narratively. RESULTS Forty-three studies evaluating 15 deep learning (DL) and 4 machine learning (ML) systems were included. Nine systems were evaluated in a single study each. Most studies were judged to be at high or unclear risk of bias in at least one QUADAS-2 domain. Sensitivity for referable DR and higher grades was ≥85% while specificity varied and was <80% for all ML systems and in 6/31 studies evaluating DL systems. Studies reported high accuracy for detection of ungradable images, but the latter were analysed and reported inconsistently. Seven studies reported that AI was more sensitive but less specific than human graders. CONCLUSIONS AI-based systems are more sensitive than human graders and could be safe to use in clinical practice but have variable specificity. However, for many systems evidence is limited, at high risk of bias and may not generalise across settings. Therefore, pre-implementation assessment in the target clinical pathway is essential to obtain reliable and applicable accuracy estimates.
Collapse
Affiliation(s)
- Zhivko Zhelev
- Exeter Test Group, University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Jaime Peters
- Exeter Test Group, University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Morwenna Rogers
- NIHR ARC South West Peninsula (PenARC), University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Michael Allen
- University of Exeter Medical School, University of Exeter, Exeter, UK
| | | | | | | | - Christopher Hyde
- Exeter Test Group, University of Exeter Medical School, University of Exeter, Exeter, UK
| |
Collapse
|
3
|
Nadeem MW, Goh HG, Hussain M, Liew SY, Andonovic I, Khan MA. Deep Learning for Diabetic Retinopathy Analysis: A Review, Research Challenges, and Future Directions. SENSORS (BASEL, SWITZERLAND) 2022; 22:6780. [PMID: 36146130 PMCID: PMC9505428 DOI: 10.3390/s22186780] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Revised: 08/02/2022] [Accepted: 08/08/2022] [Indexed: 05/12/2023]
Abstract
Deep learning (DL) enables the creation of computational models comprising multiple processing layers that learn data representations at multiple levels of abstraction. In the recent past, the use of deep learning has been proliferating, yielding promising results in applications across a growing number of fields, most notably in image processing, medical image analysis, data analysis, and bioinformatics. DL algorithms have also had a significant positive impact through yielding improvements in screening, recognition, segmentation, prediction, and classification applications across different domains of healthcare, such as those concerning the abdomen, cardiac, pathology, and retina. Given the extensive body of recent scientific contributions in this discipline, a comprehensive review of deep learning developments in the domain of diabetic retinopathy (DR) analysis, viz., screening, segmentation, prediction, classification, and validation, is presented here. A critical analysis of the relevant reported techniques is carried out, and the associated advantages and limitations highlighted, culminating in the identification of research gaps and future challenges that help to inform the research community to develop more efficient, robust, and accurate DL models for the various challenges in the monitoring and diagnosis of DR.
Collapse
Affiliation(s)
- Muhammad Waqas Nadeem
- Faculty of Information and Communication Technology (FICT), Universiti Tunku Abdul Rahman (UTAR), Kampar 31900, Malaysia
| | - Hock Guan Goh
- Faculty of Information and Communication Technology (FICT), Universiti Tunku Abdul Rahman (UTAR), Kampar 31900, Malaysia
| | - Muzammil Hussain
- Department of Computer Science, School of Systems and Technology, University of Management and Technology, Lahore 54000, Pakistan
| | - Soung-Yue Liew
- Faculty of Information and Communication Technology (FICT), Universiti Tunku Abdul Rahman (UTAR), Kampar 31900, Malaysia
| | - Ivan Andonovic
- Department of Electronic and Electrical Engineering, Royal College Building, University of Strathclyde, 204 George St., Glasgow G1 1XW, UK
| | - Muhammad Adnan Khan
- Pattern Recognition and Machine Learning Lab, Department of Software, Gachon University, Seongnam 13557, Korea
- Faculty of Computing, Riphah School of Computing and Innovation, Riphah International University, Lahore Campus, Lahore 54000, Pakistan
| |
Collapse
|
4
|
Zhang WF, Li DH, Wei QJ, Ding DY, Meng LH, Wang YL, Zhao XY, Chen YX. The Validation of Deep Learning-Based Grading Model for Diabetic Retinopathy. Front Med (Lausanne) 2022; 9:839088. [PMID: 35652075 PMCID: PMC9148973 DOI: 10.3389/fmed.2022.839088] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2021] [Accepted: 04/08/2022] [Indexed: 12/26/2022] Open
Abstract
Purpose To evaluate the performance of a deep learning (DL)-based artificial intelligence (AI) hierarchical diagnosis software, EyeWisdom V1 for diabetic retinopathy (DR). Materials and Methods The prospective study was a multicenter, double-blind, and self-controlled clinical trial. Non-dilated posterior pole fundus images were evaluated by ophthalmologists and EyeWisdom V1, respectively. The diagnosis of manual grading was considered as the gold standard. Primary evaluation index (sensitivity and specificity) and secondary evaluation index like positive predictive values (PPV), negative predictive values (NPV), etc., were calculated to evaluate the performance of EyeWisdom V1. Results A total of 1,089 fundus images from 630 patients were included, with a mean age of (56.52 ± 11.13) years. For any DR, the sensitivity, specificity, PPV, and NPV were 98.23% (95% CI 96.93-99.08%), 74.45% (95% CI 69.95-78.60%), 86.38% (95% CI 83.76-88.72%), and 96.23% (95% CI 93.50-98.04%), respectively; For sight-threatening DR (STDR, severe non-proliferative DR or worse), the above indicators were 80.47% (95% CI 75.07-85.14%), 97.96% (95% CI 96.75-98.81%), 92.38% (95% CI 88.07-95.50%), and 94.23% (95% CI 92.46-95.68%); For referral DR (moderate non-proliferative DR or worse), the sensitivity and specificity were 92.96% (95% CI 90.66-94.84%) and 93.32% (95% CI 90.65-95.42%), with the PPV of 94.93% (95% CI 92.89-96.53%) and the NPV of 90.78% (95% CI 87.81-93.22%). The kappa score of EyeWisdom V1 was 0.860 (0.827-0.890) with the AUC of 0.958 for referral DR. Conclusion The EyeWisdom V1 could provide reliable DR grading and referral recommendation based on the fundus images of diabetics.
Collapse
Affiliation(s)
- Wen-fei Zhang
- Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China
- Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | | | - Qi-jie Wei
- Visionary Intelligence Ltd., Beijing, China
| | | | - Li-hui Meng
- Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China
- Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Yue-lin Wang
- Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China
- Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Xin-yu Zhao
- Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China
- Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - You-xin Chen
- Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China
- Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| |
Collapse
|
5
|
Untangling Computer-Aided Diagnostic System for Screening Diabetic Retinopathy Based on Deep Learning Techniques. SENSORS 2022; 22:s22051803. [PMID: 35270949 PMCID: PMC8914671 DOI: 10.3390/s22051803] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 02/16/2022] [Accepted: 02/17/2022] [Indexed: 01/27/2023]
Abstract
Diabetic Retinopathy (DR) is a predominant cause of visual impairment and loss. Approximately 285 million worldwide population is affected with diabetes, and one-third of these patients have symptoms of DR. Specifically, it tends to affect the patients with 20 years or more with diabetes, but it can be reduced by early detection and proper treatment. Diagnosis of DR by using manual methods is a time-consuming and expensive task which involves trained ophthalmologists to observe and evaluate DR using digital fundus images of the retina. This study aims to systematically find and analyze high-quality research work for the diagnosis of DR using deep learning approaches. This research comprehends the DR grading, staging protocols and also presents the DR taxonomy. Furthermore, identifies, compares, and investigates the deep learning-based algorithms, techniques, and, methods for classifying DR stages. Various publicly available dataset used for deep learning have also been analyzed and dispensed for descriptive and empirical understanding for real-time DR applications. Our in-depth study shows that in the last few years there has been an increasing inclination towards deep learning approaches. 35% of the studies have used Convolutional Neural Networks (CNNs), 26% implemented the Ensemble CNN (ECNN) and, 13% Deep Neural Networks (DNN) are amongst the most used algorithms for the DR classification. Thus using the deep learning algorithms for DR diagnostics have future research potential for DR early detection and prevention based solution.
Collapse
|
6
|
Baget-Bernaldiz M, Pedro RA, Santos-Blanco E, Navarro-Gil R, Valls A, Moreno A, Rashwan HA, Puig D. Testing a Deep Learning Algorithm for Detection of Diabetic Retinopathy in a Spanish Diabetic Population and with MESSIDOR Database. Diagnostics (Basel) 2021; 11:diagnostics11081385. [PMID: 34441319 PMCID: PMC8394605 DOI: 10.3390/diagnostics11081385] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Revised: 07/16/2021] [Accepted: 07/28/2021] [Indexed: 12/23/2022] Open
Abstract
Background: The aim of the present study was to test our deep learning algorithm (DLA) by reading the retinographies. Methods: We tested our DLA built on convolutional neural networks in 14,186 retinographies from our population and 1200 images extracted from MESSIDOR. The retinal images were graded both by the DLA and independently by four retina specialists. Results of the DLA were compared according to accuracy (ACC), sensitivity (S), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC), distinguishing between identification of any type of DR (any DR) and referable DR (RDR). Results: The results of testing the DLA for identifying any DR in our population were: ACC = 99.75, S = 97.92, SP = 99.91, PPV = 98.92, NPV = 99.82, and AUC = 0.983. When detecting RDR, the results were: ACC = 99.66, S = 96.7, SP = 99.92, PPV = 99.07, NPV = 99.71, and AUC = 0.988. The results of testing the DLA for identifying any DR with MESSIDOR were: ACC = 94.79, S = 97.32, SP = 94.57, PPV = 60.93, NPV = 99.75, and AUC = 0.959. When detecting RDR, the results were: ACC = 98.78, S = 94.64, SP = 99.14, PPV = 90.54, NPV = 99.53, and AUC = 0.968. Conclusions: Our DLA performed well, both in detecting any DR and in classifying those eyes with RDR in a sample of retinographies of type 2 DM patients in our population and the MESSIDOR database.
Collapse
Affiliation(s)
- Marc Baget-Bernaldiz
- Ophthalmology Service, Hospital Universitat Sant Joan, Institut de Investigació Sanitària Pere Virgili [IISPV], Universitat Rovira & Virgili, 43204 Reus, Spain; (M.B.-B.); (E.S.-B.); (R.N.-G.)
| | - Romero-Aroca Pedro
- Ophthalmology Service, Hospital Universitat Sant Joan, Institut de Investigació Sanitària Pere Virgili [IISPV], Universitat Rovira & Virgili, 43204 Reus, Spain; (M.B.-B.); (E.S.-B.); (R.N.-G.)
- Correspondence:
| | - Esther Santos-Blanco
- Ophthalmology Service, Hospital Universitat Sant Joan, Institut de Investigació Sanitària Pere Virgili [IISPV], Universitat Rovira & Virgili, 43204 Reus, Spain; (M.B.-B.); (E.S.-B.); (R.N.-G.)
| | - Raul Navarro-Gil
- Ophthalmology Service, Hospital Universitat Sant Joan, Institut de Investigació Sanitària Pere Virgili [IISPV], Universitat Rovira & Virgili, 43204 Reus, Spain; (M.B.-B.); (E.S.-B.); (R.N.-G.)
| | - Aida Valls
- Department of Computer Engineering and Mathematics, Universitat Rovira & Virgili, 43204 Reus, Spain; (A.V.); (A.M.); (H.A.R.); (D.P.)
| | - Antonio Moreno
- Department of Computer Engineering and Mathematics, Universitat Rovira & Virgili, 43204 Reus, Spain; (A.V.); (A.M.); (H.A.R.); (D.P.)
| | - Hatem A. Rashwan
- Department of Computer Engineering and Mathematics, Universitat Rovira & Virgili, 43204 Reus, Spain; (A.V.); (A.M.); (H.A.R.); (D.P.)
| | - Domenec Puig
- Department of Computer Engineering and Mathematics, Universitat Rovira & Virgili, 43204 Reus, Spain; (A.V.); (A.M.); (H.A.R.); (D.P.)
| |
Collapse
|
7
|
Lee AY, Yanagihara RT, Lee CS, Blazes M, Jung HC, Chee YE, Gencarella MD, Gee H, Maa AY, Cockerham GC, Lynch M, Boyko EJ. Multicenter, Head-to-Head, Real-World Validation Study of Seven Automated Artificial Intelligence Diabetic Retinopathy Screening Systems. Diabetes Care 2021; 44:1168-1175. [PMID: 33402366 PMCID: PMC8132324 DOI: 10.2337/dc20-1877] [Citation(s) in RCA: 74] [Impact Index Per Article: 24.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 10/25/2020] [Indexed: 02/03/2023]
Abstract
OBJECTIVE With rising global prevalence of diabetic retinopathy (DR), automated DR screening is needed for primary care settings. Two automated artificial intelligence (AI)-based DR screening algorithms have U.S. Food and Drug Administration (FDA) approval. Several others are under consideration while in clinical use in other countries, but their real-world performance has not been evaluated systematically. We compared the performance of seven automated AI-based DR screening algorithms (including one FDA-approved algorithm) against human graders when analyzing real-world retinal imaging data. RESEARCH DESIGN AND METHODS This was a multicenter, noninterventional device validation study evaluating a total of 311,604 retinal images from 23,724 veterans who presented for teleretinal DR screening at the Veterans Affairs (VA) Puget Sound Health Care System (HCS) or Atlanta VA HCS from 2006 to 2018. Five companies provided seven algorithms, including one with FDA approval, that independently analyzed all scans, regardless of image quality. The sensitivity/specificity of each algorithm when classifying images as referable DR or not were compared with original VA teleretinal grades and a regraded arbitrated data set. Value per encounter was estimated. RESULTS Although high negative predictive values (82.72-93.69%) were observed, sensitivities varied widely (50.98-85.90%). Most algorithms performed no better than humans against the arbitrated data set, but two achieved higher sensitivities, and one yielded comparable sensitivity (80.47%, P = 0.441) and specificity (81.28%, P = 0.195). Notably, one had lower sensitivity (74.42%) for proliferative DR (P = 9.77 × 10-4) than the VA teleretinal graders. Value per encounter varied at $15.14-$18.06 for ophthalmologists and $7.74-$9.24 for optometrists. CONCLUSIONS The DR screening algorithms showed significant performance differences. These results argue for rigorous testing of all such algorithms on real-world data before clinical implementation.
Collapse
Affiliation(s)
- Aaron Y Lee
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA .,Department of Ophthalmology, Puget Sound Veteran Affairs, Seattle, WA.,eScience Institute, University of Washington, Seattle, WA
| | - Ryan T Yanagihara
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA
| | - Cecilia S Lee
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA.,Department of Ophthalmology, Puget Sound Veteran Affairs, Seattle, WA
| | - Marian Blazes
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA
| | - Hoon C Jung
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA.,Department of Ophthalmology, Puget Sound Veteran Affairs, Seattle, WA
| | - Yewlin E Chee
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA
| | - Michael D Gencarella
- Department of Ophthalmology, University of Washington School of Medicine, Seattle, WA
| | - Harry Gee
- Office of Information and Technology, Clinical Imaging, Seattle, WA
| | - April Y Maa
- Department of Ophthalmology, Emory University School of Medicine, Atlanta, GA.,Regional Telehealth Services, Veterans Affairs Southeast Network Veterans Integrated Service Networks (VISN) 7, Duluth, GA
| | - Glenn C Cockerham
- Veterans Health Administration, Specialty Care Services, Washington, DC.,Ophthalmology Service, Stanford University School of Medicine, Palo Alto, CA
| | - Mary Lynch
- Department of Ophthalmology, Emory University School of Medicine, Atlanta, GA.,Ophthalmology Section, Atlanta Veterans Affairs Medical Center, Atlanta, GA
| | - Edward J Boyko
- Seattle Epidemiologic Research and Information Center, Department of Veterans Affairs Medical Center, Seattle, WA.,Department of Medicine, University of Washington, Seattle, WA
| |
Collapse
|