1
|
Wang Y, Han X, Li C, Luo L, Yin Q, Zhang J, Peng G, Shi D, He M. Impact of Gold-Standard Label Errors on Evaluating Performance of Deep Learning Models in Diabetic Retinopathy Screening: Nationwide Real-World Validation Study. J Med Internet Res 2024; 26:e52506. [PMID: 39141915 PMCID: PMC11358665 DOI: 10.2196/52506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 12/30/2023] [Accepted: 03/22/2024] [Indexed: 08/16/2024] Open
Abstract
BACKGROUND For medical artificial intelligence (AI) training and validation, human expert labels are considered the gold standard that represents the correct answers or desired outputs for a given data set. These labels serve as a reference or benchmark against which the model's predictions are compared. OBJECTIVE This study aimed to assess the accuracy of a custom deep learning (DL) algorithm on classifying diabetic retinopathy (DR) and further demonstrate how label errors may contribute to this assessment in a nationwide DR-screening program. METHODS Fundus photographs from the Lifeline Express, a nationwide DR-screening program, were analyzed to identify the presence of referable DR using both (1) manual grading by National Health Service England-certificated graders and (2) a DL-based DR-screening algorithm with validated good lab performance. To assess the accuracy of labels, a random sample of images with disagreement between the DL algorithm and the labels was adjudicated by ophthalmologists who were masked to the previous grading results. The error rates of labels in this sample were then used to correct the number of negative and positive cases in the entire data set, serving as postcorrection labels. The DL algorithm's performance was evaluated against both pre- and postcorrection labels. RESULTS The analysis included 736,083 images from 237,824 participants. The DL algorithm exhibited a gap between the real-world performance and the lab-reported performance in this nationwide data set, with a sensitivity increase of 12.5% (from 79.6% to 92.5%, P<.001) and a specificity increase of 6.9% (from 91.6% to 98.5%, P<.001). In the random sample, 63.6% (560/880) of negative images and 5.2% (140/2710) of positive images were misclassified in the precorrection human labels. High myopia was the primary reason for misclassifying non-DR images as referable DR images, while laser spots were predominantly responsible for misclassified referable cases. The estimated label error rate for the entire data set was 1.2%. The label correction was estimated to bring about a 12.5% enhancement in the estimated sensitivity of the DL algorithm (P<.001). CONCLUSIONS Label errors based on human image grading, although in a small percentage, can significantly affect the performance evaluation of DL algorithms in real-world DR screening.
Collapse
Affiliation(s)
- Yueye Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, China (Hong Kong)
| | - Xiaotong Han
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Cong Li
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Lixia Luo
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Qiuxia Yin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Jian Zhang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Guankai Peng
- Guangzhou Vision Tech Medical Technology Co, Ltd, Guangzhou, China
| | - Danli Shi
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, China (Hong Kong)
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, China (Hong Kong)
| | - Mingguang He
- School of Optometry, The Hong Kong Polytechnic University, Kowloon, China (Hong Kong)
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, China (Hong Kong)
- Centre for Eye and Vision Research, Hong Kong, China (Hong Kong)
| |
Collapse
|
2
|
Shou BL, Venkatesh K, Chen C, Ghidey R, Lee JH, Wang J, Channa R, Wolf RM, Abramoff MD, Liu TYA. Risk Factors for Nondiagnostic Imaging in a Real-World Deployment of Artificial Intelligence Diabetic Retinal Examinations in an Integrated Healthcare System: Maximizing Workflow Efficiency Through Predictive Dilation. J Diabetes Sci Technol 2024; 18:302-308. [PMID: 37798955 DOI: 10.1177/19322968231201654] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/07/2023]
Abstract
OBJECTIVE In the pivotal clinical trial that led to Food and Drug Administration De Novo "approval" of the first fully autonomous artificial intelligence (AI) diabetic retinal disease diagnostic system, a reflexive dilation protocol was used. Using real-world deployment data before implementation of reflexive dilation, we identified factors associated with nondiagnostic results. These factors allow a novel predictive dilation workflow, where patients most likely to benefit from pharmacologic dilation are dilated a priori to maximize efficiency and patient satisfaction. METHODS Retrospective review of patients who were assessed with autonomous AI at Johns Hopkins Medicine (8/2020 to 5/2021). We constructed a multivariable logistic regression model for nondiagnostic results to compare characteristics of patients with and without diagnostic results, using adjusted odds ratio (aOR). P < .05 was considered statistically significant. RESULTS Of 241 patients (59% female; median age = 59), 123 (51%) had nondiagnostic results. In multivariable analysis, type 1 diabetes (T1D, aOR = 5.82, 95% confidence interval [CI]: 1.45-23.40, P = .01), smoking (aOR = 2.86, 95% CI: 1.36-5.99, P = .005), and age (every 10-year increase, aOR = 2.12, 95% CI: 1.62-2.77, P < .001) were associated with nondiagnostic results. Following feature elimination, a predictive model was created using T1D, smoking, age, race, sex, and hypertension as inputs. The model showed an area under the receiver-operator characteristics curve of 0.76 in five-fold cross-validation. CONCLUSIONS We used factors associated with nondiagnostic results to design a novel, predictive dilation workflow, where patients most likely to benefit from pharmacologic dilation are dilated a priori. This new workflow has the potential to be more efficient than reflexive dilation, thus maximizing the number of at-risk patients receiving their diabetic retinal examinations.
Collapse
Affiliation(s)
- Benjamin L Shou
- School of Medicine, The Johns Hopkins University, Baltimore, MD, USA
| | - Kesavan Venkatesh
- Whiting School of Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Chang Chen
- Bloomberg School of Public Health, The Johns Hopkins University, Baltimore, MD, USA
| | - Ronel Ghidey
- Bloomberg School of Public Health, The Johns Hopkins University, Baltimore, MD, USA
| | - Jae Hyoung Lee
- Bloomberg School of Public Health, The Johns Hopkins University, Baltimore, MD, USA
| | - Jiangxia Wang
- Bloomberg School of Public Health, The Johns Hopkins University, Baltimore, MD, USA
| | - Roomasa Channa
- Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, WI, USA
| | - Risa M Wolf
- Department of Pediatrics, Division of Pediatric Endocrinology, The Johns Hopkins University, Baltimore, MD, USA
| | - Michael D Abramoff
- Department of Ophthalmology and Visual Sciences, The University of Iowa, Iowa City, IA, USA
| | - T Y Alvin Liu
- Wilmer Eye Institute, The Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
3
|
Zhang D, Wu C, Yang Z, Yin H, Liu Y, Li W, Huang H, Jin Z. The application of artificial intelligence in EUS. Endosc Ultrasound 2024; 13:65-75. [PMID: 38947752 PMCID: PMC11213611 DOI: 10.1097/eus.0000000000000053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/02/2024] Open
Abstract
Artificial intelligence (AI) is an epoch-making technology, among which the 2 most advanced parts are machine learning and deep learning algorithms that have been further developed by machine learning, and it has been partially applied to assist EUS diagnosis. AI-assisted EUS diagnosis has been reported to have great value in the diagnosis of pancreatic tumors and chronic pancreatitis, gastrointestinal stromal tumors, esophageal early cancer, biliary tract, and liver lesions. The application of AI in EUS diagnosis still has some urgent problems to be solved. First, the development of sensitive AI diagnostic tools requires a large amount of high-quality training data. Second, there is overfitting and bias in the current AI algorithms, leading to poor diagnostic reliability. Third, the value of AI still needs to be determined in prospective studies. Fourth, the ethical risks of AI need to be considered and avoided.
Collapse
Affiliation(s)
- Deyu Zhang
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| | - Chang Wu
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| | - Zhenghui Yang
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| | - Hua Yin
- Department of Gastroenterology, General Hospital of Ningxia Medical University, Yinchuan 750004, Ningxia Hui Autonomous Region, China
| | - Yue Liu
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| | - Wanshun Li
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| | - Haojie Huang
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| | - Zhendong Jin
- Department of Gastroenterology, Changhai hospital, Naval Medical University, Shanghai 200433, China
| |
Collapse
|
4
|
Wolf RM, Channa R, Liu TYA, Zehra A, Bromberger L, Patel D, Ananthakrishnan A, Brown EA, Prichett L, Lehmann HP, Abramoff MD. Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial. Nat Commun 2024; 15:421. [PMID: 38212308 PMCID: PMC10784572 DOI: 10.1038/s41467-023-44676-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 12/21/2023] [Indexed: 01/13/2024] Open
Abstract
Diabetic retinopathy can be prevented with screening and early detection. We hypothesized that autonomous artificial intelligence (AI) diabetic eye exams at the point-of-care would increase diabetic eye exam completion rates in a racially and ethnically diverse youth population. AI for Children's diabetiC Eye ExamS (NCT05131451) is a parallel randomized controlled trial that randomized youth (ages 8-21 years) with type 1 and type 2 diabetes to intervention (autonomous artificial intelligence diabetic eye exam at the point of care), or control (scripted eye care provider referral and education) in an academic pediatric diabetes center. The primary outcome was diabetic eye exam completion rate within 6 months. The secondary outcome was the proportion of participants who completed follow-through with an eye care provider if deemed appropriate. Diabetic eye exam completion rate was significantly higher (100%, 95%CI: 95.5%, 100%) in the intervention group (n = 81) than the control group (n = 83) (22%, 95%CI: 14.2%, 32.4%)(p < 0.001). In the intervention arm, 25/81 participants had an abnormal result, of whom 64% (16/25) completed follow-through with an eye care provider, compared to 22% in the control arm (p < 0.001). Autonomous AI increases diabetic eye exam completion rates in youth with diabetes.
Collapse
Affiliation(s)
- Risa M Wolf
- Department of Pediatrics, Division of Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, USA.
| | - Roomasa Channa
- Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, WI, USA
| | - T Y Alvin Liu
- Wilmer Eye Institute at the Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Anum Zehra
- Department of Pediatrics, Division of Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Lee Bromberger
- Department of Pediatrics, Division of Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Dhruva Patel
- Department of Pediatrics, Division of Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | | | - Elizabeth A Brown
- Department of Pediatrics, Division of Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Laura Prichett
- Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core, Baltimore, MD, USA
| | - Harold P Lehmann
- Section on Biomedical Informatics and Data Science, Johns Hopkins University, Baltimore, MD, USA
| | - Michael D Abramoff
- Department of Ophthalmology and Visual Sciences, The University of Iowa, Iowa City, IA, USA
- Digital Diagnostics Inc, Coralville, IA, USA
- Iowa City VA Medical Center, Iowa City, IA, USA
- Department of Biomedical Engineering, The University of Iowa, Iowa City, IA, USA
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| |
Collapse
|
5
|
Dow ER, Khan NC, Chen KM, Mishra K, Perera C, Narala R, Basina M, Dang J, Kim M, Levine M, Phadke A, Tan M, Weng K, Do DV, Moshfeghi DM, Mahajan VB, Mruthyunjaya P, Leng T, Myung D. AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy. OPHTHALMOLOGY SCIENCE 2023; 3:100330. [PMID: 37449051 PMCID: PMC10336195 DOI: 10.1016/j.xops.2023.100330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Revised: 05/04/2023] [Accepted: 05/08/2023] [Indexed: 07/18/2023]
Abstract
Objective Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases. Design Prospective cohort study and retrospective analysis. Participants Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic. Methods Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist. Main Outcome Measures Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists. Results The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%-100%) but lower specificity (60.3% specificity; 95% CI, 47.7%-72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%-88.3%; 96.9% specificity; 95% CI, 93.5%-100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%-100%) and a specificity of 98.2% (95% CI, 94.6%-100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters. Conclusions Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy. Financial Disclosures Proprietary or commercial disclosure may be found after the references.
Collapse
Affiliation(s)
- Eliot R. Dow
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
- Veterans Affairs Palo Alto Health Care System, Palo Alto, California
| | - Nergis C. Khan
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Karen M. Chen
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Kapil Mishra
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Chandrashan Perera
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Ramsudha Narala
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Marina Basina
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Jimmy Dang
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Michael Kim
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Marcie Levine
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Anuradha Phadke
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Marilyn Tan
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Kirsti Weng
- Stanford Healthcare, Stanford University, Palo Alto, California
| | - Diana V. Do
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Darius M. Moshfeghi
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Vinit B. Mahajan
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
- Veterans Affairs Palo Alto Health Care System, Palo Alto, California
| | - Prithvi Mruthyunjaya
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - Theodore Leng
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| | - David Myung
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, California
| |
Collapse
|
6
|
Li L, Lin D, Lin Z, Li M, Lian Z, Zhao L, Wu X, Liu L, Liu J, Wei X, Luo M, Zeng D, Yan A, Iao WC, Shang Y, Xu F, Xiang W, He M, Fu Z, Wang X, Deng Y, Fan X, Ye Z, Wei M, Zhang J, Liu B, Li J, Ding X, Lin H. DeepQuality improves infant retinopathy screening. NPJ Digit Med 2023; 6:192. [PMID: 37845275 PMCID: PMC10579317 DOI: 10.1038/s41746-023-00943-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 10/05/2023] [Indexed: 10/18/2023] Open
Abstract
Image quality variation is a prominent cause of performance degradation for intelligent disease diagnostic models in clinical applications. Image quality issues are particularly prominent in infantile fundus photography due to poor patient cooperation, which poses a high risk of misdiagnosis. Here, we developed a deep learning-based image quality assessment and enhancement system (DeepQuality) for infantile fundus images to improve infant retinopathy screening. DeepQuality can accurately detect various quality defects concerning integrity, illumination, and clarity with area under the curve (AUC) values ranging from 0.933 to 0.995. It can also comprehensively score the overall quality of each fundus photograph. By analyzing 2,015,758 infantile fundus photographs from real-world settings using DeepQuality, we found that 58.3% of them had varying degrees of quality defects, and large variations were observed among different regions and categories of hospitals. Additionally, DeepQuality provides quality enhancement based on the results of quality assessment. After quality enhancement, the performance of retinopathy of prematurity (ROP) diagnosis of clinicians was significantly improved. Moreover, the integration of DeepQuality and AI diagnostic models can effectively improve the model performance for detecting ROP. This study may be an important reference for the future development of other image-based intelligent disease screening systems.
Collapse
Affiliation(s)
- Longhui Li
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Duoru Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
| | - Zhenzhe Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Mingyuan Li
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Zhangkai Lian
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Lanqin Zhao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Xiaohang Wu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Lixue Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Jiali Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Xiaoyue Wei
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Mingjie Luo
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Danqi Zeng
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Anqi Yan
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Wai Cheng Iao
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Yuanjun Shang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Fabao Xu
- Department of Ophthalmology, Qilu Hospital, Shandong University, Jinan, Shandong, China
| | - Wei Xiang
- Department of Clinical Laboratory Medicine, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, China
| | - Muchen He
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Zhe Fu
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Xueyu Wang
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Yaru Deng
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Xinyan Fan
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Zhijun Ye
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Meirong Wei
- Department of Ophthalmology, Maternal and Children's Hospital, Liuzhou, Guangxi, China
| | - Jianping Zhang
- Department of Ophthalmology, Maternal and Children's Hospital, Liuzhou, Guangxi, China
| | - Baohai Liu
- Department of Ophthalmology, Maternal and Children's Hospital, Linyi, Shandong, China
| | - Jianqiao Li
- Department of Ophthalmology, Qilu Hospital, Shandong University, Jinan, Shandong, China
| | - Xiaoyan Ding
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China
| | - Haotian Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
- Hainan Eye Hospital and Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Haikou, Hainan, China.
- Center for Precision Medicine and Department of Genetics and Biomedical Informatics, Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, Guangdong, China.
| |
Collapse
|
7
|
Abdulazeem H, Whitelaw S, Schauberger G, Klug SJ. A systematic review of clinical health conditions predicted by machine learning diagnostic and prognostic models trained or validated using real-world primary health care data. PLoS One 2023; 18:e0274276. [PMID: 37682909 PMCID: PMC10491005 DOI: 10.1371/journal.pone.0274276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 08/29/2023] [Indexed: 09/10/2023] Open
Abstract
With the advances in technology and data science, machine learning (ML) is being rapidly adopted by the health care sector. However, there is a lack of literature addressing the health conditions targeted by the ML prediction models within primary health care (PHC) to date. To fill this gap in knowledge, we conducted a systematic review following the PRISMA guidelines to identify health conditions targeted by ML in PHC. We searched the Cochrane Library, Web of Science, PubMed, Elsevier, BioRxiv, Association of Computing Machinery (ACM), and IEEE Xplore databases for studies published from January 1990 to January 2022. We included primary studies addressing ML diagnostic or prognostic predictive models that were supplied completely or partially by real-world PHC data. Studies selection, data extraction, and risk of bias assessment using the prediction model study risk of bias assessment tool were performed by two investigators. Health conditions were categorized according to international classification of diseases (ICD-10). Extracted data were analyzed quantitatively. We identified 106 studies investigating 42 health conditions. These studies included 207 ML prediction models supplied by the PHC data of 24.2 million participants from 19 countries. We found that 92.4% of the studies were retrospective and 77.3% of the studies reported diagnostic predictive ML models. A majority (76.4%) of all the studies were for models' development without conducting external validation. Risk of bias assessment revealed that 90.8% of the studies were of high or unclear risk of bias. The most frequently reported health conditions were diabetes mellitus (19.8%) and Alzheimer's disease (11.3%). Our study provides a summary on the presently available ML prediction models within PHC. We draw the attention of digital health policy makers, ML models developer, and health care professionals for more future interdisciplinary research collaboration in this regard.
Collapse
Affiliation(s)
- Hebatullah Abdulazeem
- Chair of Epidemiology, Department of Sport and Health Sciences, Technical University of Munich (TUM), Munich, Germany
| | - Sera Whitelaw
- Faculty of Medicine and Health Sciences, McGill University, Montreal, Quebec, Canada
| | - Gunther Schauberger
- Chair of Epidemiology, Department of Sport and Health Sciences, Technical University of Munich (TUM), Munich, Germany
| | - Stefanie J. Klug
- Chair of Epidemiology, Department of Sport and Health Sciences, Technical University of Munich (TUM), Munich, Germany
| |
Collapse
|
8
|
Zhelev Z, Peters J, Rogers M, Allen M, Kijauskaite G, Seedat F, Wilkinson E, Hyde C. Test accuracy of artificial intelligence-based grading of fundus images in diabetic retinopathy screening: A systematic review. J Med Screen 2023; 30:97-112. [PMID: 36617971 PMCID: PMC10399100 DOI: 10.1177/09691413221144382] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 11/14/2022] [Accepted: 11/18/2022] [Indexed: 01/10/2023]
Abstract
OBJECTIVES To systematically review the accuracy of artificial intelligence (AI)-based systems for grading of fundus images in diabetic retinopathy (DR) screening. METHODS We searched MEDLINE, EMBASE, the Cochrane Library and the ClinicalTrials.gov from 1st January 2000 to 27th August 2021. Accuracy studies published in English were included if they met the pre-specified inclusion criteria. Selection of studies for inclusion, data extraction and quality assessment were conducted by one author with a second reviewer independently screening and checking 20% of titles. Results were analysed narratively. RESULTS Forty-three studies evaluating 15 deep learning (DL) and 4 machine learning (ML) systems were included. Nine systems were evaluated in a single study each. Most studies were judged to be at high or unclear risk of bias in at least one QUADAS-2 domain. Sensitivity for referable DR and higher grades was ≥85% while specificity varied and was <80% for all ML systems and in 6/31 studies evaluating DL systems. Studies reported high accuracy for detection of ungradable images, but the latter were analysed and reported inconsistently. Seven studies reported that AI was more sensitive but less specific than human graders. CONCLUSIONS AI-based systems are more sensitive than human graders and could be safe to use in clinical practice but have variable specificity. However, for many systems evidence is limited, at high risk of bias and may not generalise across settings. Therefore, pre-implementation assessment in the target clinical pathway is essential to obtain reliable and applicable accuracy estimates.
Collapse
Affiliation(s)
- Zhivko Zhelev
- Exeter Test Group, University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Jaime Peters
- Exeter Test Group, University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Morwenna Rogers
- NIHR ARC South West Peninsula (PenARC), University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Michael Allen
- University of Exeter Medical School, University of Exeter, Exeter, UK
| | | | | | | | - Christopher Hyde
- Exeter Test Group, University of Exeter Medical School, University of Exeter, Exeter, UK
| |
Collapse
|
9
|
Mirabedini S, Musavi H, Makhlough A, Hashemi-Sooteh MB, Zargari M. Association of S19W polymorphism in APOA5 gene and serum lipid levels in patients with type 2 diabetic nephropathy. Horm Mol Biol Clin Investig 2023; 44:243-249. [PMID: 36855913 DOI: 10.1515/hmbci-2022-0056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Accepted: 02/11/2023] [Indexed: 03/02/2023]
Abstract
OBJECTIVES Type 2 diabetic Mellitus (T2DM) is the most common systemic and endocrine disease in humans, and diabetic nephropathy is one of the most serious complications of this disorder. The polymorphisms in the apolipoprotein A5 (ApoA5) gene are strongly related to hypertriglyceridemia and are considered a predisposing factor for diabetic nephropathy. The current study proposed to examine the association of APOA5-S19W polymorphism with serum lipids levels in patients with type 2 diabetic nephropathy in Mazandaran province. METHODS This case-control study was designed to determine the association of APOA5-S19W polymorphism with plasma lipid profile in 161 T2DM patients with nephropathy (DN+), without nephropathy (DN-), and in 58 healthy individuals. Lipid profile values were measured using Pars Azmoun commercial kits. S19W variant, one of the polymorphisms of the APOA5 gene, was determined by PCR-restriction fragment length polymorphism (PCR-RFLP) and Taq1 restriction enzyme. RESULTS In comparison between the three groups, DN+ had a higher mean TG than DN- and the control group (p<0.001). The incidence of the G allele in DN+ was not significant compared to groups of DN-. Comparing the relationship between the mean of biochemical variables with CC and CG genotypes showed that the mean level of TG in people with CC genotype was increased compared to people with CG genotype in diabetic patients. However, this increase was not significant (p=0.19). CONCLUSIONS There was no association between SNP APOA5 S19W and serum lipids in diabetic patients with and without nephropathy.
Collapse
Affiliation(s)
- Shivasadat Mirabedini
- Department of Clinical Biochemistry and Medical Genetics, Molecular and Cell Biology Research Center, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran
| | - Hadis Musavi
- Department of Clinical Biochemistry, School of Medicine, Babol University of Medical Sciences, Babol, Iran
| | - Atieh Makhlough
- Department of Internal Medicine, Diabetes Research Center, Sari Imam Khomeini Hospital, Mazandaran University of Medical Sciences, Sari, Iran
| | - Mohammad-Bagher Hashemi-Sooteh
- Department of Clinical Biochemistry and Medical Genetics, Molecular and Cell Biology Research Center, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran
| | - Mehryar Zargari
- Department of Clinical Biochemistry and Medical Genetics, Molecular and Cell Biology Research Center, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran
| |
Collapse
|
10
|
Alshahrani M, Al-Jabbar M, Senan EM, Ahmed IA, Saif JAM. Hybrid Methods for Fundus Image Analysis for Diagnosis of Diabetic Retinopathy Development Stages Based on Fusion Features. Diagnostics (Basel) 2023; 13:2783. [PMID: 37685321 PMCID: PMC10486790 DOI: 10.3390/diagnostics13172783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Revised: 08/22/2023] [Accepted: 08/24/2023] [Indexed: 09/10/2023] Open
Abstract
Diabetic retinopathy (DR) is a complication of diabetes that damages the delicate blood vessels of the retina and leads to blindness. Ophthalmologists rely on diagnosing the retina by imaging the fundus. The process takes a long time and needs skilled doctors to diagnose and determine the stage of DR. Therefore, automatic techniques using artificial intelligence play an important role in analyzing fundus images for the detection of the stages of DR development. However, diagnosis using artificial intelligence techniques is a difficult task and passes through many stages, and the extraction of representative features is important in reaching satisfactory results. Convolutional Neural Network (CNN) models play an important and distinct role in extracting features with high accuracy. In this study, fundus images were used for the detection of the developmental stages of DR by two proposed methods, each with two systems. The first proposed method uses GoogLeNet with SVM and ResNet-18 with SVM. The second method uses Feed-Forward Neural Networks (FFNN) based on the hybrid features extracted by first using GoogLeNet, Fuzzy color histogram (FCH), Gray Level Co-occurrence Matrix (GLCM), and Local Binary Pattern (LBP); followed by ResNet-18, FCH, GLCM and LBP. All the proposed methods obtained superior results. The FFNN network with hybrid features of ResNet-18, FCH, GLCM, and LBP obtained 99.7% accuracy, 99.6% precision, 99.6% sensitivity, 100% specificity, and 99.86% AUC.
Collapse
Affiliation(s)
- Mohammed Alshahrani
- Computer Department, Applied College, Najran University, Najran 66462, Saudi Arabia;
| | - Mohammed Al-Jabbar
- Computer Department, Applied College, Najran University, Najran 66462, Saudi Arabia;
| | - Ebrahim Mohammed Senan
- Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, Alrazi University, Sana’a, Yemen
| | | | | |
Collapse
|
11
|
Xie H, Li Z, Wu C, Zhao Y, Lin C, Wang Z, Wang C, Gu Q, Wang M, Zheng Q, Jiang J, Chen W. Deep learning for detecting visually impaired cataracts using fundus images. Front Cell Dev Biol 2023; 11:1197239. [PMID: 37576595 PMCID: PMC10416247 DOI: 10.3389/fcell.2023.1197239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Accepted: 07/20/2023] [Indexed: 08/15/2023] Open
Abstract
Purpose: To develop a visual function-based deep learning system (DLS) using fundus images to screen for visually impaired cataracts. Materials and methods: A total of 8,395 fundus images (5,245 subjects) with corresponding visual function parameters collected from three clinical centers were used to develop and evaluate a DLS for classifying non-cataracts, mild cataracts, and visually impaired cataracts. Three deep learning algorithms (DenseNet121, Inception V3, and ResNet50) were leveraged to train models to obtain the best one for the system. The performance of the system was evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Results: The AUC of the best algorithm (DenseNet121) on the internal test dataset and the two external test datasets were 0.998 (95% CI, 0.996-0.999) to 0.999 (95% CI, 0.998-1.000),0.938 (95% CI, 0.924-0.951) to 0.966 (95% CI, 0.946-0.983) and 0.937 (95% CI, 0.918-0.953) to 0.977 (95% CI, 0.962-0.989), respectively. In the comparison between the system and cataract specialists, better performance was observed in the system for detecting visually impaired cataracts (p < 0.05). Conclusion: Our study shows the potential of a function-focused screening tool to identify visually impaired cataracts from fundus images, enabling timely patient referral to tertiary eye hospitals.
Collapse
Affiliation(s)
- He Xie
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Zhongwen Li
- Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China
| | - Chengchao Wu
- School of Electronic Engineering, Xi’an University of Posts and Telecommunications, Xi’an, China
| | - Yitian Zhao
- Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China
- Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, China
| | - Chengmin Lin
- Department of Ophthalmology, Wenzhou Hospital of Integrated Traditional Chinese and Western Medicine, Wenzhou, China
| | - Zhouqian Wang
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Chenxi Wang
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Qinyi Gu
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Minye Wang
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Qinxiang Zheng
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
- Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China
| | - Jiewei Jiang
- School of Electronic Engineering, Xi’an University of Posts and Telecommunications, Xi’an, China
| | - Wei Chen
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
- Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China
| |
Collapse
|
12
|
Penha FM, Priotto BM, Hennig F, Przysiezny B, Wiethorn BA, Orsi J, Nagel IBF, Wiggers B, Stuchi JA, Lencione D, de Souza Prado PV, Yamanaka F, Lojudice F, Malerbi FK. Single retinal image for diabetic retinopathy screening: performance of a handheld device with embedded artificial intelligence. Int J Retina Vitreous 2023; 9:41. [PMID: 37430345 DOI: 10.1186/s40942-023-00477-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 06/23/2023] [Indexed: 07/12/2023] Open
Abstract
BACKGROUND Diabetic retinopathy (DR) is a leading cause of blindness. Our objective was to evaluate the performance of an artificial intelligence (AI) system integrated into a handheld smartphone-based retinal camera for DR screening using a single retinal image per eye. METHODS Images were obtained from individuals with diabetes during a mass screening program for DR in Blumenau, Southern Brazil, conducted by trained operators. Automatic analysis was conducted using an AI system (EyerMaps™, Phelcom Technologies LLC, Boston, USA) with one macula-centered, 45-degree field of view retinal image per eye. The results were compared to the assessment by a retinal specialist, considered as the ground truth, using two images per eye. Patients with ungradable images were excluded from the analysis. RESULTS A total of 686 individuals (average age 59.2 ± 13.3 years, 56.7% women, diabetes duration 12.1 ± 9.4 years) were included in the analysis. The rates of insulin use, daily glycemic monitoring, and systemic hypertension treatment were 68.4%, 70.2%, and 70.2%, respectively. Although 97.3% of patients were aware of the risk of blindness associated with diabetes, more than half of them underwent their first retinal examination during the event. The majority (82.5%) relied exclusively on the public health system. Approximately 43.4% of individuals were either illiterate or had not completed elementary school. DR classification based on the ground truth was as follows: absent or nonproliferative mild DR 86.9%, more than mild (mtm) DR 13.1%. The AI system achieved sensitivity, specificity, positive predictive value, and negative predictive value percentages (95% CI) for mtmDR as follows: 93.6% (87.8-97.2), 71.7% (67.8-75.4), 42.7% (39.3-46.2), and 98.0% (96.2-98.9), respectively. The area under the ROC curve was 86.4%. CONCLUSION The portable retinal camera combined with AI demonstrated high sensitivity for DR screening using only one image per eye, offering a simpler protocol compared to the traditional approach of two images per eye. Simplifying the DR screening process could enhance adherence rates and overall program coverage.
Collapse
Affiliation(s)
- Fernando Marcondes Penha
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil.
- Botelho Hospital da Visão, Rua 2 de Setembro, 2958, Blumenau, 89052-504, SC, Brazil.
| | - Bruna Milene Priotto
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Francini Hennig
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Bernardo Przysiezny
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Bruno Antunes Wiethorn
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | - Julia Orsi
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | | | - Brenda Wiggers
- Fundacao Universidade Regional de Blumenau, Rua Antonio Veiga 140, Blumenau, 89030-903, SC, Brazil
| | | | | | | | | | - Fernando Lojudice
- Bayer Healthcare - Brazil, São Paulo, SP, Brazil
- Cell and Molecular Theraphy Center (NUCEL), School of Medicine, University of São Paulo, São Paulo, SP, Brazil
| | - Fernando Korn Malerbi
- Department of Ophthalmology, Federal University of São Paulo (UNIFESP), São Paulo, SP, Brazil
| |
Collapse
|
13
|
Fujihara K, Yamada Harada M, Horikawa C, Iwanaga M, Tanaka H, Nomura H, Sui Y, Tanabe K, Yamada T, Kodama S, Kato K, Sone H. Machine learning approach to predict body weight in adults. Front Public Health 2023; 11:1090146. [PMID: 37397751 PMCID: PMC10308016 DOI: 10.3389/fpubh.2023.1090146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Accepted: 05/10/2023] [Indexed: 07/04/2023] Open
Abstract
Background Obesity is an established risk factor for non-communicable diseases such as type 2 diabetes mellitus, hypertension and cardiovascular disease. Thus, weight control is a key factor in the prevention of non-communicable diseases. A simple and quick method to predict weight change over a few years could be helpful for weight management in clinical settings. Methods We examined the ability of a machine learning model that we constructed to predict changes in future body weight over 3 years using big data. Input in the machine learning model were three-year data on 50,000 Japanese persons (32,977 men) aged 19-91 years who underwent annual health examinations. The predictive formulas that used heterogeneous mixture learning technology (HMLT) to predict body weight in the subsequent 3 years were validated for 5,000 persons. The root mean square error (RMSE) was used to evaluate accuracy compared with multiple regression. Results The machine learning model utilizing HMLT automatically generated five predictive formulas. The influence of lifestyle on body weight was found to be large in people with a high body mass index (BMI) at baseline (BMI ≥29.93 kg/m2) and in young people (<24 years) with a low BMI (BMI <23.44 kg/m2). The RMSE was 1.914 in the validation set which reflects ability comparable to that of the multiple regression model of 1.890 (p = 0.323). Conclusion The HMLT-based machine learning model could successfully predict weight change over 3 years. Our model could automatically identify groups whose lifestyle profoundly impacted weight loss and factors the influenced body weight change in individuals. Although this model must be validated in other populations, including other ethnic groups, before being widely implemented in global clinical settings, results suggested that this machine learning model could contribute to individualized weight management.
Collapse
Affiliation(s)
- Kazuya Fujihara
- Department of Endocrinology and Metabolism, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Mayuko Yamada Harada
- Department of Endocrinology and Metabolism, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Chika Horikawa
- Department of Health and Nutrition, Faculty of Human Life Studies, University of Niigata Prefecture, Niigata, Japan
| | - Midori Iwanaga
- Department of Endocrinology and Metabolism, Faculty of Medicine, Niigata University, Niigata, Japan
| | | | | | | | | | - Takaho Yamada
- Department of Endocrinology and Metabolism, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Satoru Kodama
- Department of Endocrinology and Metabolism, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Kiminori Kato
- Department of Prevention of Noncommunicable Diseases and Promotion of Health Checkup, Niigata University, Niigata, Japan
| | - Hirohito Sone
- Department of Endocrinology and Metabolism, Faculty of Medicine, Niigata University, Niigata, Japan
| |
Collapse
|
14
|
Wawer Matos PA, Reimer RP, Rokohl AC, Caldeira L, Heindl LM, Große Hokamp N. Artificial Intelligence in Ophthalmology - Status Quo and Future Perspectives. Semin Ophthalmol 2023; 38:226-237. [PMID: 36356300 DOI: 10.1080/08820538.2022.2139625] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
Artificial intelligence (AI) is an emerging technology in healthcare and holds the potential to disrupt many arms in medical care. In particular, disciplines using medical imaging modalities, including e.g. radiology but ophthalmology as well, are already confronted with a wide variety of AI implications. In ophthalmologic research, AI has demonstrated promising results limited to specific diseases and imaging tools, respectively. Yet, implementation of AI in clinical routine is not widely spread due to availability, heterogeneity in imaging techniques and AI methods. In order to describe the status quo, this narrational review provides a brief introduction to AI ("what the ophthalmologist needs to know"), followed by an overview of different AI-based applications in ophthalmology and a discussion on future challenges.Abbreviations: Age-related macular degeneration, AMD; Artificial intelligence, AI; Anterior segment OCT, AS-OCT; Coronary artery calcium score, CACS; Convolutional neural network, CNN; Deep convolutional neural network, DCNN; Diabetic retinopathy, DR; Machine learning, ML; Optical coherence tomography, OCT; Retinopathy of prematurity, ROP; Support vector machine, SVM; Thyroid-associated ophthalmopathy, TAO.
Collapse
Affiliation(s)
| | - Robert P Reimer
- Department of Diagnostic and Interventional Radiology, University Hospital of Cologne, Köln, Germany
| | - Alexander C Rokohl
- Department of Ophthalmology, University Hospital of Cologne, Köln, Germany
| | - Liliana Caldeira
- Department of Diagnostic and Interventional Radiology, University Hospital of Cologne, Köln, Germany
| | - Ludwig M Heindl
- Department of Ophthalmology, University Hospital of Cologne, Köln, Germany
| | - Nils Große Hokamp
- Department of Diagnostic and Interventional Radiology, University Hospital of Cologne, Köln, Germany
| |
Collapse
|
15
|
Zhu A, Tailor P, Verma R, Zhang I, Schott B, Ye C, Szirth B, Habiel M, Khouri AS. Implementation of deep learning artificial intelligence in vision-threatening disease screenings for an underserved community during COVID-19. J Telemed Telecare 2023:1357633X231158832. [PMID: 36908254 PMCID: PMC10014445 DOI: 10.1177/1357633x231158832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2023]
Abstract
INTRODUCTION Age-related macular degeneration, diabetic retinopathy, and glaucoma are vision-threatening diseases that are leading causes of vision loss. Many studies have validated deep learning artificial intelligence for image-based diagnosis of vision-threatening diseases. Our study prospectively investigated deep learning artificial intelligence applications in student-run non-mydriatic screenings for an underserved, primarily Hispanic community during COVID-19. METHODS Five supervised student-run community screenings were held in West New York, New Jersey. Participants underwent non-mydriatic 45-degree retinal imaging by medical students. Images were uploaded to a cloud-based deep learning artificial intelligence for vision-threatening disease referral. An on-site tele-ophthalmology grader and remote clinical ophthalmologist graded images, with adjudication by a senior ophthalmologist to establish the gold standard diagnosis, which was used to assess the performance of deep learning artificial intelligence. RESULTS A total of 385 eyes from 195 screening participants were included (mean age 52.43 ± 14.5 years, 40.0% female). A total of 48 participants were referred for at least one vision-threatening disease. Deep learning artificial intelligence marked 150/385 (38.9%) eyes as ungradable, compared to 10/385 (2.6%) ungradable as per the human gold standard (p < 0.001). Deep learning artificial intelligence had 63.2% sensitivity, 94.5% specificity, 32.0% positive predictive value, and 98.4% negative predictive value in vision-threatening disease referrals. Deep learning artificial intelligence successfully referred all 4 eyes with multiple vision-threatening diseases. Deep learning artificial intelligence graded images (35.6 ± 13.3 s) faster than the tele-ophthalmology grader (129 ± 41.0) and clinical ophthalmologist (68 ± 21.9, p < 0.001). DISCUSSION Deep learning artificial intelligence can increase the efficiency and accessibility of vision-threatening disease screenings, particularly in underserved communities. Deep learning artificial intelligence should be adaptable to different environments. Consideration should be given to how deep learning artificial intelligence can best be utilized in a real-world application, whether in computer-aided or autonomous diagnosis.
Collapse
Affiliation(s)
- Aretha Zhu
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Priya Tailor
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Rashika Verma
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Isis Zhang
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Brian Schott
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Catherine Ye
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Bernard Szirth
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Miriam Habiel
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Albert S Khouri
- 537707Institute of Ophthalmology & Visual Science, Rutgers New Jersey Medical School, Newark, NJ, USA
| |
Collapse
|
16
|
Mehra AA, Softing A, Guner MK, Hodge DO, Barkmeier AJ. Diabetic Retinopathy Telemedicine Outcomes With Artificial Intelligence-Based Image Analysis, Reflex Dilation, and Image Overread. Am J Ophthalmol 2022; 244:125-132. [PMID: 35970206 DOI: 10.1016/j.ajo.2022.08.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Revised: 08/03/2022] [Accepted: 08/03/2022] [Indexed: 01/30/2023]
Abstract
PURPOSE To examine real-world telemedicine outcomes of diabetic retinopathy (DR) screening with artificial intelligence (AI)-based image analysis, reflex dilation, and secondary image overread in a primary care setting. DESIGN Validity and reliability analysis. METHODS Single institution review of 1052 consecutive adult patients who received diabetic retinopathy photoscreening in the primary care setting over an 18-month period. Nonmydriatic fundus photographs were acquired and analyzed by the IDx-DR AI-based system. When nonmydriatic images were ungradable, reflex dilation (1% tropicamide) and mydriatic photography were performed for repeat AI-based analysis. Manual overread was performed on all images. Patient demographics, clinical characteristics, and screening outcomes were recorded. RESULTS A total of 965 of 1052 patients (91.7%) had AI-gradable fundus photographs: 580 had gradable nonmydriatic imaging (55.1%) and 440 of 472 patients with ungradable nonmydriatic photographs had reflex dilation (93.2%). One hundred thirty-eight of 965 patients (14.3%) were AI-graded as "positive" (greater than mild NPDR) and 827 of 965 were "negative" (85.7%), with 100% sensitivity (95% CI 90.8-100%), 89.2% specificity (95% CI 87.0-91.1%), 27.5% positive predictive value (95% CI 24.0-31.4%), and 100% negative predictive value (95% CI 99.6-100%) compared with manual overread assessment of greater than mild NPDR requiring further evaluation with a comprehensive dilated examination. Image gradeability was inversely related to patient age: 93.5% gradable (61.9% nonmydriatic) for patients aged <70 years vs 85.3% (31.0% nonmydriatic) for patients aged 70+ years (P < .001). CONCLUSION Incorporation of AI-based image analysis into real-world primary care diabetic retinopathy screening yielded no false negative results and offered excellent image gradeability within a protocol combining nonmydriatic fundus photography and pharmacologic dilation, as needed. Image gradeability was lower with increasing patient age.
Collapse
Affiliation(s)
- Ankur A Mehra
- From Department of Ophthalmology, Mayo Clinic, Rochester, USA (A.A.M, A.S, M.K.G, A.J.B)
| | - Alaina Softing
- From Department of Ophthalmology, Mayo Clinic, Rochester, USA (A.A.M, A.S, M.K.G, A.J.B)
| | | | - David O Hodge
- Department of Quantitative Health Sciences, Mayo Clinic, Jacksonville, USA (D.O.H)
| | - Andrew J Barkmeier
- Department of Quantitative Health Sciences, Mayo Clinic, Jacksonville, USA (D.O.H).
| |
Collapse
|
17
|
Lin PK, Chiu YH, Huang CJ, Wang CY, Pan ML, Wang DW, Mark Liao HY, Chen YS, Kuan CH, Lin SY, Chen LF. PADAr: physician-oriented artificial intelligence-facilitating diagnosis aid for retinal diseases. J Med Imaging (Bellingham) 2022; 9:044501. [PMID: 35903415 PMCID: PMC9311486 DOI: 10.1117/1.jmi.9.4.044501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 07/01/2022] [Indexed: 11/14/2022] Open
Abstract
Purpose: Retinopathy screening via digital imaging is promising for early detection and timely treatment, and tracking retinopathic abnormality over time can help to reveal the risk of disease progression. We developed an innovative physician-oriented artificial intelligence-facilitating diagnosis aid system for retinal diseases for screening multiple retinopathies and monitoring the regions of potential abnormality over time. Approach: Our dataset contains 4908 fundus images from 304 eyes with image-level annotations, including diabetic retinopathy, age-related macular degeneration, cellophane maculopathy, pathological myopia, and healthy control (HC). The screening model utilized a VGG-based feature extractor and multiple-binary convolutional neural network-based classifiers. Images in time series were aligned via affine transforms estimated through speeded-up robust features. Heatmaps of retinopathy were generated from the feature extractor using gradient-weighted class activation mapping++, and individual candidate retinopathy sites were identified from the heatmaps using clustering algorithm. Nested cross-validation with a train-to-test split of 80% to 20% was used to evaluate the performance of the screening model. Results: Our screening model achieved 99% accuracy, 93% sensitivity, and 97% specificity in discriminating between patients with retinopathy and HCs. For discriminating between types of retinopathy, our model achieved an averaged performance of 80% accuracy, 78% sensitivity, 94% specificity, 79% F1-score, and Cohen's kappa coefficient of 0.70. Moreover, visualization results were also shown to provide reasonable candidate sites of retinopathy. Conclusions: Our results demonstrated the capability of the proposed model for extracting diagnostic information of the abnormality and lesion locations, which allows clinicians to focus on patient-centered treatment and untangles the pathological plausibility hidden in deep learning models.
Collapse
Affiliation(s)
- Po-Kang Lin
- National Taiwan University, Graduate Institute of Biomedical Electronics and Bioinformatics, Taipei, Taiwan.,National Yang Ming Chiao Tung University, School of Medicine, Department of Ophthalmology, Taipei, Taiwan.,Taipei Veterans General Hospital, Department of Ophthalmology, Taipei, Taiwan
| | - Yu-Hsien Chiu
- National Yang Ming Chiao Tung University, Institute of Brain Science, College of Medicine, Taipei, Taiwan.,Academia Sinica, Institute of Information Science, Taipei, Taiwan
| | - Chiu-Jung Huang
- National Yang Ming Chiao Tung University, Institute of Brain Science, College of Medicine, Taipei, Taiwan.,Taipei Veterans General Hospital, Integrated Brain Research Unit, Department of Medical Research, Taipei, Taiwan
| | - Chien-Yao Wang
- Academia Sinica, Institute of Information Science, Taipei, Taiwan
| | - Mei-Lien Pan
- Academia Sinica, Institute of Information Science, Taipei, Taiwan.,National Yang Ming Chiao Tung University, Information Technology Service Center, Taipei, Taiwan
| | - Da-Wei Wang
- Academia Sinica, Institute of Information Science, Taipei, Taiwan
| | | | - Yong-Sheng Chen
- National Yang Ming Chiao Tung University, Department of Computer Science, Hsinchu, Taiwan
| | - Chieh-Hsiung Kuan
- National Taiwan University, Department of Electrical Engineering, Taipei, Taiwan.,National Taiwan University, Gaduate Institute of Biomedical Electronics and Bioinformatics, Taipei, Taiwan
| | - Shih-Yen Lin
- National Yang Ming Chiao Tung University, Department of Computer Science, Hsinchu, Taiwan
| | - Li-Fen Chen
- National Yang Ming Chiao Tung University, Institute of Brain Science, College of Medicine, Taipei, Taiwan.,Taipei Veterans General Hospital, Integrated Brain Research Unit, Department of Medical Research, Taipei, Taiwan
| |
Collapse
|
18
|
Held LA, Wewetzer L, Steinhäuser J. Determinants of the implementation of an artificial intelligence-supported device for the screening of diabetic retinopathy in primary care - a qualitative study. Health Informatics J 2022; 28:14604582221112816. [PMID: 35921547 DOI: 10.1177/14604582221112816] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Diabetic retinopathy is a microvascular complication of diabetes mellitus that is usually asymptomatic in the early stages. Therefore, its timely detection and treatment are essential. First pilot projects exist to establish a smartphone-based and AI-supported screening of DR in primary care. This study explored health professionals' perceptions of potential barriers and enablers of using a screening such as this in primary care to understand the mechanisms that could influence implementation into routine clinical practice. Semi-structured telephone interviews were conducted and analysed with the help of qualitative analysis of Mayring. The following main influencing factors to implementation have been identified: personal attitude, organisation, time, financial factors, education, support, technical requirement, influence on profession and patient welfare. Most determinants could be relocated in the behaviour change wheel, a validated implementation model. Further research on the patients' perspective and a ranking of the determinants found is needed.
Collapse
Affiliation(s)
- Linda A Held
- Institute of Family Medicine, 54360University Medical Center Schleswig-Holstein, Campus Lübeck, Germany
| | - Larisa Wewetzer
- Institute of Family Medicine, 54360University Medical Center Schleswig-Holstein, Campus Lübeck, Germany
| | - Jost Steinhäuser
- Institute of Family Medicine, 54360University Medical Center Schleswig-Holstein, Campus Lübeck, Germany
| |
Collapse
|
19
|
Alexopoulos P, Madu C, Wollstein G, Schuman JS. The Development and Clinical Application of Innovative Optical Ophthalmic Imaging Techniques. Front Med (Lausanne) 2022; 9:891369. [PMID: 35847772 PMCID: PMC9279625 DOI: 10.3389/fmed.2022.891369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Accepted: 05/23/2022] [Indexed: 11/22/2022] Open
Abstract
The field of ophthalmic imaging has grown substantially over the last years. Massive improvements in image processing and computer hardware have allowed the emergence of multiple imaging techniques of the eye that can transform patient care. The purpose of this review is to describe the most recent advances in eye imaging and explain how new technologies and imaging methods can be utilized in a clinical setting. The introduction of optical coherence tomography (OCT) was a revolution in eye imaging and has since become the standard of care for a plethora of conditions. Its most recent iterations, OCT angiography, and visible light OCT, as well as imaging modalities, such as fluorescent lifetime imaging ophthalmoscopy, would allow a more thorough evaluation of patients and provide additional information on disease processes. Toward that goal, the application of adaptive optics (AO) and full-field scanning to a variety of eye imaging techniques has further allowed the histologic study of single cells in the retina and anterior segment. Toward the goal of remote eye care and more accessible eye imaging, methods such as handheld OCT devices and imaging through smartphones, have emerged. Finally, incorporating artificial intelligence (AI) in eye images has the potential to become a new milestone for eye imaging while also contributing in social aspects of eye care.
Collapse
Affiliation(s)
- Palaiologos Alexopoulos
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, United States
| | - Chisom Madu
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, United States
| | - Gadi Wollstein
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, United States
- Department of Biomedical Engineering, NYU Tandon School of Engineering, Brooklyn, NY, United States
- Center for Neural Science, College of Arts & Science, New York University, New York, NY, United States
| | - Joel S. Schuman
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, United States
- Department of Biomedical Engineering, NYU Tandon School of Engineering, Brooklyn, NY, United States
- Center for Neural Science, College of Arts & Science, New York University, New York, NY, United States
- Department of Electrical and Computer Engineering, NYU Tandon School of Engineering, Brooklyn, NY, United States
| |
Collapse
|
20
|
Malerbi FK, Andrade RE, Morales PH, Stuchi JA, Lencione D, de Paulo JV, Carvalho MP, Nunes FS, Rocha RM, Ferraz DA, Belfort R. Diabetic Retinopathy Screening Using Artificial Intelligence and Handheld Smartphone-Based Retinal Camera. J Diabetes Sci Technol 2022; 16:716-723. [PMID: 33435711 PMCID: PMC9294565 DOI: 10.1177/1932296820985567] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
BACKGROUND Portable retinal cameras and deep learning (DL) algorithms are novel tools adopted by diabetic retinopathy (DR) screening programs. Our objective is to evaluate the diagnostic accuracy of a DL algorithm and the performance of portable handheld retinal cameras in the detection of DR in a large and heterogenous type 2 diabetes population in a real-world, high burden setting. METHOD Participants underwent fundus photographs of both eyes with a portable retinal camera (Phelcom Eyer). Classification of DR was performed by human reading and a DL algorithm (PhelcomNet), consisting of a convolutional neural network trained on a dataset of fundus images captured exclusively with the portable device; both methods were compared. We calculated the area under the curve (AUC), sensitivity, and specificity for more than mild DR. RESULTS A total of 824 individuals with type 2 diabetes were enrolled at Itabuna Diabetes Campaign, a subset of 679 (82.4%) of whom could be fully assessed. The algorithm sensitivity/specificity was 97.8 % (95% CI 96.7-98.9)/61.4 % (95% CI 57.7-65.1); AUC was 0·89. All false negative cases were classified as moderate non-proliferative diabetic retinopathy (NPDR) by human grading. CONCLUSIONS The DL algorithm reached a good diagnostic accuracy for more than mild DR in a real-world, high burden setting. The performance of the handheld portable retinal camera was adequate, with over 80% of individuals presenting with images of sufficient quality. Portable devices and artificial intelligence tools may increase coverage of DR screening programs.
Collapse
Affiliation(s)
- Fernando Korn Malerbi
- Department of Ophthalmology and Visual
Sciences, Federal University of São Paulo, São Paulo, Brazil
- Instituto Paulista de Estudos e
Pesquisas em Oftalmologia, IPEPO, Vision Institute, São Paulo, Brazil
- Fernando Korn Malerbi, Federal University of
São Paulo, Rua Botucatu, 822. São Paulo, SP 04039-032, Brazil.
| | - Rafael Ernane Andrade
- Department of Ophthalmology and Visual
Sciences, Federal University of São Paulo, São Paulo, Brazil
- Hospital de Olhos Beira Rio, Itabuna,
BA, Brazil
| | - Paulo Henrique Morales
- Department of Ophthalmology and Visual
Sciences, Federal University of São Paulo, São Paulo, Brazil
- Instituto Paulista de Estudos e
Pesquisas em Oftalmologia, IPEPO, Vision Institute, São Paulo, Brazil
| | | | | | | | | | | | | | - Daniel A. Ferraz
- Instituto Paulista de Estudos e
Pesquisas em Oftalmologia, IPEPO, Vision Institute, São Paulo, Brazil
- NIHR Biomedical Research Centre for
Ophthalmology, Moorfields Eye Hospital, NHS Foundation Trust and UCL Institute of
Ophthalmology, London, UK
| | - Rubens Belfort
- Department of Ophthalmology and Visual
Sciences, Federal University of São Paulo, São Paulo, Brazil
- Instituto Paulista de Estudos e
Pesquisas em Oftalmologia, IPEPO, Vision Institute, São Paulo, Brazil
| |
Collapse
|
21
|
Google’s new AI technology detects cardiac issues using retinal scan. APPLIED NANOSCIENCE 2022. [DOI: 10.1007/s13204-021-02208-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
22
|
Potapenko I, Kristensen M, Thiesson B, Ilginis T, Lykke Sørensen T, Nouri Hajari J, Fuchs J, Hamann S, Cour M. Detection of oedema on optical coherence tomography images using deep learning model trained on noisy clinical data. Acta Ophthalmol 2022; 100:103-110. [PMID: 33991170 DOI: 10.1111/aos.14895] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2020] [Accepted: 04/18/2021] [Indexed: 12/28/2022]
Abstract
PURPOSE To meet the demands imposed by the continuing growth of the Age-related macular degeneration (AMD) patient population, automation of follow-ups by detecting retinal oedema using deep learning might be a viable approach. However, preparing and labelling data for training is time consuming. In this study, we investigate the feasibility of training a convolutional neural network (CNN) to accurately detect retinal oedema on optical coherence tomography (OCT) images of AMD patients with labels derived directly from clinical treatment decisions, without extensive preprocessing or relabelling. METHODS A total of 50 439 OCT images with associated treatment information were retrieved from databases at the Department of Ophthalmology, Rigshospitalet, Copenhagen, Denmark between 01.06.2007 and 01.06.2018. A CNN was trained on the retrieved data with the recorded treatment decisions as labels and validated on a subset of the data relabelled by three ophthalmologists to denote presence of oedema. RESULTS Moderate inter-grader agreement on presence of oedema in the relabelled data was found (76.4%). Despite different training and validation labels, the CNN performed on par with inter-grader agreement in detecting oedema on OCT images (AUC 0.97, accuracy 90.9%) and previously published models based on relabelled datasets. CONCLUSION The level of performance shown by the current model might make it valuable in detecting disease activity in automated AMD patient follow-up systems. Our approach demonstrates that high accuracy is not necessarily constrained by incongruent training and validation labels. These results might encourage the use of existing clinical databases for development of deep learning based algorithms without labour-intensive preprocessing in the future.
Collapse
Affiliation(s)
- Ivan Potapenko
- Department of Ophthalmology Rigshospitalet Copenhagen Denmark
- Faculty of Health and Medical Sciences University of Copenhagen Copenhagen Denmark
| | | | - Bo Thiesson
- Enversion A/S Aarhus Denmark
- Department of Engineering Aarhus University Aarhus Denmark
| | - Tomas Ilginis
- Department of Ophthalmology Rigshospitalet Copenhagen Denmark
| | - Torben Lykke Sørensen
- Faculty of Health and Medical Sciences University of Copenhagen Copenhagen Denmark
- Department of Ophthalmology Zealand University Hospital Roskilde Denmark
| | | | - Josefine Fuchs
- Department of Ophthalmology Rigshospitalet Copenhagen Denmark
| | - Steffen Hamann
- Department of Ophthalmology Rigshospitalet Copenhagen Denmark
- Faculty of Health and Medical Sciences University of Copenhagen Copenhagen Denmark
| | - Morten Cour
- Department of Ophthalmology Rigshospitalet Copenhagen Denmark
- Faculty of Health and Medical Sciences University of Copenhagen Copenhagen Denmark
| |
Collapse
|
23
|
Li N, Ma M, Lai M, Gu L, Kang M, Wang Z, Jiao S, Dang K, Deng J, Ding X, Zhen Q, Zhang A, Shen T, Zheng Z, Wang Y, Peng Y. A stratified analysis of a deep learning algorithm in the diagnosis of diabetic retinopathy in a real-world study. J Diabetes 2022; 14:111-120. [PMID: 34889059 PMCID: PMC9060020 DOI: 10.1111/1753-0407.13241] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Revised: 11/06/2021] [Accepted: 11/12/2021] [Indexed: 01/22/2023] Open
Abstract
BACKGROUND The aim of our research was to prospectively explore the clinical value of a deep learning algorithm (DLA) to detect referable diabetic retinopathy (DR) in different subgroups stratified by types of diabetes, blood pressure, sex, BMI, age, glycosylated hemoglobin (HbA1c), diabetes duration, urine albumin-to-creatinine ratio (UACR), and estimated glomerular filtration rate (eGFR) at a real-world diabetes center in China. METHODS A total of 1147 diabetic patients from Shanghai General Hospital were recruited from October 2018 to August 2019. Retinal fundus images were graded by the DLA, and the detection of referable DR (moderate nonproliferative DR or worse) was compared with a reference standard generated by one certified retinal specialist with more than 12 years of experience. The performance of DLA across different subgroups stratified by types of diabetes, blood pressure, sex, BMI, age, HbA1c, diabetes duration, UACR, and eGFR was evaluated. RESULTS For all 1674 gradable images, the area under the receiver operating curve, sensitivity, and specificity of the DLA for referable DR were 0.942 (95% CI, 0.920-0.964), 85.1% (95% CI, 83.4%-86.8%), and 95.6% (95% CI, 94.6%-96.6%), respectively. The DLA showed consistent performance across most subgroups, while it showed superior performance in the subgroups of patients with type 1 diabetes, UACR ≥ 30 mg/g, and eGFR < 90 mL/min/1.73m2 . CONCLUSIONS This study showed that the DLA was a reliable alternative method for the detection of referable DR and performed superior in patients with type 1 diabetes and diabetic nephropathy who were prone to DR.
Collapse
Affiliation(s)
- Na Li
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Mingming Ma
- Department of OphthalmologyShanghai General Hospital, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye DiseasesShanghaiChina
| | - Mengyu Lai
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Liping Gu
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Mei Kang
- Clinical Research CenterShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | | | | | | | | | | | - Qin Zhen
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Aifang Zhang
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Tingting Shen
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Zhi Zheng
- Department of OphthalmologyShanghai General Hospital, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye DiseasesShanghaiChina
| | - Yufan Wang
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| | - Yongde Peng
- Department of Endocrinology and MetabolismShanghai General Hospital, Shanghai Jiao Tong UniversityShanghaiChina
| |
Collapse
|
24
|
[Use of artificial intelligence in screening for diabetic retinopathy at a tertiary diabetes center]. Ophthalmologe 2022; 119:705-713. [PMID: 35080640 DOI: 10.1007/s00347-021-01556-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Revised: 11/27/2021] [Accepted: 12/01/2021] [Indexed: 10/19/2022]
Abstract
BACKGROUND In 2018, IDx-DR was approved as a method to determine the degree of diabetic retinopathy (DR) using artificial intelligence (AI) by the FDA. METHODS We integrated IDx-DR into the consultation at a diabetology focus clinic and report the agreement between IDx-DR and fundoscopy as well as IDx-DR and ophthalmological image assessment and the influence of different camera systems. RESULTS Adequate image quality in miosis was achieved more frequently with the Topcon camera (n = 456; NW400, Topcon Medical Systems, Oakland, NJ, USA) compared with the Zeiss camera (n = 47; Zeiss VISUCAM 500, Carl Zeiss Meditec AG, Jena, Germany). Overall, IDx-DR analysis in miosis was possible in approximately 60% of the patients. All patients in whom IDx-DR analysis in miosis was not possible could be assessed by fundoscopy with dilated pupils. Within the group of images that could be evaluated, there was agreement between IDx-DR and ophthalmic fundoscopy in approximately 55%, overestimation of severity by IDx-DR in approximately 40% and underestimation in approximately 4%. The sensitivity (specificity) for detecting severe retinopathy requiring treatment was 95.7% (89.1%) for cases with fundus images that could be evaluated and 65.2% (66.7%) when all cases were considered (including those without images in miosis which could be evaluated). The kappa coefficient of 0.334 (p < 0.001) shows sufficient agreement between IDx-DR and physician's image analysis based on the fundus photograph, considering all patients with IDx-DR analysis that could be evaluated. The comparison between IDx-DR and the physician's funduscopy under the same conditions shows a low agreement with a kappa value of 0.168 (p < 0.001). CONCLUSION The present study shows the possibilities and limitations of AI-assisted DR screening. A major limitation is that sufficient images cannot be obtained in miosis in approximately 40% of patients. When sufficient images were available the IDx-DR and ophthalmological diagnosis matched in more than 50% of cases. Underestimation of severity by IDx-DR occurred only rarely. For integration into an ophthalmologist's practice, this system seems suitable. Without access to an ophthalmologist the high rate of insufficient images in miosis represents an important limitation.
Collapse
|
25
|
End-to-end diabetic retinopathy grading based on fundus fluorescein angiography images using deep learning. Graefes Arch Clin Exp Ophthalmol 2022; 260:1663-1673. [DOI: 10.1007/s00417-021-05503-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2021] [Revised: 10/11/2021] [Accepted: 11/14/2021] [Indexed: 12/14/2022] Open
|
26
|
Tsai MJ, Hsieh YT, Tsai CH, Chen M, Hsieh AT, Tsai CW, Chen ML. Cross-Camera External Validation for Artificial Intelligence Software in Diagnosis of Diabetic Retinopathy. J Diabetes Res 2022; 2022:5779276. [PMID: 35308093 PMCID: PMC8926465 DOI: 10.1155/2022/5779276] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Revised: 02/12/2022] [Accepted: 02/17/2022] [Indexed: 11/18/2022] Open
Abstract
AIMS To investigate the applicability of deep learning image assessment software VeriSee DR to different color fundus cameras for the screening of diabetic retinopathy (DR). METHODS Color fundus images of diabetes patients taken with three different nonmydriatic fundus cameras, including 477 Topcon TRC-NW400, 459 Topcon TRC-NW8 series, and 471 Kowa nonmyd 8 series that were judged as "gradable" by one ophthalmologist were enrolled for validation. VeriSee DR was then used for the diagnosis of referable DR according to the International Clinical Diabetic Retinopathy Disease Severity Scale. Gradability, sensitivity, and specificity were calculated for each camera model. RESULTS All images (100%) from the three camera models were gradable for VeriSee DR. The sensitivity for diagnosing referable DR in the TRC-NW400, TRC-NW8, and non-myd 8 series was 89.3%, 94.6%, and 95.7%, respectively, while the specificity was 94.2%, 90.4%, and 89.3%, respectively. Neither the sensitivity nor the specificity differed significantly between these camera models and the original camera model used for VeriSee DR development (p = 0.40, p = 0.065, respectively). CONCLUSIONS VeriSee DR was applicable to a variety of color fundus cameras with 100% agreement with ophthalmologists in terms of gradability and good sensitivity and specificity for the diagnosis of referable DR.
Collapse
Affiliation(s)
- Meng-Ju Tsai
- Department of Ophthalmology, Taoyuan General Hospital, Ministry of Health and Welfare, Taoyuan, Taiwan
| | - Yi-Ting Hsieh
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| | | | | | - An-Tsz Hsieh
- Hsieh's Endocrinologic Clinic, New Taipei, Taiwan
- Department of Internal Medicine, School of Medicine, National Defense Medical Center, Taipei, Taiwan
| | | | | |
Collapse
|
27
|
Lin T, Gubitosi-Klug RA, Channa R, Wolf RM. Pediatric Diabetic Retinopathy: Updates in Prevalence, Risk Factors, Screening, and Management. Curr Diab Rep 2021; 21:56. [PMID: 34902076 DOI: 10.1007/s11892-021-01436-x] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/22/2021] [Indexed: 02/06/2023]
Abstract
PURPOSE OF REVIEW Diabetic retinopathy (DR) is a microvascular complication of diabetes mellitus and a major cause of vision loss worldwide. The purpose of this review is to provide an update on the prevalence of diabetic retinopathy in youth, discuss risk factors, and review recent advances in diabetic retinopathy screening. RECENT FINDINGS While DR has long been considered a microvascular complication, recent data suggests that retinal neurodegeneration may precede the vascular changes associated with DR. The prevalence of DR has decreased in type 1 diabetes (T1D) patients following the results of the Diabetes Control and Complications Trial and implementation of intensive insulin therapy, with prevalence ranging from 14-20% before the year 2000 to 3.7-6% after 2000. In contrast, the prevalence of diabetic retinopathy in pediatric type 2 diabetes (T2D) is higher, ranging from 9.1-50%. Risk factors for diabetic retinopathy are well established and include glycemic control, diabetes duration, hypertension, and hyperlipidemia, whereas diabetes technology use including insulin pumps and continuous glucose monitors has been shown to have protective effects. Screening for DR is recommended for youth with T1D once they are aged ≥ 11 years or puberty has started and diabetes duration of 3-5 years. Pediatric T2D patients are advised to undergo screening at or soon after diagnosis, and annually thereafter, due to the insidious nature of T2D. Recent advances in DR screening methods including point of care and artificial intelligence technology have increased access to DR screening, while being cost-saving to patients and cost-effective to healthcare systems. While the prevalence of diabetic retinopathy in youth with T1D has been declining over the last few decades, there has been a significant increase in the prevalence of DR in youth with T2D. Improving access to diabetic retinopathy screening using novel screening methods may help improve detection and early treatment of diabetic retinopathy.
Collapse
Affiliation(s)
- Tyger Lin
- Department of Pediatrics, Division of Pediatric Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, 21287, USA
| | - Rose A Gubitosi-Klug
- Department of Pediatrics, Division of Endocrinology, Case Western Reserve University School of Medicine and Rainbow Babies and Children's Hospital, Cleveland, OH, USA
| | - Roomasa Channa
- Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, WI, USA
| | - Risa M Wolf
- Department of Pediatrics, Division of Pediatric Endocrinology, Johns Hopkins School of Medicine, Baltimore, MD, 21287, USA.
| |
Collapse
|
28
|
Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice. Prog Retin Eye Res 2021; 90:101034. [PMID: 34902546 DOI: 10.1016/j.preteyeres.2021.101034] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Revised: 12/03/2021] [Accepted: 12/06/2021] [Indexed: 01/14/2023]
Abstract
An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.
Collapse
|
29
|
Development of deep learning-based detecting systems for pathologic myopia using retinal fundus images. Commun Biol 2021; 4:1225. [PMID: 34702997 PMCID: PMC8548495 DOI: 10.1038/s42003-021-02758-y] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Accepted: 10/06/2021] [Indexed: 12/31/2022] Open
Abstract
Globally, cases of myopia have reached epidemic levels. High myopia and pathological myopia (PM) are the leading cause of visual impairment and blindness in China, demanding a large volume of myopia screening tasks to control the rapid growing myopic prevalence. It is desirable to develop the automatically intelligent system to facilitate these time- and labor- consuming tasks. In this study, we designed a series of deep learning systems to detect PM and myopic macular lesions according to a recent international photographic classification system (META-PM) classification based on color fundus images. Notably, our systems recorded robust performance both in the test and external validation dataset. The performance was comparable to the general ophthalmologist and retinal specialist. With the extensive adoption of this technology, effective mass screening for myopic population will become feasible on a national scale. Lu et al. develop a deep-learning based detection system for identifying pathological myopia and myopic macular legions in retinal fundus images. The system performance is comparable to human experts, but much faster, easing the burden of human time on screening for myopia.
Collapse
|
30
|
Pandey M, Gupta A. A systematic review of the automatic kidney segmentation methods in abdominal images. Biocybern Biomed Eng 2021. [DOI: 10.1016/j.bbe.2021.10.006] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
31
|
Abbasgholizadeh Rahimi S, Légaré F, Sharma G, Archambault P, Zomahoun HTV, Chandavong S, Rheault N, T Wong S, Langlois L, Couturier Y, Salmeron JL, Gagnon MP, Légaré J. Application of Artificial Intelligence in Community-Based Primary Health Care: Systematic Scoping Review and Critical Appraisal. J Med Internet Res 2021; 23:e29839. [PMID: 34477556 PMCID: PMC8449300 DOI: 10.2196/29839] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 05/29/2021] [Accepted: 05/31/2021] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Research on the integration of artificial intelligence (AI) into community-based primary health care (CBPHC) has highlighted several advantages and disadvantages in practice regarding, for example, facilitating diagnosis and disease management, as well as doubts concerning the unintended harmful effects of this integration. However, there is a lack of evidence about a comprehensive knowledge synthesis that could shed light on AI systems tested or implemented in CBPHC. OBJECTIVE We intended to identify and evaluate published studies that have tested or implemented AI in CBPHC settings. METHODS We conducted a systematic scoping review informed by an earlier study and the Joanna Briggs Institute (JBI) scoping review framework and reported the findings according to PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis-Scoping Reviews) reporting guidelines. An information specialist performed a comprehensive search from the date of inception until February 2020, in seven bibliographic databases: Cochrane Library, MEDLINE, EMBASE, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), ScienceDirect, and IEEE Xplore. The selected studies considered all populations who provide and receive care in CBPHC settings, AI interventions that had been implemented, tested, or both, and assessed outcomes related to patients, health care providers, or CBPHC systems. Risk of bias was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Two authors independently screened the titles and abstracts of the identified records, read the selected full texts, and extracted data from the included studies using a validated extraction form. Disagreements were resolved by consensus, and if this was not possible, the opinion of a third reviewer was sought. A third reviewer also validated all the extracted data. RESULTS We retrieved 22,113 documents. After the removal of duplicates, 16,870 documents were screened, and 90 peer-reviewed publications met our inclusion criteria. Machine learning (ML) (41/90, 45%), natural language processing (NLP) (24/90, 27%), and expert systems (17/90, 19%) were the most commonly studied AI interventions. These were primarily implemented for diagnosis, detection, or surveillance purposes. Neural networks (ie, convolutional neural networks and abductive networks) demonstrated the highest accuracy, considering the given database for the given clinical task. The risk of bias in diagnosis or prognosis studies was the lowest in the participant category (4/49, 4%) and the highest in the outcome category (22/49, 45%). CONCLUSIONS We observed variabilities in reporting the participants, types of AI methods, analyses, and outcomes, and highlighted the large gap in the effective development and implementation of AI in CBPHC. Further studies are needed to efficiently guide the development and implementation of AI interventions in CBPHC settings.
Collapse
Affiliation(s)
- Samira Abbasgholizadeh Rahimi
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada.,Mila-Quebec AI Institute, Montreal, QC, Canada
| | - France Légaré
- Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada.,VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada
| | - Gauri Sharma
- Faculty of Engineering, Dayalbagh Educational Institute, Agra, India
| | - Patrick Archambault
- Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada.,VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada
| | - Herve Tchala Vignon Zomahoun
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Quebec SPOR-Support Unit, Quebec City, QC, Canada
| | - Sam Chandavong
- Faculty of Science and Engineering, Université Laval, Quebec City, QC, Canada
| | - Nathalie Rheault
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Quebec SPOR-Support Unit, Quebec City, QC, Canada
| | - Sabrina T Wong
- School of Nursing, University of British Columbia, Vancouver, BC, Canada.,Center for Health Services and Policy Research, University of British Columbia, Vancouver, BC, Canada
| | - Lyse Langlois
- Department of Industrial Relations, Université Laval, Quebec City, QC, Canada.,OBVIA - Quebec International Observatory on the social impacts of AI and digital technology, Quebec City, QC, Canada
| | - Yves Couturier
- School of Social Work, University of Sherbrooke, Sherbrooke, QC, Canada
| | - Jose L Salmeron
- Department of Data Science, University Pablo de Olavide, Seville, Spain
| | | | - Jean Légaré
- Arthritis Alliance of Canada, Montreal, QC, Canada
| |
Collapse
|
32
|
Diagnostic performance of deep-learning-based screening methods for diabetic retinopathy in primary care-A meta-analysis. PLoS One 2021; 16:e0255034. [PMID: 34375355 PMCID: PMC8354436 DOI: 10.1371/journal.pone.0255034] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 07/09/2021] [Indexed: 02/01/2023] Open
Abstract
Background Diabetic retinopathy (DR) affects 10–24% of patients with diabetes mellitus type 1 or 2 in the primary care (PC) sector. As early detection is crucial for treatment, deep learning screening methods in PC setting could potentially aid in an accurate and timely diagnosis. Purpose The purpose of this meta-analysis was to determine the current state of knowledge regarding deep learning (DL) screening methods for DR in PC. Data sources A systematic literature search was conducted using Medline, Web of Science, and Scopus to identify suitable studies. Study selection Suitable studies were selected by two researchers independently. Studies assessing DL methods and the suitability of these screening systems (diagnostic parameters such as sensitivity and specificity, information on datasets and setting) in PC were selected. Excluded were studies focusing on lesions, applying conventional diagnostic imaging tools, conducted in secondary or tertiary care, and all publication types other than original research studies on human subjects. Data extraction The following data was extracted from included studies: authors, title, year of publication, objectives, participants, setting, type of intervention/method, reference standard, grading scale, outcome measures, dataset, risk of bias, and performance measures. Data synthesis and conclusion The summed sensitivity of all included studies was 87% and specificity was 90%. Given a prevalence of DR of 10% in patients with DM Type 2 in PC, the negative predictive value is 98% while the positive predictive value is 49%. Limitations Selected studies showed a high variation in sample size and quality and quantity of available data.
Collapse
|
33
|
Cen LP, Ji J, Lin JW, Ju ST, Lin HJ, Li TP, Wang Y, Yang JF, Liu YF, Tan S, Tan L, Li D, Wang Y, Zheng D, Xiong Y, Wu H, Jiang J, Wu Z, Huang D, Shi T, Chen B, Yang J, Zhang X, Luo L, Huang C, Zhang G, Huang Y, Ng TK, Chen H, Chen W, Pang CP, Zhang M. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nat Commun 2021; 12:4828. [PMID: 34376678 PMCID: PMC8355164 DOI: 10.1038/s41467-021-25138-w] [Citation(s) in RCA: 69] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Accepted: 07/22/2021] [Indexed: 02/05/2023] Open
Abstract
Retinal fundus diseases can lead to irreversible visual impairment without timely diagnoses and appropriate treatments. Single disease-based deep learning algorithms had been developed for the detection of diabetic retinopathy, age-related macular degeneration, and glaucoma. Here, we developed a deep learning platform (DLP) capable of detecting multiple common referable fundus diseases and conditions (39 classes) by using 249,620 fundus images marked with 275,543 labels from heterogenous sources. Our DLP achieved a frequency-weighted average F1 score of 0.923, sensitivity of 0.978, specificity of 0.996 and area under the receiver operating characteristic curve (AUC) of 0.9984 for multi-label classification in the primary test dataset and reached the average level of retina specialists. External multihospital test, public data test and tele-reading application also showed high efficiency for multiple retinal diseases and conditions detection. These results indicate that our DLP can be applied for retinal fundus disease triage, especially in remote areas around the world.
Collapse
Affiliation(s)
- Ling-Ping Cen
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Jie Ji
- Network & Information Centre, Shantou University, Shantou, Guangdong, China
- Shantou University Medical College, Shantou, Guangdong, China
- XuanShi Med Tech (Shanghai) Company Limited, Shanghai, China
| | - Jian-Wei Lin
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Si-Tong Ju
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Hong-Jie Lin
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Tai-Ping Li
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Yun Wang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Jian-Feng Yang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Yu-Fen Liu
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Shaoying Tan
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Li Tan
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Dongjie Li
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Yifan Wang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Dezhi Zheng
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Yongqun Xiong
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Hanfu Wu
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Jingjing Jiang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Zhenggen Wu
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Dingguo Huang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Tingkun Shi
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Binyao Chen
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Jianling Yang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Xiaoling Zhang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Li Luo
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Chukai Huang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Guihua Zhang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Yuqiang Huang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Tsz Kin Ng
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
- Shantou University Medical College, Shantou, Guangdong, China
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Shatin, Hong Kong
| | - Haoyu Chen
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Weiqi Chen
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
| | - Chi Pui Pang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Shatin, Hong Kong
| | - Mingzhi Zhang
- Joint Shantou International Eye Centre of Shantou University and The Chinese University of Hong Kong, Shantou, Guangdong, China.
| |
Collapse
|
34
|
Computed Tomography Angiography under Deep Learning in the Treatment of Atherosclerosis with Rapamycin. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:4543702. [PMID: 34336152 PMCID: PMC8321726 DOI: 10.1155/2021/4543702] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/24/2021] [Revised: 06/30/2021] [Accepted: 07/15/2021] [Indexed: 12/24/2022]
Abstract
The clinical characteristics and vascular computed tomography (CT) imaging characteristics of patients were explored so as to assist clinicians in diagnosing patients with atherosclerosis. 316 patients with atherosclerosis who were hospitalized for emergency treatment were treated with rapamycin (RAPA) in the hospital. A group of manually delineated left ventricular myocardia (LVM) on the patient's coronary computed tomography angiography (CCTA) were selected as the region of interest for imaging features extracted. The CCTA images of 80% of patients were randomly selected for training, and those of 20% of patients were used for verification. The correlation matrix method was used to remove redundant image omics features under different correlation thresholds. In the validation set, CCTA diagnostic parameters were about 40 times higher than the manually segmented data. The average dice similarity coefficient was 91.6%. The proposed method also produced a very small centroid distance (mean 1.058 mm, standard deviation 1.245 mm) and volume difference (mean 1.640), with a segmentation time of about 1.45 ± 0.51 s, compared to about 744.8 ± 117.49 s for physician manual segmentation. Therefore, the deep learning model effectively segmented the atherosclerotic lesion area, measured and assisted the diagnosis of future atherosclerosis clinical cases, improved medical efficiency, and accurately identified the patient's lesion area. It had great application potential in helping diagnosis and curative effect analysis of atherosclerosis.
Collapse
|
35
|
Wu JH, Liu TYA, Hsu WT, Ho JHC, Lee CC. Performance and Limitation of Machine Learning Algorithms for Diabetic Retinopathy Screening: Meta-analysis. J Med Internet Res 2021; 23:e23863. [PMID: 34407500 PMCID: PMC8406115 DOI: 10.2196/23863] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Revised: 11/19/2020] [Accepted: 04/30/2021] [Indexed: 12/23/2022] Open
Abstract
Background Diabetic retinopathy (DR), whose standard diagnosis is performed by human experts, has high prevalence and requires a more efficient screening method. Although machine learning (ML)–based automated DR diagnosis has gained attention due to recent approval of IDx-DR, performance of this tool has not been examined systematically, and the best ML technique for use in a real-world setting has not been discussed. Objective The aim of this study was to systematically examine the overall diagnostic accuracy of ML in diagnosing DR of different categories based on color fundus photographs and to determine the state-of-the-art ML approach. Methods Published studies in PubMed and EMBASE were searched from inception to June 2020. Studies were screened for relevant outcomes, publication types, and data sufficiency, and a total of 60 out of 2128 (2.82%) studies were retrieved after study selection. Extraction of data was performed by 2 authors according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and the quality assessment was performed according to the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Meta-analysis of diagnostic accuracy was pooled using a bivariate random effects model. The main outcomes included diagnostic accuracy, sensitivity, and specificity of ML in diagnosing DR based on color fundus photographs, as well as the performances of different major types of ML algorithms. Results The primary meta-analysis included 60 color fundus photograph studies (445,175 interpretations). Overall, ML demonstrated high accuracy in diagnosing DR of various categories, with a pooled area under the receiver operating characteristic (AUROC) ranging from 0.97 (95% CI 0.96-0.99) to 0.99 (95% CI 0.98-1.00). The performance of ML in detecting more-than-mild DR was robust (sensitivity 0.95; AUROC 0.97), and by subgroup analyses, we observed that robust performance of ML was not limited to benchmark data sets (sensitivity 0.92; AUROC 0.96) but could be generalized to images collected in clinical practice (sensitivity 0.97; AUROC 0.97). Neural network was the most widely used method, and the subgroup analysis revealed a pooled AUROC of 0.98 (95% CI 0.96-0.99) for studies that used neural networks to diagnose more-than-mild DR. Conclusions This meta-analysis demonstrated high diagnostic accuracy of ML algorithms in detecting DR on color fundus photographs, suggesting that state-of-the-art, ML-based DR screening algorithms are likely ready for clinical applications. However, a significant portion of the earlier published studies had methodology flaws, such as the lack of external validation and presence of spectrum bias. The results of these studies should be interpreted with caution.
Collapse
Affiliation(s)
- Jo-Hsuan Wu
- Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, CA, United States
| | - T Y Alvin Liu
- Retina Division, Wilmer Eye Institute, The Johns Hopkins Medicine, Baltimore, MD, United States
| | - Wan-Ting Hsu
- Harvard TH Chan School of Public Health, Boston, MA, United States
| | | | - Chien-Chang Lee
- Health Data Science Research Group, National Taiwan University Hospital, Taipei, Taiwan.,The Centre for Intelligent Healthcare, National Taiwan University Hospital, Taipei, Taiwan.,Department of Emergency Medicine, National Taiwan University Hospital, Taipei, Taiwan
| |
Collapse
|
36
|
Shah A, Clarida W, Amelon R, Hernaez-Ortega MC, Navea A, Morales-Olivas J, Dolz-Marco R, Verbraak F, Jorda PP, van der Heijden AA, Peris Martinez C. Validation of Automated Screening for Referable Diabetic Retinopathy With an Autonomous Diagnostic Artificial Intelligence System in a Spanish Population. J Diabetes Sci Technol 2021; 15:655-663. [PMID: 32174153 PMCID: PMC8120039 DOI: 10.1177/1932296820906212] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
PURPOSE The purpose of this study is to compare the diagnostic performance of an autonomous artificial intelligence (AI) system for the diagnosis of referable diabetic retinopathy (RDR) to manual grading by Spanish ophthalmologists. METHODS Subjects with type 1 and 2 diabetes participated in a diabetic retinopathy (DR) screening program in 2011 to 2012 in Valencia (Spain), and two images per eye were collected according to their standard protocol. Mydriatic drops were used in all patients. Retinal images-one disc and one fovea centered-were obtained under the Medical Research Ethics Committee approval and de-identified. Exams were graded by the autonomous AI system (IDx-DR, Coralville, Iowa, United States), and manually by masked ophthalmologists using adjudication. The outputs of the AI system and manual adjudicated grading were compared using sensitivity and specificity for diagnosis of both RDR and vision-threatening diabetic retinopathy (VTDR). RESULTS A total of 2680 subjects were included in the study. According to manual grading, prevalence of RDR was 111/2680 (4.14%) and of VTDR was 69/2680 (2.57%). Against manual grading, the AI system had a 100% (95% confidence interval [CI]: 97%-100%) sensitivity and 81.82% (95% CI: 80%-83%) specificity for RDR, and a 100% (95% CI: 95%-100%) sensitivity and 94.64% (95% CI: 94%-95%) specificity for VTDR. CONCLUSION Compared to manual grading by ophthalmologists, the autonomous diagnostic AI system had high sensitivity (100%) and specificity (82%) for diagnosing RDR and macular edema in people with diabetes in a screening program. Because of its immediate, point of care diagnosis, autonomous diagnostic AI has the potential to increase the accessibility of RDR screening in primary care settings.
Collapse
Affiliation(s)
- Abhay Shah
- Dx Technologies Inc, Coralville, IA,
USA
| | | | | | | | - Amparo Navea
- FISABIO OFTALMOLOGIA MEDICA (FOM),
Valencia, Spain
- Instituto de la retina, Valencia,
Spain
- Universidad Cardenal Herrera CEU,
Valencia, Spain
| | | | | | - Frank Verbraak
- Department of Ophthalmology, VUmc,
Amsterdam University Medical Centers, The Netherlands
| | | | - Amber A. van der Heijden
- Department of General Practice and
Elderly Care Medicine, VU University Medical Centre, Amsterdam, The
Netherlands
- Amsterdam Public Health Research
Institute, VU University Medical Centre, The Netherlands
| | | |
Collapse
|
37
|
Aggarwal R, Sounderajah V, Martin G, Ting DSW, Karthikesalingam A, King D, Ashrafian H, Darzi A. Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digit Med 2021; 4:65. [PMID: 33828217 PMCID: PMC8027892 DOI: 10.1038/s41746-021-00438-z] [Citation(s) in RCA: 237] [Impact Index Per Article: 79.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2020] [Accepted: 02/25/2021] [Indexed: 12/19/2022] Open
Abstract
Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.
Collapse
Affiliation(s)
- Ravi Aggarwal
- Institute of Global Health Innovation, Imperial College London, London, UK
| | | | - Guy Martin
- Institute of Global Health Innovation, Imperial College London, London, UK
| | - Daniel S W Ting
- Singapore Eye Research Institute, Singapore National Eye Center, Singapore, Singapore
| | | | - Dominic King
- Institute of Global Health Innovation, Imperial College London, London, UK
| | - Hutan Ashrafian
- Institute of Global Health Innovation, Imperial College London, London, UK.
| | - Ara Darzi
- Institute of Global Health Innovation, Imperial College London, London, UK
| |
Collapse
|
38
|
Vaghefi E, Yang S, Xie L, Hill S, Schmiedel O, Murphy R, Squirrell D. THEIA™ development, and testing of artificial intelligence-based primary triage of diabetic retinopathy screening images in New Zealand. Diabet Med 2021; 38:e14386. [PMID: 32794618 PMCID: PMC8048953 DOI: 10.1111/dme.14386] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/11/2020] [Indexed: 12/14/2022]
Abstract
AIM To develop and evaluate an artificial intelligence triage system with high sensitivity for detecting referable diabetic retinopathy and maculopathy, while maintaining high specificity for non-referable disease, for clinical implementation within the New Zealand national diabetic retinopathy screening programme. METHODS The THEIA™ artificial intelligence system for retinopathy and maculopathy screening, was developed at Toku Eyes using routinely collected retinal screening datasets from two of the largest district health boards in Auckland, New Zealand: the Auckland District Health Board and the Counties Manukau District Health Board. All retinal images from consecutive individuals receiving retinal screening between January 2009 and December 2018 were used. Images were labelled as non-sight-threatening, potentially referable or sight-threatening for New Zealand implementation, or as referable (potentially referable + sight-threatening)/non-referable (non-sight-threatening) for global comparison. RESULTS Data from 32 354 unique people with diabetes (63 843 when including multiple visits) were available, of which 95-97%, 0.9-2.4% and 1.1-3.1% were categorized as non-sight-threatening, potentially referable and sight-threatening, respectively. Using the referable/non-referable categories, THEIA achieved overall sensitivity of 94% (95% CI 92-95) in the Auckland District Health Board and 95% (95% CI 92-97) in the Counties Manukau District Health Board datasets, while preserving specificity of 63% (95% CI 62-64) for the Auckland District Health Board and 61% (95% CI 60-62) for the Counties Manukau District Health Board. Implementing THEIA into a New Zealand national diabetic screening programme could significantly reduce the manual grading load. CONCLUSION THEIA, an artificial intelligence tool to assist in clinical decision-making, tailored to the needs of the New Zealand national diabetic screening programme, delivered high sensitivity for detecting referable retinopathy within the multi-ethnic New Zealand population with diabetes.
Collapse
Affiliation(s)
- E. Vaghefi
- Toku EyesAucklandNew Zealand
- School of Optometry and Vision ScienceAucklandNew Zealand
| | - S. Yang
- School of Optometry and Vision ScienceAucklandNew Zealand
- School of Computer SciencesUniversity of AucklandAucklandNew Zealand
| | - L. Xie
- School of Optometry and Vision ScienceAucklandNew Zealand
| | - S. Hill
- Department of OphthalmologyAucklandNew Zealand
| | - O. Schmiedel
- Auckland Diabetes CentreAuckland District Health Board
| | - R. Murphy
- School of MedicineAucklandNew Zealand
| | - D. Squirrell
- Toku EyesAucklandNew Zealand
- Department of OphthalmologyAucklandNew Zealand
| |
Collapse
|
39
|
Li YH, Sheu WHH, Chou CC, Lin CH, Cheng YS, Wang CY, Wu CL, Lee IT. The Clinical Influence after Implementation of Convolutional Neural Network-Based Software for Diabetic Retinopathy Detection in the Primary Care Setting. Life (Basel) 2021; 11:life11030200. [PMID: 33807545 PMCID: PMC8035657 DOI: 10.3390/life11030200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Revised: 02/04/2021] [Accepted: 03/03/2021] [Indexed: 11/16/2022] Open
Abstract
Deep learning-based software is developed to assist physicians in terms of diagnosis; however, its clinical application is still under investigation. We integrated deep-learning-based software for diabetic retinopathy (DR) grading into the clinical workflow of an endocrinology department where endocrinologists grade for retinal images and evaluated the influence of its implementation. A total of 1432 images from 716 patients and 1400 images from 700 patients were collected before and after implementation, respectively. Using the grading by ophthalmologists as the reference standard, the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) to detect referable DR (RDR) were 0.91 (0.87-0.96), 0.90 (0.87-0.92), and 0.90 (0.87-0.93) at the image level; and 0.91 (0.81-0.97), 0.84 (0.80-0.87), and 0.87 (0.83-0.91) at the patient level. The monthly RDR rate dropped from 55.1% to 43.0% after implementation. The monthly percentage of finishing grading within the allotted time increased from 66.8% to 77.6%. There was a wide range of agreement values between the software and endocrinologists after implementation (kappa values of 0.17-0.65). In conclusion, we observed the clinical influence of deep-learning-based software on graders without the retinal subspecialty. However, the validation using images from local datasets is recommended before clinical implementation.
Collapse
Affiliation(s)
- Yu-Hsuan Li
- Division of Endocrinology and Metabolism, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (Y.-H.L.); (W.H.-H.S.)
- Department of Computer Science & Information Engineering, National Taiwan University, Taipei 10617, Taiwan
| | - Wayne Huey-Herng Sheu
- Division of Endocrinology and Metabolism, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (Y.-H.L.); (W.H.-H.S.)
- School of Medicine, National Yang-Ming University, Taipei 11221, Taiwan
- Rong Hsing Research Center for Translational Medicine, National Chung Hsing University, Taichung 40227, Taiwan
| | - Chien-Chih Chou
- Department of Ophthalmology, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (C.-C.C.); (C.-H.L.); (Y.-S.C.); (C.-Y.W.)
| | - Chun-Hsien Lin
- Department of Ophthalmology, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (C.-C.C.); (C.-H.L.); (Y.-S.C.); (C.-Y.W.)
| | - Yuan-Shao Cheng
- Department of Ophthalmology, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (C.-C.C.); (C.-H.L.); (Y.-S.C.); (C.-Y.W.)
| | - Chun-Yuan Wang
- Department of Ophthalmology, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (C.-C.C.); (C.-H.L.); (Y.-S.C.); (C.-Y.W.)
| | - Chieh Liang Wu
- Department of Critical Care Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan;
- Department of Automatic Control Engineering, Feng Chia University, Taichung 40703, Taiwan
| | - I.-Te Lee
- Division of Endocrinology and Metabolism, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan; (Y.-H.L.); (W.H.-H.S.)
- School of Medicine, National Yang-Ming University, Taipei 11221, Taiwan
- School of Medicine, Chung Shan Medical University, Taichung 40201, Taiwan
- College of Science, Tunghai University, Taichung 40704, Taiwan
- Correspondence:
| |
Collapse
|
40
|
Wolf RM, Liu TYA, Thomas C, Prichett L, Zimmer-Galler I, Smith K, Abramoff MD, Channa R. The SEE Study: Safety, Efficacy, and Equity of Implementing Autonomous Artificial Intelligence for Diagnosing Diabetic Retinopathy in Youth. Diabetes Care 2021; 44:781-787. [PMID: 33479160 DOI: 10.2337/dc20-1671] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/05/2020] [Accepted: 12/23/2020] [Indexed: 02/03/2023]
Abstract
OBJECTIVE Diabetic retinopathy (DR) is a leading cause of vision loss worldwide. Screening for DR is recommended in children and adolescents, but adherence is poor. Recently, autonomous artificial intelligence (AI) systems have been developed for early detection of DR and have been included in the American Diabetes Association's guidelines for screening in adults. We sought to determine the diagnostic efficacy of autonomous AI for the diabetic eye exam in youth with diabetes. RESEARCH DESIGN AND METHODS In this prospective study, point-of-care diabetic eye exam was implemented using a nonmydriatic fundus camera with an autonomous AI system for detection of DR in a multidisciplinary pediatric diabetes center. Sensitivity, specificity, and diagnosability of AI was compared with consensus grading by retinal specialists, who were masked to AI output. Adherence to screening guidelines was measured before and after AI implementation. RESULTS Three hundred ten youth with diabetes aged 5-21 years were included, of whom 4.2% had DR. Diagnosability of AI was 97.5% (302 of 310). The sensitivity and specificity of AI to detect more-than-mild DR was 85.7% (95% CI 42.1-99.6%) and 79.3% (74.3-83.8%), respectively, compared with the reference standard as defined by retina specialists. Adherence improved from 49% to 95% after AI implementation. CONCLUSIONS Use of a nonmydriatic fundus camera with autonomous AI was safe and effective for the diabetic eye exam in youth in our study. Adherence to screening guidelines improved with AI implementation. As the prevalence of diabetes increases in youth and adherence to screening guidelines remains suboptimal, effective strategies for diabetic eye exams in this population are needed.
Collapse
Affiliation(s)
- Risa M Wolf
- Division of Endocrinology, Department of Pediatrics, Johns Hopkins School of Medicine, Baltimore, MD
| | - T Y Alvin Liu
- Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, MD
| | - Chrystal Thomas
- Division of Endocrinology, Department of Pediatrics, Johns Hopkins School of Medicine, Baltimore, MD
| | - Laura Prichett
- Biostatistics, Epidemiology, and Data Management Core, Johns Hopkins School of Medicine, Baltimore, MD
| | | | - Kerry Smith
- Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, MD
| | - Michael D Abramoff
- Department of Ophthalmology and Visual Sciences, The University of Iowa, Iowa City, IA.,IDx, Coralville, IA.,Iowa City VA Medical Center, Iowa City, IA.,Department of Biomedical Engineering, The University of Iowa, Iowa City, IA.,Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA
| | - Roomasa Channa
- Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, MD .,Department of Ophthalmology and Visual Sciences, University of Wisconsin, Madison, WI
| |
Collapse
|
41
|
Wolf RM, Channa R, Abramoff MD, Lehmann HP. Cost-effectiveness of Autonomous Point-of-Care Diabetic Retinopathy Screening for Pediatric Patients With Diabetes. JAMA Ophthalmol 2021; 138:1063-1069. [PMID: 32880616 DOI: 10.1001/jamaophthalmol.2020.3190] [Citation(s) in RCA: 42] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Importance Screening for diabetic retinopathy is recommended for children with type 1 diabetes (T1D) and type 2 diabetes (T2D), yet screening rates remain low. Point-of-care diabetic retinopathy screening using autonomous artificial intelligence (AI) has become available, providing immediate results in the clinic setting, but the cost-effectiveness of this strategy compared with standard examination is unknown. Objective To assess the cost-effectiveness of detecting and treating diabetic retinopathy and its sequelae among children with T1D and T2D using AI diabetic retinopathy screening vs standard screening by an eye care professional (ECP). Design, Setting, and Participants In this economic evaluation, parameter estimates were obtained from the literature from 1994 to 2019 and assessed from March 2019 to January 2020. Parameters included out-of-pocket cost for autonomous AI screening, ophthalmology visits, and treating diabetic retinopathy; probability of undergoing standard retinal examination; relative odds of undergoing screening; and sensitivity, specificity, and diagnosability of the ECP screening examination and autonomous AI screening. Main Outcomes and Measures Costs or savings to the patient based on mean patient payment for diabetic retinopathy screening examination and cost-effectiveness based on costs or savings associated with the number of true-positive results identified by diabetic retinopathy screening. Results In this study, the expected true-positive proportions for standard ophthalmologic screening by an ECP were 0.006 for T1D and 0.01 for T2D, and the expected true-positive proportions for autonomous AI were 0.03 for T1D and 0.04 for T2D. The base case scenario of 20% adherence estimated that use of autonomous AI would result in a higher mean patient payment ($8.52 for T1D and $10.85 for T2D) than conventional ECP screening ($7.91 for T1D and $8.20 for T2D). However, autonomous AI screening was the preferred strategy when at least 23% of patients adhered to diabetic retinopathy screening. Conclusions and Relevance These results suggest that point-of-care diabetic retinopathy screening using autonomous AI systems is effective and cost saving for children with diabetes and their caregivers at recommended adherence rates.
Collapse
Affiliation(s)
- Risa M Wolf
- Department of Pediatrics, Division of Pediatric Endocrinology, Johns Hopkins School of Medicine, Baltimore, Maryland
| | - Roomasa Channa
- Department of Ophthalmology, Baylor College of Medicine, Houston, Texas
| | - Michael D Abramoff
- Department of Ophthalmology and Visual Sciences, The University of Iowa, Iowa City.,Digital Diagnostics, Coralville, Iowa.,Iowa City Veterans Affairs Medical Center, Iowa City, Iowa.,Department of Biomedical Engineering, The University of Iowa, Iowa City.,Department of Electrical and Computer Engineering, The University of Iowa, Iowa City
| | - Harold P Lehmann
- Division of Health Sciences Informatics, Johns Hopkins School of Medicine, Baltimore, Maryland
| |
Collapse
|
42
|
Benjamin JE, Sun J, Cohen D, Matz J, Barbera A, Henderer J, Cheng L, Grachevskaya J, Shah R, Zhang Y. A 15 month experience with a primary care-based telemedicine screening program for diabetic retinopathy. BMC Ophthalmol 2021; 21:70. [PMID: 33541295 PMCID: PMC7859899 DOI: 10.1186/s12886-021-01828-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2020] [Accepted: 01/21/2021] [Indexed: 11/10/2022] Open
Abstract
Background Using telemedicine for diabetic retinal screening is becoming popular especially amongst at-risk urban communities with poor access to care. The goal of the diabetic telemedicine project at Temple University Hospital is to improve cost-effective access to appropriate retinal care to those in need of close monitoring and/or treatment. Methods This will be a retrospective review of 15 months of data from March 2016 to May 2017. We will investigate how many patients were screened, how interpretable the photographs were, how often the photographs generated a diagnosis of diabetic retinopathy (DR) based on the screening photo, and how many patients followed-up for an exam in the office, if indicated. Results Six-hundred eighty-nine (689) digital retinal screening exams on 1377 eyes of diabetic patients were conducted in Temple’s primary care clinic. The majority of the photographs were read to have no retinopathy (755, 54.8%). Among all of the screening exams, 357 (51.8%) triggered a request for a referral to ophthalmology. Four-hundred forty-nine (449, 32.6%) of the photos were felt to be uninterpretable by the clinician. Referrals were meant to be requested for DR found in one or both eyes, inability to assess presence of retinopathy in one or both eyes, or for suspicion of a different ophthalmic diagnosis. Sixty-seven patients (9.7%) were suspected to have another ophthalmic condition based on other findings in the retinal photographs. Among the 34 patients that were successfully completed a referral visit to Temple ophthalmology, there was good concordance between the level of DR detected by their screening fundus photographs and visit diagnosis. Conclusions Although a little more than half of the patients did not have diabetic eye disease, about half needed a referral to ophthalmology. However, only 9.5% of the referral-warranted exams actually received an eye exam. Mere identification of referral-warranted diabetic retinopathy and other ophthalmic conditions is not enough. A successful telemedicine screening program must close the communication gap between screening and diagnosis by reviewer to provide timely follow-up by eye care specialists.
Collapse
Affiliation(s)
- James E Benjamin
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Justin Sun
- Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Devin Cohen
- Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Joseph Matz
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Angela Barbera
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Jeffrey Henderer
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Lorrie Cheng
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Julia Grachevskaya
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Rajnikant Shah
- Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA
| | - Yi Zhang
- Department of Ophthalmology, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, USA.
| |
Collapse
|
43
|
Liaw WR, Westfall JM, Williamson TS, Jabbarpour Y, Bazemore A. Primary Care: The Actual Intelligence Required for Artificial Intelligence to Advance Health Care and Improve Health (Preprint). JMIR Med Inform 2021; 10:e27691. [PMID: 35258464 PMCID: PMC8941433 DOI: 10.2196/27691] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 03/30/2021] [Accepted: 02/06/2022] [Indexed: 11/17/2022] Open
Abstract
With conversational agents triaging symptoms, cameras aiding diagnoses, and remote sensors monitoring vital signs, the use of artificial intelligence (AI) outside of hospitals has the potential to improve health, according to a recently released report from the National Academy of Medicine. Despite this promise, the success of AI is not guaranteed, and stakeholders need to be involved with its development to ensure that the resulting tools can be easily used by clinicians, protect patient privacy, and enhance the value of the care delivered. A crucial stakeholder group missing from the conversation is primary care. As the nation’s largest delivery platform, primary care will have a powerful impact on whether AI is adopted and subsequently exacerbates health disparities. To leverage these benefits, primary care needs to serve as a medical home for AI, broaden its teams and training, and build on government initiatives and funding.
Collapse
Affiliation(s)
- Winston R Liaw
- Department of Health Systems and Population Health Sciences, University of Houston, Houston, TX, United States
| | - John M Westfall
- Robert Graham Center for Policy Studies in Primary Care, Washington, DC, United States
| | - Tyler S Williamson
- Centre for Health Informatics, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Yalda Jabbarpour
- Robert Graham Center for Policy Studies in Primary Care, Washington, DC, United States
| | - Andrew Bazemore
- Center for Professionalism and Value in Health Care, American Board of Family Medicine, Washington, DC, United States
| |
Collapse
|
44
|
Lu L, Ren P, Lu Q, Zhou E, Yu W, Huang J, He X, Han W. Analyzing fundus images to detect diabetic retinopathy (DR) using deep learning system in the Yangtze River delta region of China. ANNALS OF TRANSLATIONAL MEDICINE 2021; 9:226. [PMID: 33708853 PMCID: PMC7940941 DOI: 10.21037/atm-20-3275] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background This study aimed to establish and evaluate an artificial intelligence-based deep learning system (DLS) for automatic detection of diabetic retinopathy. This could be important in developing an advanced tele-screening system for diabetic retinopathy. Methods A DLS with a convolutional neural network was developed to recognize fundus images of referable diabetic retinopathy. A total data set of 41,866 color fundus images were obtained from 17 cities in the Yangtze River Delta Urban Agglomeration (YRDUA). Five experienced retinal specialists and 15 ophthalmologists were recruited to verify images. For training, 80% of the data set was used, and the other 20% served as the validation data set. To effectively understand the learning process, the DLS automatically superimposed a heatmap on the original image. The regions utilized by the DLS were highlighted for diagnosis. Results Using the local validation data set, the DLS achieved an area under the curve of 0.9824. Based on the manual screening criteria, an operating point was set at about 0.9 sensitivity to evaluate the DLS. Specificity was recorded at 0.9609 and sensitivity was 0.9003. The DLSs showed excellent reliability, repeatability, and high efficiency. After analyzing the misclassification, it was found that 88.6% of the false-positives were mild non-proliferative diabetic retinopathy (NPDR) whereas, 81.6% of the false-negatives were intraretinal microvascular abnormalities. Conclusions The DLS efficiently detected fundus images from complex sources in the real world. Incorporating DLS technology in tele-screening will advance the current screening programs to offer a cost-effective and time-efficient solution for detecting diabetic retinopathy.
Collapse
Affiliation(s)
- Li Lu
- Department of Ophthalmology, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China.,Department of Ophthalmology, The First Affiliated Hospital of University of Science and Technology of China, Hefei, China
| | - Peifang Ren
- Department of Ophthalmology, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Qianyi Lu
- Department of Ophthalmology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Enliang Zhou
- Department of Ophthalmology, The First Affiliated Hospital of University of Science and Technology of China, Hefei, China
| | - Wangshu Yu
- Department of Ophthalmology, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Jiani Huang
- Department of Ophthalmology, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Xiaoying He
- Department of Ophthalmology, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Wei Han
- Department of Ophthalmology, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| |
Collapse
|
45
|
Fujihara K, Matsubayashi Y, Harada Yamada M, Yamamoto M, Iizuka T, Miyamura K, Hasegawa Y, Maegawa H, Kodama S, Yamazaki T, Sone H. Machine Learning Approach to Decision Making for Insulin Initiation in Japanese Patients With Type 2 Diabetes (JDDM 58): Model Development and Validation Study. JMIR Med Inform 2021; 9:e22148. [PMID: 33502325 PMCID: PMC7875702 DOI: 10.2196/22148] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2020] [Revised: 11/23/2020] [Accepted: 12/12/2020] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND Applications of machine learning for the early detection of diseases for which a clear-cut diagnostic gold standard exists have been evaluated. However, little is known about the usefulness of machine learning approaches in the decision-making process for decisions such as insulin initiation by diabetes specialists for which no absolute standards exist in clinical settings. OBJECTIVE The objectives of this study were to examine the ability of machine learning models to predict insulin initiation by specialists and whether the machine learning approach could support decision making by general physicians for insulin initiation in patients with type 2 diabetes. METHODS Data from patients prescribed hypoglycemic agents from December 2009 to March 2015 were extracted from diabetes specialists' registries, resulting in a sample size of 4860 patients who had received initial monotherapy with either insulin (n=293) or noninsulin (n=4567). Neural network output was insulin initiation ranging from 0 to 1 with a cutoff of >0.5 for the dichotomous classification. Accuracy, recall, and area under the receiver operating characteristic curve (AUC) were calculated to compare the ability of machine learning models to make decisions regarding insulin initiation to the decision-making ability of logistic regression and general physicians. By comparing the decision-making ability of machine learning and logistic regression to that of general physicians, 7 cases were chosen based on patient information as the gold standard based on the agreement of 8 of the 9 specialists. RESULTS The AUCs, accuracy, and recall of logistic regression were higher than those of machine learning (AUCs of 0.89-0.90 for logistic regression versus 0.67-0.74 for machine learning). When the examination was limited to cases receiving insulin, discrimination by machine learning was similar to that of logistic regression analysis (recall of 0.05-0.68 for logistic regression versus 0.11-0.52 for machine learning). Accuracies of logistic regression, a machine learning model (downsampling ratio of 1:8), and general physicians were 0.80, 0.70, and 0.66, respectively, for 43 randomly selected cases. For the 7 gold standard cases, the accuracies of logistic regression and the machine learning model were 1.00 and 0.86, respectively, with a downsampling ratio of 1:8, which were higher than the accuracy of general physicians (ie, 0.43). CONCLUSIONS Although we found no superior performance of machine learning over logistic regression, machine learning had higher accuracy in prediction of insulin initiation than general physicians, defined by diabetes specialists' choice of the gold standard. Further study is needed before the use of machine learning-based decision support systems for insulin initiation can be incorporated into clinical practice.
Collapse
Affiliation(s)
- Kazuya Fujihara
- Department of Internal Medicine, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Yasuhiro Matsubayashi
- Department of Internal Medicine, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Mayuko Harada Yamada
- Department of Internal Medicine, Faculty of Medicine, Niigata University, Niigata, Japan
| | - Masahiko Yamamoto
- Department of Internal Medicine, Faculty of Medicine, Niigata University, Niigata, Japan
| | | | | | | | - Hiroshi Maegawa
- Department of Internal Medicine, Shiga University of Medical Science, Shiga, Japan
| | - Satoru Kodama
- Department of Internal Medicine, Faculty of Medicine, Niigata University, Niigata, Japan
| | | | - Hirohito Sone
- Department of Internal Medicine, Faculty of Medicine, Niigata University, Niigata, Japan
| |
Collapse
|
46
|
Straňák Z, Penčák M, Veith M. ARTEFICIAL INTELLIGENCE IN DIABETIC RETINOPATHY SCREENING. A REVIEW. CESKA A SLOVENSKA OFTALMOLOGIE : CASOPIS CESKE OFTALMOLOGICKE SPOLECNOSTI A SLOVENSKE OFTALMOLOGICKE SPOLECNOSTI 2021; 77:224-231. [PMID: 34666491 DOI: 10.31348/2021/6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
OBJECTIVE The aim of this comprehensive paper is to acquaint the readers with evaluation of the retinal images using the arteficial intelligence (AI). Main focus of the paper is diabetic retinophaty (DR) screening. The basic principles of the artificial intelligence and algorithms that are already used in clinical practice or are shortly before approval will be described. METHODOLOGY Describing the basic characteristics and mechanisms of different approaches to the use of AI and subsequently literary minireview clarifying the current state of knowledge in the area. RESULTS Modern systems for screening diabetic retinopathy using deep neural networks achieve a sensitivity and specificity of over 80 % in most published studies. The results of specific studies vary depending on the definition of the gold standard, number of images tested and on the evaluated parameters. CONCLUSION Evaluation of images using AI will speed up and streamline the diagnosis of DR. The use of AI will allow to keep the quality of the eye care at least on the same level despite the raising number of the patients with diabetes.
Collapse
|
47
|
Cao B, Zhang N, Zhang Y, Fu Y, Zhao D. Plasma cytokines for predicting diabetic retinopathy among type 2 diabetic patients via machine learning algorithms. Aging (Albany NY) 2020; 13:1972-1988. [PMID: 33323553 PMCID: PMC7880388 DOI: 10.18632/aging.202168] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2020] [Accepted: 10/09/2020] [Indexed: 11/25/2022]
Abstract
AIMS This study aimed to investigate changes of plasma cytokines and to develop machine learning classifiers for predicting non-proliferative diabetic retinopathy among type 2 diabetes mellitus patients. RESULTS There were 12 plasma cytokines significantly higher in the non-proliferative diabetic retinopathy group in the pilot cohort. The validation cohort showed that angiopoietin 1, platelet-derived growth factor-BB, tissue inhibitors of metalloproteinase 2 and vascular endothelial growth factor receptor 2 were significantly higher in the NPDR group. Machine learning algorithms using the random forest yielded the best performance, with sensitivity of 92.3%, specificity of 75%, PPV of 82.8%, NPV of 88.2% and area under the curve of 0.84. CONCLUSIONS Plasma angiopoietin 1, platelet-derived growth factor-BB, and vascular endothelial growth factor receptor 2 were associated with presence of non-proliferative diabetic retinopathy and may be good biomarkers that play important roles in pathophysiology of diabetic retinopathy. MATERIALS AND METHODS In pilot cohort, 60 plasma cytokines were simultaneously measured. In validation cohort, angiopoietin 1, CXC-chemokine ligand 16, platelet-derived growth factor-BB, tissue inhibitors of metalloproteinase 1, tissue inhibitors of metalloproteinase 2, and vascular endothelial growth factor receptor 2 were validated using ELISA kits. Machine learning algorithms were developed to build a prediction model for non-proliferative diabetic retinopathy.
Collapse
Affiliation(s)
- Bin Cao
- Center for Endocrine Metabolism and Immune Diseases, Beijing Luhe Hospital, Capital Medical University, Beijing 101149, China.,Beijing Key Laboratory of Diabetes Research and Care, Beijing 101149, China
| | - Ning Zhang
- Center for Endocrine Metabolism and Immune Diseases, Beijing Luhe Hospital, Capital Medical University, Beijing 101149, China.,Beijing Key Laboratory of Diabetes Research and Care, Beijing 101149, China
| | - Yuanyuan Zhang
- Center for Endocrine Metabolism and Immune Diseases, Beijing Luhe Hospital, Capital Medical University, Beijing 101149, China.,Beijing Key Laboratory of Diabetes Research and Care, Beijing 101149, China
| | - Ying Fu
- Center for Endocrine Metabolism and Immune Diseases, Beijing Luhe Hospital, Capital Medical University, Beijing 101149, China.,Beijing Key Laboratory of Diabetes Research and Care, Beijing 101149, China
| | - Dong Zhao
- Center for Endocrine Metabolism and Immune Diseases, Beijing Luhe Hospital, Capital Medical University, Beijing 101149, China.,Beijing Key Laboratory of Diabetes Research and Care, Beijing 101149, China
| |
Collapse
|
48
|
Essentials of a Robust Deep Learning System for Diabetic Retinopathy Screening: A Systematic Literature Review. J Ophthalmol 2020. [DOI: 10.1155/2020/8841927] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
This systematic review was performed to identify the specifics of an optimal diabetic retinopathy deep learning algorithm, by identifying the best exemplar research studies of the field, whilst highlighting potential barriers to clinical implementation of such an algorithm. Searching five electronic databases (Embase, MEDLINE, Scopus, PubMed, and the Cochrane Library) returned 747 unique records on 20 December 2019. Predetermined inclusion and exclusion criteria were applied to the search results, resulting in 15 highest-quality publications. A manual search through the reference lists of relevant review articles found from the database search was conducted, yielding no additional records. A validation dataset of the trained deep learning algorithms was used for creating a set of optimal properties for an ideal diabetic retinopathy classification algorithm. Potential limitations to the clinical implementation of such systems were identified as lack of generalizability, limited screening scope, and data sovereignty issues. It is concluded that deep learning algorithms in the context of diabetic retinopathy screening have reported impressive results. Despite this, the potential sources of limitations in such systems must be evaluated carefully. An ideal deep learning algorithm should be clinic-, clinician-, and camera-agnostic; complying with the local regulation for data sovereignty, storage, privacy, and reporting; whilst requiring minimum human input.
Collapse
|
49
|
Peris-Martínez C, Shaha A, Clarida W, Amelon R, Hernáez-Ortega MC, Navea A, Morales-Olivas J, Dolz-Marco R, Pérez-Jordá P, Verbraak F, Heijden AAVD. Use in clinical practice of an automated screening method of diabetic retinopathy that can be derived using a diagnostic artificial intelligence system. ACTA ACUST UNITED AC 2020; 96:117-126. [PMID: 33153819 DOI: 10.1016/j.oftal.2020.08.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Revised: 07/30/2020] [Accepted: 08/03/2020] [Indexed: 10/23/2022]
Abstract
BACKGROUND AND OBJECTIVE To compare the diagnostic performance of an autonomous diagnostic artificial intelligence (AI) system for the diagnosis of derivable diabetic retinopathy (RDR) with manual classification. MATERIALS AND METHODS Patients with type 1 and type 2 diabetes participated in a diabetic retinopathy (DR) screening program between 2011-2012. 2 images of each eye were collected. Unidentifiable retinal images were obtained, one centered on the disc and one on the fovea. The exams were classified with the autonomous AI system and manually by anonymous ophthalmologists. The results of the AI system and manual classification were compared in terms of sensitivity and specificity for the diagnosis of both (RDR) and diabetic retinopathy with decreased vision (VTDR). RESULTS 10,257 retinal inages of 5,630 eyes of 2,680 subjects were included. According to the manual classification, the prevalence of RDR was 4.14% and that of VTDR 2.57%. The AI system recorded 100% (95% CI: 97-100%) sensitivity and 81.82% (95% CI: 80 -83%) specificity for RDR, and 100% (95% CI: 95-100%) of sensitivity and 94.64% (95% CI: 94-95%) of specificity for VTDR. CONCLUSIONS Compared to the manual classification, the autonomous diagnostic AI system registered a high sensitivity (100%) and specificity (82%) in the diagnosis of RDR and macular edema in people with diabetes. Due to its immediate diagnosis, the autonomous diagnostic AI system can increase the accessibility of RDR screening in primary care settings.
Collapse
Affiliation(s)
- Cristina Peris-Martínez
- FISABIO Oftalmología Médica (FOM), Valencia, España; Universidad de Valencia, Valencia, España.
| | - Abhay Shaha
- FISABIO Oftalmología Médica (FOM), Valencia, España; Universidad de Valencia, Valencia, España; IDx Technologies Inc., Coralville, United Sates of America; European Innovative Biomedicine Institute (EIBI), Castro-urdiales, España; Instituto de la retina, Valencia, España; Universidad Cardenal Herrera CEU, Valencia, España; Oftalvist, Valencia, España; Departamento de Oftalmología, VUmc, Centros Médicos de la Universidad de Ámsterdam, Ámsterdam, Países Bajos; Departamento de Medicina General y Geriátrica, Centro Médico de la Universidad VU, Ámsterdam, Países Bajos; Instituto de Investigación en Salud Pública de Ámsterdam, Centro Médico de la Universidad VU, Ámsterdam, Países Bajos
| | - Warren Clarida
- IDx Technologies Inc., Coralville, United Sates of America
| | - Ryan Amelon
- IDx Technologies Inc., Coralville, United Sates of America
| | | | - Amparo Navea
- Instituto de la retina, Valencia, España; Universidad Cardenal Herrera CEU, Valencia, España
| | | | | | | | - Frank Verbraak
- Departamento de Oftalmología, VUmc, Centros Médicos de la Universidad de Ámsterdam, Ámsterdam, Países Bajos
| | - Amber A van der Heijden
- Departamento de Medicina General y Geriátrica, Centro Médico de la Universidad VU, Ámsterdam, Países Bajos; Instituto de Investigación en Salud Pública de Ámsterdam, Centro Médico de la Universidad VU, Ámsterdam, Países Bajos
| |
Collapse
|
50
|
Yuasa S. [Deep learning-based system for the research of pluripotent stem cell-derived cells]. Nihon Yakurigaku Zasshi 2020; 155:309-311. [PMID: 32879171 DOI: 10.1254/fpj.20037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
Deep learning technology is rapidly advancing, and is now used to solve complex problems. induced pluripotent stem cells (iPSCs) can be used for several purposes such as regenerative medicine, disease modeling study and drug screening. It is inevitable to identify iPSC-derived differentiated cells in microscopy for any use. Here, we used deep learning to establish an automated method to identify endothelial cells derived from iPSCs, without the need for immunostaining or lineage tracing.
Collapse
Affiliation(s)
- Shinsuke Yuasa
- Department of Cardiology, Keio University School of Medicine
| |
Collapse
|