Biomarkers for Progression in Diabetic Retinopathy: Expanding Personalized Medicine through Integration of AI with Electronic Health Records

Automated Machine Learning for Predicting Diabetic Retinopathy Progression From Ultra-Widefield Retinal Images

Importance

Machine learning (ML) algorithms have the potential to identify eyes with early diabetic retinopathy (DR) at increased risk for disease progression.

Objective

To create and validate automated ML models (autoML) for DR progression from ultra-widefield (UWF) retinal images.

Design, Setting and Participants

Deidentified UWF images with mild or moderate nonproliferative DR (NPDR) with 3 years of longitudinal follow-up retinal imaging or evidence of progression within 3 years were used to develop automated ML models for predicting DR progression in UWF images. All images were collected from a tertiary diabetes-specific medical center retinal image dataset. Data were collected from July to September 2022.

Exposure

Automated ML models were generated from baseline on-axis 200° UWF retinal images. Baseline retinal images were labeled for progression based on centralized reading center evaluation of baseline and follow-up images according to the clinical Early Treatment Diabetic Retinopathy Study severity scale. Images for model development were split 8-1-1 for training, optimization, and testing to detect 1 or more steps of DR progression. Validation was performed using a 328-image set from the same patient population not used in model development.

Main Outcomes and Measures

Area under the precision-recall curve (AUPRC), sensitivity, specificity, and accuracy.

Results

A total of 1179 deidentified UWF images with mild (380 [32.2%]) or moderate (799 [67.8%]) NPDR were included. DR progression was present in half of the training set (590 of 1179 [50.0%]). The model's AUPRC was 0.717 for baseline mild NPDR and 0.863 for moderate NPDR. On the validation set for eyes with mild NPDR, sensitivity was 0.72 (95% CI, 0.57-0.83), specificity was 0.63 (95% CI, 0.57-0.69), prevalence was 0.15 (95% CI, 0.12-0.20), and accuracy was 64.3%; for eyes with moderate NPDR, sensitivity was 0.80 (95% CI, 0.70-0.87), specificity was 0.72 (95% CI, 0.66-0.76), prevalence was 0.22 (95% CI, 0.19-0.27), and accuracy was 73.8%. In the validation set, 6 of 9 eyes (75%) with mild NPDR and 35 of 41 eyes (85%) with moderate NPDR progressed 2 steps or more were identified. All 4 eyes with mild NPDR that progressed within 6 months and 1 year were identified, and 8 of 9 (89%) and 17 of 20 (85%) with moderate NPDR that progressed within 6 months and 1 year, respectively, were identified.

Conclusions and Relevance

This study demonstrates the accuracy and feasibility of automated ML models for identifying DR progression developed using UWF images, especially for prediction of 2-step or greater DR progression within 1 year. Potentially, the use of ML algorithms may refine the risk of disease progression and identify those at highest short-term risk, thus reducing costs and improving vision-related outcomes.

Performance of Automated Machine Learning for Diabetic Retinopathy Image Classification from Multi-field Handheld Retinal Images

PURPOSE

To create and validate code-free automated deep learning models (AutoML) for diabetic retinopathy (DR) classification from handheld retinal images.

DESIGN

Prospective development and validation of AutoML models for DR image classification.

PARTICIPANTS

A total of 17 829 deidentified retinal images from 3566 eyes with diabetes, acquired using handheld retinal cameras in a community-based DR screening program.

METHODS

AutoML models were generated based on previously acquired 5-field (macula-centered, disc-centered, superior, inferior, and temporal macula) handheld retinal images. Each individual image was labeled using the International DR and diabetic macular edema (DME) Classification Scale by 4 certified graders at a centralized reading center under oversight by a senior retina specialist. Images for model development were split 8-1-1 for training, optimization, and testing to detect referable DR ([refDR], defined as moderate nonproliferative DR or worse or any level of DME). Internal validation was performed using a published image set from the same patient population (N = 450 images from 225 eyes). External validation was performed using a publicly available retinal imaging data set from the Asia Pacific Tele-Ophthalmology Society (N = 3662 images).

MAIN OUTCOME MEASURES

Area under the precision-recall curve (AUPRC), sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 scores.

RESULTS

Referable DR was present in 17.3%, 39.1%, and 48.0% of the training set, internal validation, and external validation sets, respectively. The model's AUPRC was 0.995 with a precision and recall of 97% using a score threshold of 0.5. Internal validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.96 (95% confidence interval [CI], 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.96 (95% CI, 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.97, and 0.96, respectively. External validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.94 (95% CI, 0.929-0.951), 0.97 (95% CI, 0.957-0.974), 0.96 (95% CI, 0.952-0.971), 0.95 (95% CI, 0.935-0.956), 0.97, and 0.96, respectively.

CONCLUSIONS

This study demonstrates the accuracy and feasibility of code-free AutoML models for identifying refDR developed using handheld retinal imaging in a community-based screening program. Potentially, the use of AutoML may increase access to machine learning models that may be adapted for specific programs that are guided by the clinical need to rapidly address disparities in health care delivery.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found after the references.

Global trends and performances in diabetic retinopathy studies: A bibliometric analysis

Objective

The objective of this study is to conduct a comprehensive bibliometric analysis to identify and evaluate global trends in diabetic retinopathy (DR) research and visualize the focus and frontiers of this field.

Methods

Diabetic retinopathy-related publications from the establishment of the Web of Science (WOS) through 1 November 2022 were retrieved for qualitative and quantitative analyses. This study analyzed annual publication counts, prolific countries, institutions, journals, and the top 10 most cited literature. The findings were presented through descriptive statistics. VOSviewer 1.6.17 was used to exhibit keywords with high frequency and national cooperation networks, while CiteSpace 5.5.R2 displayed the timeline and burst keywords for each term.

Results

A total of 10,709 references were analyzed, and the number of publications continuously increased over the investigated period. America had the highest h-index and citation frequency, contributing to the most influence. China was the most prolific country, producing 3,168 articles. The University of London had the highest productivity. The top three productive journals were from America, and Investigative Ophthalmology Visual Science had the highest number of publications. The article from Gulshan et al. (2016; co-citation counts, 2,897) served as the representative and symbolic reference. The main research topics in this area were incidence, pathogenesis, treatment, and artificial intelligence (AI). Deep learning, models, biomarkers, and optical coherence tomography angiography (OCTA) of DR were frontier hotspots.

Conclusion

Bibliometric analysis in this study provided valuable insights into global trends in DR research frontiers. Four key study directions and three research frontiers were extracted from the extensive DR-related literature. As the incidence of DR continues to increase, DR prevention and treatment have become a pressing public health concern and a significant area of research interest. In addition, the development of AI technologies and telemedicine has emerged as promising research frontiers for balancing the number of doctors and patients.

Bias and Non-Diversity of Big Data in Artificial Intelligence: Focus on Retinal Diseases

Artificial intelligence (AI) applications in healthcare will have a potentially far-reaching impact on patient care, however issues regarding algorithmic bias and fairness have recently surfaced. There is a recognized lack of diversity in the available ophthalmic datasets, with 45% of the global population having no readily accessible representative images, leading to potential misrepresentations of their unique anatomic features and ocular pathology. AI applications in retinal disease may show less accuracy with underrepresented populations that may further widen the gap of health inequality if left unaddressed. Beyond disease symptomatology, social determinants of health must be integrated into our current paradigms of disease understanding, with the goal of more personalized care. AI has the potential to decrease global healthcare inequality, but it will need to be based on a more diverse, transparent and responsible use of healthcare data.

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Background

Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application.

Objective

This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives.

Methods

A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization.

Results

Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS.

Conclusions

The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

Leveraging electronic health records for data science: common pitfalls and how to avoid them

Analysis of electronic health records (EHRs) is an increasingly common approach for studying real-world patient data. Use of routinely collected data offers several advantages compared with other study designs, including reduced administrative costs, the ability to update analysis as practice patterns evolve, and larger sample sizes. Methodologically, EHR analysis is subject to distinct challenges because data are not collected for research purposes. In this Viewpoint, we elaborate on the importance of in-depth knowledge of clinical workflows and describe six potential pitfalls to be avoided when working with EHR data, drawing on examples from the literature and our experience. We propose solutions for prevention or mitigation of factors associated with each of these six pitfalls-sample selection bias, imprecise variable definitions, limitations to deployment, variable measurement frequency, subjective treatment allocation, and model overfitting. Ultimately, we hope that this Viewpoint will guide researchers to further improve the methodological robustness of EHR analysis.

Adapting skills from genetic counseling to wearables technology research during the COVID-19 pandemic: Poised for the pivot

Genetic counselors have shown themselves to be adaptable in an evolving profession, with expansion into new sub-specialties, various non-clinical settings, and research roles. The COVID-19 pandemic caused a sudden and drastic shift in healthcare priorities. In an effort to contribute meaningfully to the COVID-19 crisis, and to adapt to a remote- and essential-only research environment, our workplace and thus our roles pivoted from genomics research to remote COVID-19 research using wearables technologies. With a deep understanding of genomic data, we were quickly able to apply similar concepts to wearables data including considering privacy implications, managing uncertain findings, and acknowledging the lack of ethnic diversity in many datasets. By sharing our own experience as an example, we hope individuals trained in genetic counseling may see opportunities for adaptation of their skills into expanding roles.