Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices

Determinants of implementing artificial intelligence-based clinical decision support tools in healthcare: a scoping review protocol

INTRODUCTION

Artificial intelligence (AI), the simulation of human intelligence processes by machines, is being increasingly leveraged to facilitate clinical decision-making. AI-based clinical decision support (CDS) tools can improve the quality of care and appropriate use of healthcare resources, and decrease healthcare provider burnout. Understanding the determinants of implementing AI-based CDS tools in healthcare delivery is vital to reap the benefits of these tools. The objective of this scoping review is to map and synthesise determinants (barriers and facilitators) to implementing AI-based CDS tools in healthcare.

METHODS AND ANALYSIS

This scoping review will follow the Joanna Briggs Institute methodology and the Preferred Reporting Items for Systematic reviews and Meta-Analysis extension for Scoping Reviews checklist. The search terms will be tailored to each database, which includes MEDLINE, Embase, CINAHL, APA PsycINFO and the Cochrane Library. Grey literature and references of included studies will also be searched. The search will include studies published from database inception until 10 May 2022. We will not limit searches by study design or language. Studies that either report determinants or describe the implementation of AI-based CDS tools in clinical practice or/and healthcare settings will be included. The identified determinants (barriers and facilitators) will be described by synthesising the themes using the Theoretical Domains Framework. The outcome variables measured will be mapped and the measures of effectiveness will be summarised using descriptive statistics.

ETHICS AND DISSEMINATION

Ethics approval is not required because all data for this study have been previously published. The findings of this review will be published in a peer-reviewed journal and presented at academic conferences. Importantly, the findings of this scoping review will be widely presented to decision-makers, health system administrators, healthcare providers, and patients and family/caregivers as part of an implementation study of an AI-based CDS for the treatment of coronary artery disease.

DeepFundus: A flow-cytometry-like image quality classifier for boosting the whole life cycle of medical artificial intelligence

Medical artificial intelligence (AI) has been moving from the research phase to clinical implementation. However, most AI-based models are mainly built using high-quality images preprocessed in the laboratory, which is not representative of real-world settings. This dataset bias proves a major driver of AI system dysfunction. Inspired by the design of flow cytometry, DeepFundus, a deep-learning-based fundus image classifier, is developed to provide automated and multidimensional image sorting to address this data quality gap. DeepFundus achieves areas under the receiver operating characteristic curves (AUCs) over 0.9 in image classification concerning overall quality, clinical quality factors, and structural quality analysis on both the internal test and national validation datasets. Additionally, DeepFundus can be integrated into both model development and clinical application of AI diagnostics to significantly enhance model performance for detecting multiple retinopathies. DeepFundus can be used to construct a data-driven paradigm for improving the entire life cycle of medical AI practice.

Machine Learning Analysis of Postkeratoplasty Endothelial Cell Images for the Prediction of Future Graft Rejection

Purpose

This study developed machine learning (ML) classifiers of postoperative corneal endothelial cell images to identify postkeratoplasty patients at risk for allograft rejection within 1 to 24 months of treatment.

Methods

Central corneal endothelium specular microscopic images were obtained from 44 patients after Descemet membrane endothelial keratoplasty (DMEK), half of whom had experienced graft rejection. After deep learning segmentation of images from all patients' last and second-to-last imaging, time points prior to rejection were analyzed (175 and 168, respectively), and 432 quantitative features were extracted assessing cellular spatial arrangements and cell intensity values. Random forest (RF) and logistic regression (LR) models were trained on novel-to-this-application features from single time points, delta-radiomics, and traditional morphometrics (endothelial cell density, coefficient of variation, hexagonality) via 10 iterations of threefold cross-validation. Final assessments were evaluated on a held-out test set.

Results

ML classifiers trained on novel-to-this-application features outperformed those trained on traditional morphometrics for predicting future graft rejection. RF and LR models predicted post-DMEK patients' allograft rejection in the held-out test set with >0.80 accuracy. RF models trained on novel features from second-to-last time points and delta-radiomics predicted post-DMEK patients' rejection with >0.70 accuracy. Cell-graph spatial arrangement, intensity, and shape features were most indicative of graft rejection.

Conclusions

ML classifiers successfully predicted future graft rejections 1 to 24 months prior to clinically apparent rejection. This technology could aid clinicians to identify patients at risk for graft rejection and guide treatment plans accordingly.

Translational Relevance

Our software applies ML techniques to clinical images and enhances patient care by detecting preclinical keratoplasty rejection.

Artificial Intelligence Software for Diabetic Eye Screening: Diagnostic Performance and Impact of Stratification

AIM

To evaluate the MONA.health artificial intelligence screening software for detecting referable diabetic retinopathy (DR) and diabetic macular edema (DME), including subgroup analysis.

METHODS

The algorithm's threshold value was fixed at the 90% sensitivity operating point on the receiver operating curve to perform the disease classification. Diagnostic performance was appraised on a private test set and publicly available datasets. Stratification analysis was executed on the private test set considering age, ethnicity, sex, insulin dependency, year of examination, camera type, image quality, and dilatation status.

RESULTS

The software displayed an area under the curve (AUC) of 97.28% for DR and 98.08% for DME on the private test set. The specificity and sensitivity for combined DR and DME predictions were 94.24 and 90.91%, respectively. The AUC ranged from 96.91 to 97.99% on the publicly available datasets for DR. AUC values were above 95% in all subgroups, with lower predictive values found for individuals above the age of 65 (82.51% sensitivity) and Caucasians (84.03% sensitivity).

CONCLUSION

We report good overall performance of the MONA.health screening software for DR and DME. The software performance remains stable with no significant deterioration of the deep learning models in any studied strata.

From 2 dimensions to 3rd dimension: Quantitative prediction of anterior chamber depth from anterior segment photographs via deep-learning

Anterior chamber depth (ACD) is a major risk factor of angle closure disease, and has been used in angle closure screening in various populations. However, ACD is measured from ocular biometer or anterior segment optical coherence tomography (AS-OCT), which are costly and may not be readily available in primary care and community settings. Thus, this proof-of-concept study aims to predict ACD from low-cost anterior segment photographs (ASPs) using deep-learning (DL). We included 2,311 pairs of ASPs and ACD measurements for algorithm development and validation, and 380 pairs for algorithm testing. We captured ASPs with a digital camera mounted on a slit-lamp biomicroscope. Anterior chamber depth was measured with ocular biometer (IOLMaster700 or Lenstar LS9000) in data used for algorithm development and validation, and with AS-OCT (Visante) in data used for testing. The DL algorithm was modified from the ResNet-50 architecture, and assessed using mean absolute error (MAE), coefficient-of-determination (R2), Bland-Altman plot and intraclass correlation coefficients (ICC). In validation, our algorithm predicted ACD with a MAE (standard deviation) of 0.18 (0.14) mm; R2 = 0.63. The MAE of predicted ACD was 0.18 (0.14) mm in eyes with open angles and 0.19 (0.14) mm in eyes with angle closure. The ICC between actual and predicted ACD measurements was 0.81 (95% CI 0.77, 0.84). In testing, our algorithm predicted ACD with a MAE of 0.23 (0.18) mm; R2 = 0.37. Saliency maps highlighted the pupil and its margin as the main structures used in ACD prediction. This study demonstrates the possibility of predicting ACD from ASPs via DL. This algorithm mimics an ocular biometer in making its prediction, and provides a foundation to predict other quantitative measurements that are relevant to angle closure screening.

Combining Optical Coherence Tomography and Optical Coherence Tomography Angiography Longitudinal Data for the Detection of Visual Field Progression in Glaucoma

PURPOSE

To use longitudinal optical coherence tomography (OCT) and OCT angiography (OCTA) data to detect glaucomatous visual field (VF) progression with a supervised machine learning approach.

DESIGN

Prospective cohort study.

METHODS

One hundred ten eyes of patients with suspected glaucoma (33.6%) and patients with glaucoma (66.4%) with a minimum of 5 24-2 VF tests and 3 optic nerve head and macula images over an average follow-up duration of 4.1 years were included. VF progression was defined using a composite measure including either a "likely progression event" on Guided Progression Analysis, a statistically significant negative slope of VF mean deviation or VF index, or a positive pointwise linear regression event. Feature-based gradient boosting classifiers were developed using different subsets of baseline and longitudinal OCT and OCTA summary parameters. The area under the receiver operating characteristic curve (AUROC) was used to compare the classification performance of different models.

RESULTS

VF progression was detected in 28 eyes (25.5%). The model with combined baseline and longitudinal OCT and OCTA parameters at the global and hemifield levels had the best classification accuracy to detect VF progression (AUROC = 0.89). Models including combined OCT and OCTA parameters had higher classification accuracy compared with those with individual subsets of OCT or OCTA features alone. Including hemifield measurements significantly improved the models' classification accuracy compared with using global measurements alone. Including longitudinal rates of change of OCT and OCTA parameters (AUROCs = 0.80-0.89) considerably increased the classification accuracy of the models with baseline measurements alone (AUROCs = 0.60-0.63).

CONCLUSIONS

Longitudinal OCTA measurements complement OCT-derived structural metrics for the evaluation of functional VF loss in patients with glaucoma.

Extracellular vesicles as a liquid biopsy for melanoma: Are we there yet?

Melanoma is the most aggressive form of skin cancer owing to its high propensity to metastasise in distant organs and develop resistance to treatment. The scarce treatment options available for melanoma underscore the need for biomarkers to guide treatment decisions. In this context, an attractive alternative to overcome the limitations of repeated tissue sampling is the analysis of peripheral blood samples, referred to as 'liquid biopsy'. In particular, the analysis of extracellular vesicles (EVs) has emerged as a promising candidate due to their role in orchestrating cancer dissemination, immune modulation, and drug resistance. As we gain insights into the role of EVs in cancer and melanoma their potential for clinical use is becoming apparent. Herein, we critically summarise the current evidence supporting EVs as biomarkers for melanoma diagnosis, prognostication, therapy response prediction, and drug resistance. EVs are proposed as a candidate biomarker for predicting therapeutic response to immune checkpoint inhibition. However, to realise the potential of EV analysis for clinical decision-making strong clinical validation is required, underscoring the need for further research in this area.

Fluorescence Angiography with Dual Fluorescence for the Early Detection and Longitudinal Quantitation of Vascular Leakage in Retinopathy

BACKGROUND

Diabetic retinopathy (DR) afflicts more than 93 million people worldwide and is a leading cause of vision loss in working adults. While DR therapies are available, early DR development may go undetected without treatment due to the lack of sufficiently sensitive tools. Therefore, early detection is critically important to enable efficient treatment before progression to vision-threatening complications. A major clinical manifestation of early DR is retinal vascular leakage that may progress from diffuse to more localized focal leakage, leading to increased retinal thickness and diabetic macular edema (DME). In preclinical research, a hallmark of DR in mouse models is diffuse retinal leakage without increased thickness or DME, which limits the utility of optical coherence tomography and fluorescein angiography (FA) for early detection. The Evans blue assay detects diffuse leakage but requires euthanasia, which precludes longitudinal studies in the same animals.

METHODS

We developed a new modality of ratiometric fluorescence angiography with dual fluorescence (FA-DF) to reliably detect and longitudinally quantify diffuse retinal vascular leakage in mouse models of induced and spontaneous DR.

RESULTS

These studies demonstrated the feasibility and sensitivity of FA-DF in detecting and quantifying retinal vascular leakage in the same mice over time during DR progression in association with chronic hyperglycemia and age.

CONCLUSIONS

These proof-of-concept studies demonstrated the promise of FA-DF as a minimally invasive method to quantify DR leakage in preclinical mouse models longitudinally.

Bias and Non-Diversity of Big Data in Artificial Intelligence: Focus on Retinal Diseases

Artificial intelligence (AI) applications in healthcare will have a potentially far-reaching impact on patient care, however issues regarding algorithmic bias and fairness have recently surfaced. There is a recognized lack of diversity in the available ophthalmic datasets, with 45% of the global population having no readily accessible representative images, leading to potential misrepresentations of their unique anatomic features and ocular pathology. AI applications in retinal disease may show less accuracy with underrepresented populations that may further widen the gap of health inequality if left unaddressed. Beyond disease symptomatology, social determinants of health must be integrated into our current paradigms of disease understanding, with the goal of more personalized care. AI has the potential to decrease global healthcare inequality, but it will need to be based on a more diverse, transparent and responsible use of healthcare data.

Vitreoretinal Surgical Instrument Tracking in Three Dimensions Using Deep Learning

Purpose

To evaluate the potential for artificial intelligence-based video analysis to determine surgical instrument characteristics when moving in the three-dimensional vitreous space.

Methods

We designed and manufactured a model eye in which we recorded choreographed videos of many surgical instruments moving throughout the eye. We labeled each frame of the videos to describe the surgical tool characteristics: tool type, location, depth, and insertional laterality. We trained two different deep learning models to predict each of the tool characteristics and evaluated model performances on a subset of images.

Results

The accuracy of the classification model on the training set is 84% for the x-y region, 97% for depth, 100% for instrument type, and 100% for laterality of insertion. The accuracy of the classification model on the validation dataset is 83% for the x-y region, 96% for depth, 100% for instrument type, and 100% for laterality of insertion. The close-up detection model performs at 67 frames per second, with precision for most instruments higher than 75%, achieving a mean average precision of 79.3%.

Conclusions

We demonstrated that trained models can track surgical instrument movement in three-dimensional space and determine instrument depth, tip location, instrument insertional laterality, and instrument type. Model performance is nearly instantaneous and justifies further investigation into application to real-world surgical videos.

Translational Relevance

Deep learning offers the potential for software-based safety feedback mechanisms during surgery or the ability to extract metrics of surgical technique that can direct research to optimize surgical outcomes.

Application of artificial intelligence to the public health education

With the global outbreak of coronavirus disease 2019 (COVID-19), public health has received unprecedented attention. The cultivation of emergency and compound professionals is the general trend through public health education. However, current public health education is limited to traditional teaching models that struggle to balance theory and practice. Fortunately, the development of artificial intelligence (AI) has entered the stage of intelligent cognition. The introduction of AI in education has opened a new era of computer-assisted education, which brought new possibilities for teaching and learning in public health education. AI-based on big data not only provides abundant resources for public health research and management but also brings convenience for students to obtain public health data and information, which is conducive to the construction of introductory professional courses for students. In this review, we elaborated on the current status and limitations of public health education, summarized the application of AI in public health practice, and further proposed a framework for how to integrate AI into public health education curriculum. With the rapid technological advancements, we believe that AI will revolutionize the education paradigm of public health and help respond to public health emergencies.

12. Retinopathy, Neuropathy, and Foot Care: Standards of Care in Diabetes-2023

The American Diabetes Association (ADA) "Standards of Care in Diabetes" includes the ADA's current clinical practice recommendations and is intended to provide the components of diabetes care, general treatment goals and guidelines, and tools to evaluate quality of care. Members of the ADA Professional Practice Committee, a multidisciplinary expert committee, are responsible for updating the Standards of Care annually, or more frequently as warranted. For a detailed description of ADA standards, statements, and reports, as well as the evidence-grading system for ADA's clinical practice recommendations and a full list of Professional Practice Committee members, please refer to Introduction and Methodology. Readers who wish to comment on the Standards of Care are invited to do so at professional.diabetes.org/SOC.

Deep Capillary Nonperfusion on OCT Angiography Predicts Complications in Eyes with Referable Nonproliferative Diabetic Retinopathy

OBJECTIVE

To evaluate the ability of capillary nonperfusion parameters on OCT angiography (OCTA) to predict the development of clinically significant outcomes in eyes with referable nonproliferative diabetic retinopathy (NPDR).

DESIGN

Prospective longitudinal observational study.

SUBJECTS

In total, 59 patients (74 eyes) with treatment-naive moderate and severe (referable) NPDR.

METHODS

Patients were imaged with OCTA at baseline and then followed-up for 1 year. We evaluated 2 OCTA capillary nonperfusion metrics, vessel density (VD) and geometric perfusion deficits (GPDs), in the superficial capillary plexus, middle capillary plexus (MCP), and deep capillary plexus (DCP). We compared the predictive accuracy of baseline OCTA metrics for clinically significant diabetic retinopathy (DR) outcomes at 1 year.

MAIN OUTCOME MEASURES

Significant clinical outcomes at 1 year, defined as 1 or more of the following-vitreous hemorrhage, center-involving diabetic macular edema, and initiation of treatment with pan-retinal photocoagulation or anti-VEGF injections.

RESULTS

Overall, 49 patients (61 eyes) returned for the 1-year follow-up. Geometric perfusion deficits and VD in the MCP and DCP correlated with clinically significant outcomes at 1 year (P < 0.001). Eyes with these outcomes had lower VD and higher GPD, indicating worse nonperfusion of the deeper retinal layers than those that remained free from complication. These differences remained significant (P = 0.046 to < 0.001) when OCTA parameters were incorporated into models that also considered sex, baseline corrected visual acuity, and baseline DR severity. Adjusted receiver operating characteristic curve for DCP GPD achieved an area under the curve (AUC) of 0.929, with sensitivity of 89% and specificity of 98%. In a separate analysis focusing on high-risk proliferative diabetic retinopathy outcomes, MCP and DCP GPD and VD remained significantly predictive with comparable AUC and sensitivities to the pooled analysis.

CONCLUSIONS

Evidence of deep capillary nonperfusion at baseline in eyes with clinically referable NPDR can predict short-term DR complications with high accuracy, suggesting that deep retinal ischemia has an important pathophysiologic role in DR progression. Our results suggest that OCTA may provide additional prognostic benefit to clinical DR staging in eyes with high risk.

Quantitative approaches in multimodal fundus imaging: State of the art and future perspectives

When it first appeared, multimodal fundus imaging revolutionized the diagnostic workup and provided extremely useful new insights into the pathogenesis of fundus diseases. The recent addition of quantitative approaches has further expanded the amount of information that can be obtained. In spite of the growing interest in advanced quantitative metrics, the scientific community has not reached a stable consensus on repeatable, standardized quantitative techniques to process and analyze the images. Furthermore, imaging artifacts may considerably affect the processing and interpretation of quantitative data, potentially affecting their reliability. The aim of this survey is to provide a comprehensive summary of the main multimodal imaging techniques, covering their limitations as well as their strengths. We also offer a thorough analysis of current quantitative imaging metrics, looking into their technical features, limitations, and interpretation. In addition, we describe the main imaging artifacts and their potential impact on imaging quality and reliability. The prospect of increasing reliance on artificial intelligence-based analyses suggests there is a need to develop more sophisticated quantitative metrics and to improve imaging technologies, incorporating clear, standardized, post-processing procedures. These measures are becoming urgent if these analyses are to cross the threshold from a research context to real-life clinical practice.

Clinical Applications of Artificial Intelligence in Glaucoma

Ophthalmology is one of the major imaging-intensive fields of medicine and thus has potential for extensive applications of artificial intelligence (AI) to advance diagnosis, drug efficacy, and other treatment-related aspects of ocular disease. AI has made impressive progress in ophthalmology within the past few years and two autonomous AI-enabled systems have received US regulatory approvals for autonomously screening for mid-level or advanced diabetic retinopathy and macular edema. While no autonomous AI-enabled system for glaucoma screening has yet received US regulatory approval, numerous assistive AI-enabled software tools are already employed in commercialized instruments for quantifying retinal images and visual fields to augment glaucoma research and clinical practice. In this literature review (non-systematic), we provide an overview of AI applications in glaucoma, and highlight some limitations and considerations for AI integration and adoption into clinical practice.

The value of artificial intelligence techniques in predicting pancreatic ductal adenocarcinoma with EUS images: A meta-analysis and systematic review

Conventional EUS plays an important role in identifying pancreatic cancer. However, the accuracy of EUS is strongly influenced by the operator's experience in performing EUS. Artificial intelligence (AI) is increasingly being used in various clinical diagnoses, especially in terms of image classification. This study aimed to evaluate the diagnostic test accuracy of AI for the prediction of pancreatic cancer using EUS images. We searched the Embase, PubMed, and Cochrane Library databases to identify studies that used endoscopic ultrasound images of pancreatic cancer and AI to predict the diagnostic accuracy of pancreatic cancer. Two reviewers extracted the data independently. The risk of bias of eligible studies was assessed using a Deek funnel plot. The quality of the included studies was measured by the QUDAS-2 tool. Seven studies involving 1110 participants were included: 634 participants with pancreatic cancer and 476 participants with nonpancreatic cancer. The accuracy of the AI for the prediction of pancreatic cancer (area under the curve) was 0.95 (95% confidence interval [CI], 0.93-0.97), with a corresponding pooled sensitivity of 93% (95% CI, 0.90-0.95), specificity of 90% (95% CI, 0.8-0.95), positive likelihood ratio 9.1 (95% CI 4.4-18.6), negative likelihood ratio 0.08 (95% CI 0.06-0.11), and diagnostic odds ratio 114 (95% CI 56-236). The methodological quality in each study was found to be the source of heterogeneity in the meta-regression combined model, which was statistically significant (P = 0.01). There was no evidence of publication bias. The accuracy of AI in diagnosing pancreatic cancer appears to be reliable. Further research and investment in AI could lead to substantial improvements in screening and early diagnosis.

Deep learning algorithms for detection of diabetic macular edema in OCT images: A systematic review and meta-analysis

PURPOSE

Artificial intelligence (AI) can detect diabetic macular edema (DME) from optical coherence tomography (OCT) images. We aimed to evaluate the performance of deep learning neural networks in DME detection.

METHODS

Embase, Pubmed, the Cochrane Library, and IEEE Xplore were searched up to August 14, 2021. We included studies using deep learning algorithms to detect DME from OCT images. Two reviewers extracted the data independently, and the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool was applied to assess the risk of bias. The study is reported according to Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA).

RESULTS

Ninteen studies involving 41005 subjects were included. The pooled sensitivity and specificity were 96.0% (95% confidence interval (CI): 93.9% to 97.3%) and 99.3% (95% CI: 98.2% to 99.7%), respectively. Subgroup analyses found that data set selection, sample size of training set and the choice of OCT devices contributed to the heterogeneity (all P < 0.05). While there was no association between the diagnostic accuracy and transfer learning adoption or image management (all P > 0.05).

CONCLUSIONS

Deep learning methods, particularly the convolutional neural networks (CNNs) could effectively detect clinically significant DME, which can provide referral suggestions to the patients.

Application of Deep Learning to Retinal-Image-Based Oculomics for Evaluation of Systemic Health: A Review

The retina is a window to the human body. Oculomics is the study of the correlations between ophthalmic biomarkers and systemic health or disease states. Deep learning (DL) is currently the cutting-edge machine learning technique for medical image analysis, and in recent years, DL techniques have been applied to analyze retinal images in oculomics studies. In this review, we summarized oculomics studies that used DL models to analyze retinal images-most of the published studies to date involved color fundus photographs, while others focused on optical coherence tomography images. These studies showed that some systemic variables, such as age, sex and cardiovascular disease events, could be consistently robustly predicted, while other variables, such as thyroid function and blood cell count, could not be. DL-based oculomics has demonstrated fascinating, "super-human" predictive capabilities in certain contexts, but it remains to be seen how these models will be incorporated into clinical care and whether management decisions influenced by these models will lead to improved clinical outcomes.

Squeeze-MNet: Precise Skin Cancer Detection Model for Low Computing IoT Devices Using Transfer Learning

Cancer remains a deadly disease. We developed a lightweight, accurate, general-purpose deep learning algorithm for skin cancer classification. Squeeze-MNet combines a Squeeze algorithm for digital hair removal during preprocessing and a MobileNet deep learning model with predefined weights. The Squeeze algorithm extracts important image features from the image, and the black-hat filter operation removes noise. The MobileNet model (with a dense neural network) was developed using the International Skin Imaging Collaboration (ISIC) dataset to fine-tune the model. The proposed model is lightweight; the prototype was tested on a Raspberry Pi 4 Internet of Things device with a Neo pixel 8-bit LED ring; a medical doctor validated the device. The average precision (AP) for benign and malignant diagnoses was 99.76% and 98.02%, respectively. Using our approach, the required dataset size decreased by 66%. The hair removal algorithm increased the accuracy of skin cancer detection to 99.36% with the ISIC dataset. The area under the receiver operating curve was 98.9%.

Classification of diabetic retinopathy: Past, present and future

Diabetic retinopathy (DR) is a leading cause of visual impairment and blindness worldwide. Since DR was first recognized as an important complication of diabetes, there have been many attempts to accurately classify the severity and stages of disease. These historical classification systems evolved as understanding of disease pathophysiology improved, methods of imaging and assessing DR changed, and effective treatments were developed. Current DR classification systems are effective, and have been the basis of major research trials and clinical management guidelines for decades. However, with further new developments such as recognition of diabetic retinal neurodegeneration, new imaging platforms such as optical coherence tomography and ultra wide-field retinal imaging, artificial intelligence and new treatments, our current classification systems have significant limitations that need to be addressed. In this paper, we provide a historical review of different classification systems for DR, and discuss the limitations of our current classification systems in the context of new developments. We also review the implications of new developments in the field, to see how they might feature in a future, updated classification.

Implementation of Machine Learning Pipelines for Clinical Practice: Development and Validation Study

BACKGROUND

Artificial intelligence (AI) technologies, such as machine learning and natural language processing, have the potential to provide new insights into complex health data. Although powerful, these algorithms rarely move from experimental studies to direct clinical care implementation.

OBJECTIVE

We aimed to describe the key components for successful development and integration of two AI technology-based research pipelines for clinical practice.

METHODS

We summarized the approach, results, and key learnings from the implementation of the following two systems implemented at a large, tertiary care children's hospital: (1) epilepsy surgical candidate identification (or epilepsy ID) in an ambulatory neurology clinic; and (2) an automated clinical trial eligibility screener (ACTES) for the real-time identification of patients for research studies in a pediatric emergency department.

RESULTS

The epilepsy ID system performed as well as board-certified neurologists in identifying surgical candidates (with a sensitivity of 71% and positive predictive value of 77%). The ACTES system decreased coordinator screening time by 12.9%. The success of each project was largely dependent upon the collaboration between machine learning experts, research and operational information technology professionals, longitudinal support from clinical providers, and institutional leadership.

CONCLUSIONS

These projects showcase novel interactions between machine learning recommendations and providers during clinical care. Our deployment provides seamless, real-time integration of AI technology to provide decision support and improve patient care.

Evidence synthesis, digital scribes, and translational challenges for artificial intelligence in healthcare

Healthcare has well-known challenges with safety, quality, and effectiveness, and many see artificial intelligence (AI) as essential to any solution. Emerging applications include the automated synthesis of best-practice research evidence including systematic reviews, which would ultimately see all clinical trial data published in a computational form for immediate synthesis. Digital scribes embed themselves in the process of care to detect, record, and summarize events and conversations for the electronic record. However, three persistent translational challenges must be addressed before AI is widely deployed. First, little effort is spent replicating AI trials, exposing patients to risks of methodological error and biases. Next, there is little reporting of patient harms from trials. Finally, AI built using machine learning may perform less effectively in different clinical settings.

Digital innovations for retinal care in diabetic retinopathy

AIM

The purpose of this review is to examine the applications of novel digital technology domains for the screening and management of patients with diabetic retinopathy (DR).

METHODS

A PubMed engine search was performed, using the terms "Telemedicine", "Digital health", "Telehealth", "Telescreening", "Artificial intelligence", "Deep learning", "Smartphone", "Triage", "Screening", "Home-based", "Monitoring", "Ophthalmology", "Diabetes", "Diabetic Retinopathy", "Retinal imaging". Full-text English language studies from January 1, 2010, to February 1, 2022, and reference lists were considered for the conceptual framework of this review.

RESULTS

Diabetes mellitus and its eye complications, including DR, are particularly well suited to digital technologies, providing an ideal model for telehealth initiatives and real-world applications. The current development in the adoption of telemedicine, artificial intelligence and remote monitoring as an alternative to or in addition to traditional forms of care will be discussed.

CONCLUSIONS

Advances in digital health have created an ecosystem ripe for telemedicine in the field of DR to thrive. Stakeholders and policymakers should adopt a participatory approach to ensure sustained implementation of these technologies after the COVID-19 pandemic. This article belongs to the Topical Collection "Diabetic Eye Disease", managed by Giuseppe Querques.

Diabetic Retinopathy Telemedicine Outcomes With Artificial Intelligence-Based Image Analysis, Reflex Dilation, and Image Overread

PURPOSE

To examine real-world telemedicine outcomes of diabetic retinopathy (DR) screening with artificial intelligence (AI)-based image analysis, reflex dilation, and secondary image overread in a primary care setting.

DESIGN

Validity and reliability analysis.

METHODS

Single institution review of 1052 consecutive adult patients who received diabetic retinopathy photoscreening in the primary care setting over an 18-month period. Nonmydriatic fundus photographs were acquired and analyzed by the IDx-DR AI-based system. When nonmydriatic images were ungradable, reflex dilation (1% tropicamide) and mydriatic photography were performed for repeat AI-based analysis. Manual overread was performed on all images. Patient demographics, clinical characteristics, and screening outcomes were recorded.

RESULTS

A total of 965 of 1052 patients (91.7%) had AI-gradable fundus photographs: 580 had gradable nonmydriatic imaging (55.1%) and 440 of 472 patients with ungradable nonmydriatic photographs had reflex dilation (93.2%). One hundred thirty-eight of 965 patients (14.3%) were AI-graded as "positive" (greater than mild NPDR) and 827 of 965 were "negative" (85.7%), with 100% sensitivity (95% CI 90.8-100%), 89.2% specificity (95% CI 87.0-91.1%), 27.5% positive predictive value (95% CI 24.0-31.4%), and 100% negative predictive value (95% CI 99.6-100%) compared with manual overread assessment of greater than mild NPDR requiring further evaluation with a comprehensive dilated examination. Image gradeability was inversely related to patient age: 93.5% gradable (61.9% nonmydriatic) for patients aged <70 years vs 85.3% (31.0% nonmydriatic) for patients aged 70+ years (P < .001).

CONCLUSION

Incorporation of AI-based image analysis into real-world primary care diabetic retinopathy screening yielded no false negative results and offered excellent image gradeability within a protocol combining nonmydriatic fundus photography and pharmacologic dilation, as needed. Image gradeability was lower with increasing patient age.

Automated artificial intelligence-based system for clinical follow-up of patients with age-related macular degeneration

PURPOSE

In this study, we investigate the potential of a novel artificial intelligence-based system for autonomous follow-up of patients treated for neovascular age-related macular degeneration (AMD).

METHODS

A temporal deep learning model was trained on a data set of 84 489 optical coherence tomography scans from AMD patients to recognize disease activity, and its performance was compared with a published non-temporal model trained on the same data (Acta Ophthalmol, 2021). An autonomous follow-up system was created by augmenting the AI model with deterministic logic to suggest treatment according to the observe-and-plan regimen. To validate the AI-based system, a data set comprising clinical decisions and imaging data from 200 follow-up consultations was collected prospectively. In each case, both the autonomous AI decision and original clinical decision were compared with an expert panel consensus.

RESULTS

The temporal AI model proved superior at detecting disease activity compared with the model without temporal input (area under the curve 0.900 (95% CI 0.894-0.906) and 0.857 (95% CI 0.846-0.867) respectively). The AI-based follow-up system could make an autonomous decision in 73% of the cases, 91.8% of which were in agreement with expert consensus. This was on par with the 87.7% agreement rate between decisions made in the clinic and expert consensus (p = 0.33).

CONCLUSIONS

The proposed autonomous follow-up system was shown to be safe and compliant with expert consensus on par with clinical practice. The system could in the future ease the pressure on public ophthalmology services from an increasing number of AMD patients.

Predicting the future development of diabetic retinopathy using a deep learning algorithm for the analysis of non-invasive retinal imaging

Aims

Diabetic retinopathy (DR) is the most common cause of vision loss in the working age. This research aimed to develop an artificial intelligence (AI) machine learning model which can predict the development of referable DR from fundus imagery of otherwise healthy eyes.

Methods

Our researchers trained a machine learning algorithm on the EyePACS data set, consisting of 156 363 fundus images. Referrable DR was defined as any level above mild on the International Clinical Diabetic Retinopathy scale.

Results

The algorithm achieved 0.81 area under receiver operating curve (AUC) when averaging scores from multiple images on the task of predicting development of referrable DR, and 0.76 AUC when using a single image.

Conclusion

Our results suggest that risk of DR may be predicted from fundus photography alone. Prediction of personalised risk of DR may become key in treatment and contribute to patient compliance across the board, particularly when supported by further prospective research.

Shifting machine learning for healthcare from development to deployment and from models to data

In the past decade, the application of machine learning (ML) to healthcare has helped drive the automation of physician tasks as well as enhancements in clinical capabilities and access to care. This progress has emphasized that, from model development to model deployment, data play central roles. In this Review, we provide a data-centric view of the innovations and challenges that are defining ML for healthcare. We discuss deep generative models and federated learning as strategies to augment datasets for improved model performance, as well as the use of the more recent transformer models for handling larger datasets and enhancing the modelling of clinical text. We also discuss data-focused problems in the deployment of ML, emphasizing the need to efficiently deliver data to ML models for timely clinical predictions and to account for natural data shifts that can deteriorate model performance.

Medicine 2032: The future of cardiovascular disease prevention with machine learning and digital health technology

Machine learning (ML) refers to computational algorithms that iteratively improve their ability to recognize patterns in data. The digitization of our healthcare infrastructure is generating an abundance of data from electronic health records, imaging, wearables, and sensors that can be analyzed by ML algorithms to generate personalized risk assessments and promote guideline-directed medical management. ML's strength in generating insights from complex medical data to guide clinical decisions must be balanced with the potential to adversely affect patient privacy, safety, health equity, and clinical interpretability. This review provides a primer on key advances in ML for cardiovascular disease prevention and how they may impact clinical practice.

Deep Capillary Geometric Perfusion Deficits on OCT Angiography Detect Clinically Referable Eyes with Diabetic Retinopathy

PURPOSE

To evaluate the sensitivity (SN) and specificity (SP) of OCT angiography (OCTA) parameters for detecting clinically referable eyes with diabetic retinopathy (DR) in a cohort of patients with diabetes mellitus (DM).

DESIGN

Retrospective, cross-sectional study.

SUBJECTS

Patients with DM with various levels of DR.

METHODS

We measured vessel density, vessel length density (VLD), and geometric perfusion deficits (GPDs) in the full retina, superficial capillary plexus (SCP), and deep capillary plexus (DCP) on 3 × 3-mm OCTA images. Geometric perfusion deficit was recently described as retinal tissue located further than 30 μm from blood vessels, excluding the foveal avascular zone (FAZ). We modified the GPD metric by including the FAZ as an additional variable. Clinically referable eyes were defined as moderate nonproliferative DR (NPDR) or worse retinopathy, or diabetic macular edema (DME). One eye from each patient was selected for the analysis based on image quality. We used a binary logistic regression model to adjust for covariates.

MAIN OUTCOME MEASURES

Sensitivity, SP, and area under the curve (AUC).

RESULTS

Seventy-one of 150 included eyes from 150 patients (52 with DM without DR, 27 with mild NPDR, 16 with moderate NPDR, 10 with severe NPDR, 30 with proliferative DR, and 15 with DME) had clinically referable DR. Geometric perfusion deficit metric that included the FAZ performed better than GPD in detecting referable DR in the SCP (P = 0.025) but not the DCP or full retina (P > 0.05 for both). Deep capillary plexus GPD had the largest AUC for detecting clinically referable eyes (AUC = 0.965, SN = 97.2%, SP = 84.8%), which was significantly larger than the AUC for vessel density of any layer (P < 0.05 for all) but not DCP VLD (P = 0.166). The cutoff value of 2.5% for DCP GPD resulted in a highly sensitive test for detecting clinically referable eyes without adjusting for covariates (AUC = 0.955, SN = 97.2%, SP = 79.7%).

CONCLUSIONS

Vascular parameters in OCTA, especially in the DCP, have the potential to identify eyes that warrant further evaluation. Geometric perfusion deficits may better distinguish these clinically referable eyes with DR than standard vessel density parameters.

Social Determinants of Health and Impact on Screening, Prevalence, and Management of Diabetic Retinopathy in Adults: A Narrative Review

Diabetic retinal disease (DRD) is the leading cause of blindness among working-aged individuals with diabetes. In the United States, underserved and minority populations are disproportionately affected by diabetic retinopathy and other diabetes-related health outcomes. In this narrative review, we describe racial disparities in the prevalence and screening of diabetic retinopathy, as well as the wide-range of disparities associated with social determinants of health (SDOH), which include socioeconomic status, geography, health-care access, and education.

Mitigating the impact of biased artificial intelligence in emergency decision-making

BACKGROUND

Prior research has shown that artificial intelligence (AI) systems often encode biases against minority subgroups. However, little work has focused on ways to mitigate the harm discriminatory algorithms can cause in high-stakes settings such as medicine.

METHODS

In this study, we experimentally evaluated the impact biased AI recommendations have on emergency decisions, where participants respond to mental health crises by calling for either medical or police assistance. We recruited 438 clinicians and 516 non-experts to participate in our web-based experiment. We evaluated participant decision-making with and without advice from biased and unbiased AI systems. We also varied the style of the AI advice, framing it either as prescriptive recommendations or descriptive flags.

RESULTS

Participant decisions are unbiased without AI advice. However, both clinicians and non-experts are influenced by prescriptive recommendations from a biased algorithm, choosing police help more often in emergencies involving African-American or Muslim men. Crucially, using descriptive flags rather than prescriptive recommendations allows respondents to retain their original, unbiased decision-making.

CONCLUSIONS

Our work demonstrates the practical danger of using biased models in health contexts, and suggests that appropriately framing decision support can mitigate the effects of AI bias. These findings must be carefully considered in the many real-world clinical scenarios where inaccurate or biased models may be used to inform important decisions.

A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study

BACKGROUND

Clinical application of artificial intelligence is limited due to the lack of interpretability and expandability in complex clinical settings. We aimed to develop an eye diseases screening system with improved interpretability and expandability based on a lesion-level dissection and tested the clinical expandability and auxiliary ability of the system.

METHODS

The four-hierarchical interpretable eye diseases screening system (IEDSS) based on a novel structural pattern named lesion atlas was developed to identify 30 eye diseases and conditions using a total of 32,026 ultra-wide field images collected from the Second Affiliated Hospital of Zhejiang University, School of Medicine (SAHZU), the First Affiliated Hospital of University of Science and Technology of China (FAHUSTC), and the Affiliated People's Hospital of Ningbo University (APHNU) in China between November 1, 2016 to February 28, 2022. The performance of IEDSS was compared with ophthalmologists and classic models trained with image-level labels. We further evaluated IEDSS in two external datasets, and tested it in a real-world scenario and an extended dataset with new phenotypes beyond the training categories. The accuracy (ACC), F1 score and confusion matrix were calculated to assess the performance of IEDSS.

FINDINGS

IEDSS reached average ACCs (aACC) of 0·9781 (95%CI 0·9739-0·9824), 0·9660 (95%CI 0·9591-0·9730) and 0·9709 (95%CI 0·9655-0·9763), frequency-weighted average F1 scores of 0·9042 (95%CI 0·8957-0·9127), 0·8837 (95%CI 0·8714-0·8960) and 0·8874 (95%CI 0·8772-0·8972) in datasets of SAHZU, APHNU and FAHUSTC, respectively. IEDSS reached a higher aACC (0·9781, 95%CI 0·9739-0·9824) compared with a multi-class image-level model (0·9398, 95%CI 0·9329-0·9467), a classic multi-label image-level model (0·9278, 95%CI 0·9189-0·9366), a novel multi-label image-level model (0·9241, 95%CI 0·9151-0·9331) and a lesion-level model without Adaboost (0·9381, 95%CI 0·9299-0·9463). In the real-world scenario, the aACC of IEDSS (0·9872, 95%CI 0·9828-0·9915) was higher than that of the senior ophthalmologist (SO) (0·9413, 95%CI 0·9321-0·9504, p = 0·000) and the junior ophthalmologist (JO) (0·8846, 95%CI 0·8722-0·8971, p = 0·000). IEDSS remained strong performance (ACC = 0·8560, 95%CI 0·8252-0·8868) compared with JO (ACC = 0·784, 95%CI 0·7479-0·8201, p= 0·003) and SO (ACC = 0·8500, 95%CI 0·8187-0·8813, p = 0·789) in the extended dataset.

INTERPRETATION

IEDSS showed excellent and stable performance in identifying common eye conditions and conditions beyond the training categories. The transparency and expandability of IEDSS could tremendously increase the clinical application range and the practical clinical value of it. It would enhance the efficiency and reliability of clinical practice, especially in remote areas with a lack of experienced specialists.

FUNDING

National Natural Science Foundation Regional Innovation and Development Joint Fund (U20A20386), Key research and development program of Zhejiang Province (2019C03020), Clinical Medical Research Centre for Eye Diseases of Zhejiang Province (2021E50007).

Artificial intelligence and deep learning in ophthalmology: Current status and future perspectives

Background

The ophthalmology field was among the first to adopt artificial intelligence (AI) in medicine. The availability of digitized ocular images and substantial data have made deep learning (DL) a popular topic.

Main text

At the moment, AI in ophthalmology is mostly used to improve disease diagnosis and assist decision-making aiming at ophthalmic diseases like diabetic retinopathy (DR), glaucoma, age-related macular degeneration (AMD), cataract and other anterior segment diseases. However, most of the AI systems developed to date are still in the experimental stages, with only a few having achieved clinical applications. There are a number of reasons for this phenomenon, including security, privacy, poor pervasiveness, trust and explainability concerns.

Conclusions

This review summarizes AI applications in ophthalmology, highlighting significant clinical considerations for adopting AI techniques and discussing the potential challenges and future directions.

Differences in receipt of recommended eye examinations by comorbidity status and healthcare utilization among nonelderly adults with diabetes

BACKGROUND

To evaluate the effect of diabetes comorbidities by baseline healthcare utilization on receipt of recommended eye examinations.

METHODS

Retrospective analysis of 310 691 nonelderly adults with type 2 diabetes in the IBM MarketScan Commercial Database from 2016 to 2019. Patients were grouped based on diabetes-concordant (related) or -discordant (unrelated) comorbidities. Logistic regression was used to estimate the prevalence ratio (PR) for eye examinations by comorbidity status, healthcare utilization, and an interaction between comorbidities and utilization, controlling for age, sex, region, and major eye disease.

RESULTS

Prevalence of biennial eye examinations varied by the four comorbidity groups: 43.5% (diabetes only), 52.7% (concordant + discordant comorbidities), 48.0% (concordant comorbidities only), and 45.3% (discordant comorbidities only). In the lowest healthcare utilization tertile, the concordant-only and concordant + discordant groups had lower prevalence of examinations compared to diabetes only (PR 0.95 [95% CI 0.92-0.98] and PR 0.91 [95% CI 0.88-0.95], respectively). In the medium utilization tertile, the discordant-only and concordant + discordant groups had lower prevalence of examinations (PR 0.89 [0.83-0.95] and PR 0.94 [0.90-0.98], respectively). In the highest utilization tertile, the concordant-only and concordant + discordant groups had higher prevalence of examinations.

CONCLUSIONS

Among patients with low healthcare utilization, having comorbid conditions is associated with lower prevalence of eye examinations. Among those with medium healthcare utilization, patients with diabetes-discordant comorbidities are particularly vulnerable. This study highlights populations of diabetes patients who would benefit from increased assistance in receiving vision-preserving eye examinations.

An overview of artificial intelligence in diabetic retinopathy and other ocular diseases

Artificial intelligence (AI), also known as machine intelligence, is a branch of science that empowers machines using human intelligence. AI refers to the technology of rendering human intelligence through computer programs. From healthcare to the precise prevention, diagnosis, and management of diseases, AI is progressing rapidly in various interdisciplinary fields, including ophthalmology. Ophthalmology is at the forefront of AI in medicine because the diagnosis of ocular diseases heavy reliance on imaging. Recently, deep learning-based AI screening and prediction models have been applied to the most common visual impairment and blindness diseases, including glaucoma, cataract, age-related macular degeneration (ARMD), and diabetic retinopathy (DR). The success of AI in medicine is primarily attributed to the development of deep learning algorithms, which are computational models composed of multiple layers of simulated neurons. These models can learn the representations of data at multiple levels of abstraction. The Inception-v3 algorithm and transfer learning concept have been applied in DR and ARMD to reuse fundus image features learned from natural images (non-medical images) to train an AI system with a fraction of the commonly used training data (<1%). The trained AI system achieved performance comparable to that of human experts in classifying ARMD and diabetic macular edema on optical coherence tomography images. In this study, we highlight the fundamental concepts of AI and its application in these four major ocular diseases and further discuss the current challenges, as well as the prospects in ophthalmology.

Diabetic Retinopathy and Eye Screening: Diabetic Patients Standpoint, Their Practice, and Barriers; A Cross-Sectional Study

Diabetes mellites (DM) is one of the most common systemic disorders in Saudi Arabia and worldwide. Diabetic retinopathy (DR) is a potentially blinding ophthalmic consequence of uncontrolled DM. The early detection of DR leads to an earlier intervention, which might be sight-saving. Our aim in this cross-sectional study is to assess patients' knowledge and practices regarding DR, and to detect the barriers for eye screening and receiving a check-up from an ophthalmologist. The study included 386 diabetic patients. One hundred and thirty-one patients (33.9%) had T1DM and 188 (48.7%) had T2DM. Most of the diabetic patients (73.3%) know that they must have an eye check-up regardless of their blood sugar level. DM was agreed to affect the retina in 80.3% of the patients, 56% of patients agree that DM complications are always symptomatic, and 84.5% know that DM could affect their eyes. The fact that blindness is a complication of diabetic retinopathy was known by 65% of the diabetic patients. A better knowledge was detected among patients older than 50 years of age (54.9%) compared to those aged less than 35 years (40.9%), which was statistically significant (p = 0.030). Additionally, 61.2% of diabetic patients who were university graduates had a significantly better knowledge in comparison to 33.3% of illiterate patients (p = 0.006). Considering the barriers to not getting one's eyes screened earlier, a lack of knowledge was reported by 38.3% of the patients, followed by lack of access to eye care (24.4%). In conclusion, there is a remarkable increase in the awareness of DR among the Saudi population. This awareness might lead to an earlier detection and management of DR.

Influence of Different Types of Retinal Cameras on the Performance of Deep Learning Algorithms in Diabetic Retinopathy Screening

Background: The aim of this study was to assess the performance of regional graders and artificial intelligence algorithms across retinal cameras with different specifications in classifying an image as gradable and ungradable. Methods: Study subjects were included from a community-based nationwide diabetic retinopathy screening program in Thailand. Various non-mydriatic fundus cameras were used for image acquisition, including Kowa Nonmyd, Kowa Nonmyd α-DⅢ, Kowa Nonmyd 7, Kowa Nonmyd WX, Kowa VX 10 α, Kowa VX 20 and Nidek AFC 210. All retinal photographs were graded by deep learning algorithms and human graders and compared with a standard reference. Results: Images were divided into two categories as gradable and ungradable images. Four thousand eight hundred fifty-two participants with 19,408 fundus images were included, of which 15,351 (79.09%) were gradable images and the remaining 4057 (20.90%) were ungradable images. Conclusions: The deep learning (DL) algorithm demonstrated better sensitivity, specificity and kappa than the human graders for all eight types of non-mydriatic fundus cameras. The deep learning system showed, more consistent diagnostic performance than the human graders across images of varying quality and camera types.

The upcoming role of Artificial Intelligence (AI) for retinal and glaucomatous diseases

In recent years, the role of artificial intelligence (AI) and deep learning (DL) models is attracting increasing global interest in the field of ophthalmology. DL models are considered the current state-of-art among the AI technologies. In fact, DL systems have the capability to recognize, quantify and describe pathological clinical features. Their role is currently being investigated for the early diagnosis and management of several retinal diseases and glaucoma. The application of DL models to fundus photographs, visual fields and optical coherence tomography (OCT) imaging has provided promising results in the early detection of diabetic retinopathy (DR), wet age-related macular degeneration (w-AMD), retinopathy of prematurity (ROP) and glaucoma. In this review we analyze the current evidence of AI applied to these ocular diseases, as well as discuss the possible future developments and potential clinical implications, without neglecting the present limitations and challenges in order to adopt AI and DL models as powerful tools in the everyday routine clinical practice.

Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan

The development of computer-aided detection (CAD) using artificial intelligence (AI) and machine learning (ML) is rapidly evolving. Submission of AI/ML-based CAD devices for regulatory approval requires information about clinical trial design and performance criteria, but the requirements vary between countries. This study compares the requirements for AI/ML-based CAD devices approved by the US Food and Drug Administration (FDA) and the Pharmaceuticals and Medical Devices Agency (PMDA) in Japan. A list of 45 FDA-approved and 12 PMDA-approved AI/ML-based CAD devices was compiled. In the USA, devices classified as computer-aided simple triage were approved based on standalone software testing, whereas devices classified as computer-aided detection/diagnosis were approved based on reader study testing. In Japan, however, there was no clear distinction between evaluation methods according to the category. In the USA, a prospective randomized controlled trial was conducted for AI/ML-based CAD devices used for the detection of colorectal polyps, whereas in Japan, such devices were approved based on standalone software testing. This study indicated that the different viewpoints of AI/ML-based CAD in the two countries influenced the selection of different evaluation methods. This study’s findings may be useful for defining a unified global development and approval standard for AI/ML-based CAD.

Validation of an autonomous artificial intelligence-based diagnostic system for holistic maculopathy screening in a routine occupational health checkup context

PURPOSE

This study aims to evaluate the ability of an autonomous artificial intelligence (AI) system for detection of the most common central retinal pathologies in fundus photography.

METHODS

Retrospective diagnostic test evaluation on a raw dataset of 5918 images (2839 individuals) evaluated with non-mydriatic cameras during routine occupational health checkups. Three camera models were employed: Optomed Aurora (field of view - FOV 50º, 88% of the dataset), ZEISS VISUSCOUT 100 (FOV 40º, 9%), and Optomed SmartScope M5 (FOV 40º, 3%). Image acquisition took 2 min per patient. Ground truth for each image of the dataset was determined by 2 masked retina specialists, and disagreements were resolved by a 3rd retina specialist. The specific pathologies considered for evaluation were "diabetic retinopathy" (DR), "Age-related macular degeneration" (AMD), "glaucomatous optic neuropathy" (GON), and "Nevus." Images with maculopathy signs that did not match the described taxonomy were classified as "Other."

RESULTS

The combination of algorithms to detect any abnormalities had an area under the curve (AUC) of 0.963 with a sensitivity of 92.9% and a specificity of 86.8%. The algorithms individually obtained are as follows: AMD AUC 0.980 (sensitivity 93.8%; specificity 95.7%), DR AUC 0.950 (sensitivity 81.1%; specificity 94.8%), GON AUC 0.889 (sensitivity 53.6% specificity 95.7%), Nevus AUC 0.931 (sensitivity 86.7%; specificity 90.7%).

CONCLUSION

Our holistic AI approach reaches high diagnostic accuracy at simultaneous detection of DR, AMD, and Nevus. The integration of pathology-specific algorithms permits higher sensitivities with minimal impact on its specificity. It also reduces the risk of missing incidental findings. Deep learning may facilitate wider screenings of eye diseases.

Performance of the AIDRScreening system in detecting diabetic retinopathy in the fundus photographs of Chinese patients: a prospective, multicenter, clinical study

Background

Diabetic retinopathy (DR) is the leading cause of blindness in the working-age population worldwide, and there is a large unmet need for DR screening in China. This observational, prospective, multicenter, gold standard-controlled study sought to evaluate the effectiveness and safety of the AIDRScreening system (v. 1.0), which is an artificial intelligence (AI)-enabled system that detects DR in the Chinese population based on fundus photographs.

Methods

Participants with diabetes mellitus (DM) were recruited. Fundus photographs (field 1 and field 2) of 1 eye in each participant were graded by the AIDRScreening system (v. 1.0) to detect referable DR (RDR). The results were compared to those of the masked manual grading (gold standard) system by the Zhongshan Image Reading Center. The primary outcomes were the sensitivity and specificity of the AIDRScreening system in detecting RDR. The other outcomes evaluated included the system's diagnostic accuracy, positive predictive value, negative predictive value, diagnostic accuracy gain rate, and average diagnostic time gain rate.

Results

Among the 1,001 enrolled participants with DM, 962 (96.1%) were included in the final analyses. The participants had a median age of 60.61 years (range: 20.18-85.78 years), and 48.2% were men. The manual grading system detected RDR in 399 (41.48%) participants. The AIDRScreening system had a sensitivity of 86.72% (95% CI: 83.39-90.05%) and a specificity of 96.09% (95% CI: 94.14-97.54%) in the detection of RDR, and a false-positive rate of 3.91%. The diagnostic accuracy gain rate of the AIDRScreening system was 16.57% higher than that of the investigator, while the average diagnostic time gain rate was -37.32% lower.

Conclusions

The automated AIDRScreening system can detect RDR with high accuracy, but cannot detect maculopathy. The implementation of the AIDRScreening system may increase the efficiency of DR screening.

Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage

BACKGROUND

Intracranial hemorrhage (ICH) requires emergent medical treatment for positive outcomes. While previous artificial intelligence (AI) solutions achieved rapid diagnostics, none were shown to improve the performance of radiologists in detecting ICHs. Here, we show that the Caire ICH artificial intelligence system enhances a radiologist's ICH diagnosis performance.

METHODS

A dataset of non-contrast-enhanced axial cranial computed tomography (CT) scans (n=532) were labeled for the presence or absence of an ICH. If an ICH was detected, its ICH subtype was identified. After a washout period, the three radiologists reviewed the same dataset with the assistance of the Caire ICH system. Performance was measured with respect to reader agreement, accuracy, sensitivity, and specificity when compared to the ground truth, defined as reader consensus.

RESULTS

Caire ICH improved the inter-reader agreement on average by 5.76% in a dataset with an ICH prevalence of 74.3%. Further, radiologists using Caire ICH detected an average of 18 more ICHs and significantly increased their accuracy by 6.15%, their sensitivity by 4.6%, and their specificity by 10.62%. The Caire ICH system also improved the radiologist's ability to accurately identify the ICH subtypes present.

CONCLUSION

The Caire ICH device significantly improves the performance of a cohort of radiologists. Such a device has the potential to be a tool that can improve patient outcomes and reduce misdiagnosis of ICH.

The requirements for performing artificial-intelligence-related research and model development

Artificial intelligence research in health care has undergone tremendous growth in the last several years thanks to the explosion of digital health care data and systems that can leverage large amounts of data to learn patterns that can be applied to clinical tasks. In addition, given broad acceleration in machine learning across industries like transportation, media and commerce, there has been a significant growth in demand for machine-learning practitioners such as engineers and data scientists, who have skill sets that can be applied to health care use cases but who simultaneously lack important health care domain expertise. The purpose of this paper is to discuss the requirements of building an artificial-intelligence research enterprise including the research team, technical software/hardware, and procurement and curation of health care data.

Medical Staff and Resident Preferences for Using Deep Learning in Eye Disease Screening: Discrete Choice Experiment

Background

Deep learning–assisted eye disease diagnosis technology is increasingly applied in eye disease screening. However, no research has suggested the prerequisites for health care service providers and residents willing to use it.

Objective

The aim of this paper is to reveal the preferences of health care service providers and residents for using artificial intelligence (AI) in community-based eye disease screening, particularly their preference for accuracy.

Methods

Discrete choice experiments for health care providers and residents were conducted in Shanghai, China. In total, 34 medical institutions with adequate AI-assisted screening experience participated. A total of 39 medical staff and 318 residents were asked to answer the questionnaire and make a trade-off among alternative screening strategies with different attributes, including missed diagnosis rate, overdiagnosis rate, screening result feedback efficiency, level of ophthalmologist involvement, organizational form, cost, and screening result feedback form. Conditional logit models with the stepwise selection method were used to estimate the preferences.

Results

Medical staff preferred high accuracy: The specificity of deep learning models should be more than 90% (odds ratio [OR]=0.61 for 10% overdiagnosis; P<.001), which was much higher than the Food and Drug Administration standards. However, accuracy was not the residents’ preference. Rather, they preferred to have the doctors involved in the screening process. In addition, when compared with a fully manual diagnosis, AI technology was more favored by the medical staff (OR=2.08 for semiautomated AI model and OR=2.39 for fully automated AI model; P<.001), while the residents were in disfavor of the AI technology without doctors’ supervision (OR=0.24; P<.001).

Conclusions

Deep learning model under doctors’ supervision is strongly recommended, and the specificity of the model should be more than 90%. In addition, digital transformation should help medical staff move away from heavy and repetitive work and spend more time on communicating with residents.