Automated machine learning model for fundus image classification by health-care professionals with no coding experience

To assess the feasibility of code-free deep learning (CFDL) platforms in the prediction of binary outcomes from fundus images in ophthalmology, evaluating two distinct online-based platforms (Google Vertex and Amazon Rekognition), and two distinct datasets. Two publicly available datasets, Messidor-2 and BRSET, were utilized for model development. The Messidor-2 consists of fundus photographs from diabetic patients and the BRSET is a multi-label dataset. The CFDL platforms were used to create deep learning models, with no preprocessing of the images, by a single ophthalmologist without coding expertise. The performance metrics employed to evaluate the models were F1 score, area under curve (AUC), precision and recall. The performance metrics for referable diabetic retinopathy and macular edema were above 0.9 for both tasks and CDFL. The Google Vertex models demonstrated superior performance compared to the Amazon models, with the BRSET dataset achieving the highest accuracy (AUC of 0.994). Multi-classification tasks using only BRSET achieved similar overall performance between platforms, achieving AUC of 0.994 for laterality, 0.942 for age grouping, 0.779 for genetic sex identification, 0.857 for optic, and 0.837 for normality with Google Vertex. The study demonstrates the feasibility of using automated machine learning platforms for predicting binary outcomes from fundus images in ophthalmology. It highlights the high accuracy achieved by the models in some tasks and the potential of CFDL as an entry-friendly platform for ophthalmologists to familiarize themselves with machine learning concepts.

Retrospective case series of high-density silicone oil (Oxane HD) in severe proliferative vitreorretinal retinal detachment patients

BACKGROUND

Describe complications and clinical outcomes of heavy silicone oil (HSO) Oxane HD® use as an alternative to overcome the challenges of performing vitrectomy to treat tractional and rhegmatogenous retinal detachments with proliferative vitreoretinopathy (PVR).

METHODS

A retrospective, observational study was performed on patients from one center from August 2014 to Aug 2023. It was included patients who underwent surgery using HSO Oxane HD® to treat rhegmatogenous retinal detachment with PVR or mixed tractional and rhegmatogenous diabetic retinal detachment. Severely ill patients who could not attend to follow up were excluded. The primary outcome was successful retinal attachment at first postoperative month. A descriptive analysis was performed.

RESULTS

Among the 31 patients, 29 (93.5%) underwent surgeries due to rhegmatogenous retinal detachment and two (6.5%) for diabetic retinal detachment. The primary anatomic success was achieved in 27 (87.1%) patients. At the final visit, 17 (56.6%) had vision better than 20/400 (range, 20/30 to light perception). The vision was stable or improved in 22 (76.8%) patients at the end of follow-up. Nineteen (61.3%) patients required hypotensive eye drops after HSO use and twelve (38.7%) still required hypotensive eye drops at the final follow-up; three (9.7%) patients required additional glaucoma surgeries.

CONCLUSIONS

HSO is safe and useful for complex retinal detachments cases specially with inferior tears and PVR. Ocular hypertension is frequent and usually clinically controlled with hypotensive eyedrops. Close postoperatively follow-up is advised due to the ocular complications, particularly elevated intraocular pressure and emulsification.

BRSET: A Brazilian Multilabel Ophthalmological Dataset of Retina Fundus Photos

Introduction

The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups.

Methods

Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used Dino V2 Base for feature extraction, with 70% training and 30% testing subsets. Support Vector Machines (SVM) and Logistic Regression (LR) were employed with weighted training. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score.

Results

BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. Models were trained on BRSET in three prediction tasks: "diabetes diagnosis"; "sex classification"; and "diabetic retinopathy diagnosis".

Discussion

BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.

Recovery course of persistent posterior subretinal fluid after successful repair of rhegmatogenous retinal detachment

PURPOSE

To investigate best corrected visual acuity (BCVA), subretinal fluid (SRF) absorption time or ellipsoid zone (EZ) restoration time and various variables in patients with persistent SRF after successful primary repair of rhegmatogenous retinal detachment (RRD).

METHODS

This retrospective multicenter study allowed independent analysis of the healing pattern by two observers based on composite of serial cross-sectional macular optical coherence tomography (OCT) scans. Univariate and multivariate analyses were implemented.

RESULTS

One hundred and three cases had persistent SRF after pars plana vitrectomy, scleral buckling, or pneumatic retinopexy. By univariate analysis, SRF resolution time correlated positively with the number of retinal breaks (p < 0.001) and with increased myopia (p = 0.011). Using multivariate analysis, final BCVA (log MAR) correlated positively with age, duration of RRD, initial BCVA (OR = 3.28; [95%CI = 1.44-7.47]; p = 0.015), and SRF resolution time (OR = 0.46 [95%CI 0.21-1.05]; p = 0.049). EZ restoration time was longer with increasing number of retinal tears (OR = 0.67; [95%CI 0.29-1.52]; p = 0.030), worse final BCVA, and presence of macula-off RRD (OR = 0.26; [95%CI 0.08-0.88]; p = 0.056). SRF resolution time correlated marginally with prone position.

CONCLUSIONS

Residual posterior SRF is more common in eyes with multiple breaks or in myopic eyes. Final BCVA is better in younger subjects and in eyes with shorter duration of RRD. Persistent SRF is a self-limited disorder with a mean resolution of 11.2 months with good visual prognosis improving from a mean baseline logMAR of 1.08 to 0.25 at one year.

Ophthalmology emergency department visits in a Brazilian tertiary hospital over the last 11 years: data analysis

PURPOSE

This study aimed to describe the visits profile to Hospital São Paulo's ophthalmology emergency department, a 24-hour public open-access tertiary-care service in São Paulo, Brazil, that belongs to Federal University of São Paulo, over the last 11 years.

METHODS

A cross-sectional retrospective study was conducted, including all patients (n=634,726) admitted to the ophthalmology emergency department of Hospital São Paulo between January 2009 and December 2019.

RESULTS

From 2009 to 2019, the number of patients' presentations increased to 39.2%, with considerable visits variation across the period. The median age was 38 ± 20.4 years. Males represented 53.3%, and single-visit patients represented 53.1%. A total of 79.5% of patients' presentations occurred from 7 am to 5 pm, and 80.8% of patients' presentations occurred during regular weekdays. The most frequent diagnoses were conjunctivitis, blepharitis, keratitis, hordeolum/chalazion, and corneal foreign body.

CONCLUSIONS

Over the study period, presentations significantly increased in number, with nonurgent visits predominance, and a low number of single-visit patients. Our results demonstrate the ophthalmic visits profile and can lead to changes in the public health system to improve the quality of care and ophthalmology emergency access in São Paulo city.

Artificial intelligence for telemedicine diabetic retinopathy screening: a review

PURPOSE

This study aims to compare artificial intelligence (AI) systems applied in diabetic retinopathy (DR) teleophthalmology screening, currently deployed systems, fairness initiatives and the challenges for implementation.

METHODS

The review included articles retrieved from PubMed/Medline/EMBASE literature search strategy regarding telemedicine, DR and AI. The screening criteria included human articles in English, Portuguese or Spanish and related to telemedicine and AI for DR screening. The author's affiliations and the study's population income group were classified according to the World Bank Country and Lending Groups.

RESULTS

The literature search yielded a total of 132 articles, and nine were included after full-text assessment. The selected articles were published between 2004 and 2020 and were grouped as telemedicine systems, algorithms, economic analysis and image quality assessment. Four telemedicine systems that perform a quality assessment, image preprocessing and pathological screening were reviewed. A data and post-deployment bias assessment are not performed in any of the algorithms, and none of the studies evaluate the social impact implementations. There is a lack of representativeness in the reviewed articles, with most authors and target populations from high-income countries and no low-income country representation.

CONCLUSIONS

Telemedicine and AI hold great promise for augmenting decision-making in medical care, expanding patient access and enhancing cost-effectiveness. Economic studies and social science analysis are crucial to support the implementation of AI in teleophthalmology screening programs. Promoting fairness and generalizability in automated systems combined with telemedicine screening programs is not straightforward. Improving data representativeness, reducing biases and promoting equity in deployment and post-deployment studies are all critical steps in model development.

Fairness and generalizability of OCT normative databases: a comparative analysis

PURPOSE

In supervised Machine Learning algorithms, labels and reports are important in model development. To provide a normality assessment, the OCT has an in-built normative database that provides a color base scale from the measurement database comparison. This article aims to evaluate and compare normative databases of different OCT machines, analyzing patient demographic, contrast inclusion and exclusion criteria, diversity index, and statistical approach to assess their fairness and generalizability.

METHODS

Data were retrieved from Cirrus, Avanti, Spectralis, and Triton's FDA-approval and equipment manual. The following variables were compared: number of eyes and patients, inclusion and exclusion criteria, statistical approach, sex, race and ethnicity, age, participant country, and diversity index.

RESULTS

Avanti OCT has the largest normative database (640 eyes). In every database, the inclusion and exclusion criteria were similar, including adult patients and excluding pathological eyes. Spectralis has the largest White (79.7%) proportionately representation, Cirrus has the largest Asian (24%), and Triton has the largest Black (22%) patient representation. In all databases, the statistical analysis applied was Regression models. The sex diversity index is similar in all datasets, and comparable to the ten most populous contries. Avanti dataset has the highest diversity index in terms of race, followed by Cirrus, Triton, and Spectralis.

CONCLUSION

In all analyzed databases, the data framework is static, with limited upgrade options and lacking normative databases for new modules. As a result, caution in OCT normality interpretation is warranted. To address these limitations, there is a need for more diverse, representative, and open-access datasets that take into account patient demographics, especially considering the development of supervised Machine Learning algorithms in healthcare.

Fairness and generalisability in deep learning of retinopathy of prematurity screening algorithms: a literature review

BACKGROUND

Retinopathy of prematurity (ROP) is a vasoproliferative disease responsible for more than 30 000 blind children worldwide. Its diagnosis and treatment are challenging due to the lack of specialists, divergent diagnostic concordance and variation in classification standards. While artificial intelligence (AI) can address the shortage of professionals and provide more cost-effective management, its development needs fairness, generalisability and bias controls prior to deployment to avoid producing harmful unpredictable results. This review aims to compare AI and ROP study's characteristics, fairness and generalisability efforts.

METHODS

Our review yielded 220 articles, of which 18 were included after full-text assessment. The articles were classified into ROP severity grading, plus detection, detecting treatment requiring, ROP prediction and detection of retinal zones.

RESULTS

All the article's authors and included patients are from middle-income and high-income countries, with no low-income countries, South America, Australia and Africa Continents representation.Code is available in two articles and in one on request, while data are not available in any article. 88.9% of the studies use the same retinal camera. In two articles, patients' sex was described, but none applied a bias control in their models.

CONCLUSION

The reviewed articles included 180 228 images and reported good metrics, but fairness, generalisability and bias control remained limited. Reproducibility is also a critical limitation, with few articles sharing codes and none sharing data. Fair and generalisable ROP and AI studies are needed that include diverse datasets, data and code sharing, collaborative research, and bias control to avoid unpredictable and harmful deployments.

Digital Education for the Deployment of Artificial Intelligence in Health Care

Artificial Intelligence (AI) represents a significant milestone in health care's digital transformation. However, traditional health care education and training often lack digital competencies. To promote safe and effective AI implementation, health care professionals must acquire basic knowledge of machine learning and neural networks, critical evaluation of data sets, integration within clinical workflows, bias control, and human-machine interaction in clinical settings. Additionally, they should understand the legal and ethical aspects of digital health care and the impact of AI adoption. Misconceptions and fears about AI systems could jeopardize its real-life implementation. However, there are multiple barriers to promoting electronic health literacy, including time constraints, overburdened curricula, and the shortage of capacitated professionals. To overcome these challenges, partnerships among developers, professional societies, and academia are essential. Integrating specialists from different backgrounds, including data specialists, lawyers, and social scientists, can significantly contribute to combating digital illiteracy and promoting safe AI implementation in health care.

Clinical validation of a smartphone-based retinal camera for diabetic retinopathy screening

AIMS

This study aims to compare the performance of a handheld fundus camera (Eyer) and standard tabletop fundus cameras (Visucam 500, Visucam 540, and Canon CR-2) for diabetic retinopathy and diabetic macular edema screening.

METHODS

This was a multicenter, cross-sectional study that included images from 327 individuals with diabetes. The participants underwent pharmacological mydriasis and fundus photography in two fields (macula and optic disk centered) with both strategies. All images were acquired by trained healthcare professionals, de-identified, and graded independently by two masked ophthalmologists, with a third senior ophthalmologist adjudicating in discordant cases. The International Classification of Diabetic Retinopathy was used for grading, and demographic data, diabetic retinopathy classification, artifacts, and image quality were compared between devices. The tabletop senior ophthalmologist adjudication label was used as the ground truth for comparative analysis. A univariate and stepwise multivariate logistic regression was performed to determine the relationship of each independent factor in referable diabetic retinopathy.

RESULTS

The mean age of participants was 57.03 years (SD 16.82, 9-90 years), and the mean duration of diabetes was 16.35 years (SD 9.69, 1-60 years). Age (P = .005), diabetes duration (P = .004), body mass index (P = .005), and hypertension (P < .001) were statistically different between referable and non-referable patients. Multivariate logistic regression analysis revealed a positive association between male sex (OR 1.687) and hypertension (OR 3.603) with referable diabetic retinopathy. The agreement between devices for diabetic retinopathy classification was 73.18%, with a weighted kappa of 0.808 (almost perfect). The agreement for macular edema was 88.48%, with a kappa of 0.809 (almost perfect). For referable diabetic retinopathy, the agreement was 85.88%, with a kappa of 0.716 (substantial), sensitivity of 0.906, and specificity of 0.808. As for image quality, 84.02% of tabletop fundus camera images were gradable and 85.31% of the Eyer images were gradable.

CONCLUSIONS

Our study shows that the handheld retinal camera Eyer performed comparably to standard tabletop fundus cameras for diabetic retinopathy and macular edema screening. The high agreement with tabletop devices, portability, and low costs makes the handheld retinal camera a promising tool for increasing coverage of diabetic retinopathy screening programs, particularly in low-income countries. Early diagnosis and treatment have the potential to prevent avoidable blindness, and the present validation study brings evidence that supports its contribution to diabetic retinopathy early diagnosis and treatment.

An emergency room influx and trauma cases prediction in a Brazilian ophthalmological hospital by an ophthalmologist without code experience

PURPOSE

The emergency medical service is a fundamental part of healthcare, albeit crowded emergency rooms lead to delayed and low-quality assistance in actual urgent cases. Machine-learning algorithms can provide a smart and effective estimation of emergency patients' volume, which was previously restricted to artificial intelligence (AI) experts in coding and computer science but is now feasible by anyone without any coding experience through auto machine learning. This study aimed to create a machine-learning model designed by an ophthalmologist without any coding experience using AutoML to predict the influx in the emergency department and trauma cases.

METHODS

A dataset of 356,611 visits at Hospital da Universidade Federal de São Paulo from January 01, 2014 to December 31, 2019 was included in the model training, which included visits/day and the international classification disease code. The training and prediction were made with the Amazon Forecast by 2 ophthalmologists with no prior coding experience.

RESULTS

The forecast period predicted a mean emergency patient volume of 216.27/day in p90, 180.75/day in p50, and 140.35/day in p10, and a mean of 7.42 trauma cases/ day in p90, 3.99/day in p50, and 0.56/day in p10. In January of 2020, there were a total of 6,604 patient visits and a mean of 206.37 patients/day, which is 13.5% less than the p50 prediction. This period involved a total of 199 trauma cases and a mean of 6.21 cases/day, which is 55.77% more traumas than that by the p50 prediction.

CONCLUSIONS

The development of models was previously restricted to data scientists' experts in coding and computer science, but transfer learning autoML has enabled AI development by any person with no code experience mandatory. This study model showed a close value to the actual 2020 January visits, and the only factors that may have influenced the results between the two approaches are holidays and dataset size. This is the first study to apply AutoML in hospital visits forecast, showing a close prediction of the actual hospital influx.

Impact of SARS-CoV-2 pandemic on ophthalmological emergency visits: 1 year of experience

PURPOSE

The COVID-19 pandemic began in March 2020 and changed the healthcare system overall. The pandemic led to resource allocation changes, overloading of intensive care units, apprehensiveness of patients to seek medical care not related to COVID-19, and an abrupt reduction in all nonurgent consultations and surgeries. This study evaluated the impact on an ophthalmological emergency room for one year by assessing the correlation between societal lockdown phases and COVID-19 mortality.

METHODS

An observational, retrospective study was conducted that included all patients admitted to the Ophthalmology Emergency Department between January 1, 2019, and March 28, 2021. The visits were classified into prepandemic and pandemic groups that were then compared.

RESULTS

In the prepandemic period, the hospital registered a total of 71,485 visits with a mean of 194.78 ± 49.74 daily visits. In the pandemic group, there was a total of 41,791 visits with a mean of 114.18 ± 43.12 daily visits, which was a 41.4% decrease. A significant decrea-se (16.4 p<0.001) was observed in the prevalence of acute conjunctivitis, and a significant increase (6.4%; p<0.01) was observed in the prevalence of corneal foreign body disorders. A negative correlation was identified between the COVID-19 death rate and the ophthalmological inflow rates.

CONCLUSION

This one-year analysis showed a reduction of 41.4% in emergency department visits and a significant decrease in infectious conditions. A change in hygiene habits and social distancing could explain this reduction, and the increased prevalence of trauma consultations highlighted the need for preventive and educative measures during these types of restrictive periods.

Diabetic retinopathy classification for supervised machine learning algorithms

BACKGROUND

Artificial intelligence and automated technology were first reported more than 70 years ago and nowadays provide unprecedented diagnostic accuracy, screening capacity, risk stratification, and workflow optimization. Diabetic retinopathy is an important cause of preventable blindness worldwide, and artificial intelligence technology provides precocious diagnosis, monitoring, and guide treatment. High-quality exams are fundamental in supervised artificial intelligence algorithms, but the lack of ground truth standards in retinal exams datasets is a problem.

MAIN BODY

In this article, ETDRS, NHS, ICDR, SDGS diabetic retinopathy grading, and manual annotation are described and compared in publicly available datasets. The various DR labeling systems generate a fundamental problem for AI datasets. Possible solutions are standardization of DR classification and direct retinal-finding identifications.

CONCLUSION

Reliable labeling methods also need to be considered in datasets with more trustworthy labeling.