A deep network DeepOpacityNet for detection of cataracts from color fundus photographs

BACKGROUND

Cataract diagnosis typically requires in-person evaluation by an ophthalmologist. However, color fundus photography (CFP) is widely performed outside ophthalmology clinics, which could be exploited to increase the accessibility of cataract screening by automated detection.

METHODS

DeepOpacityNet was developed to detect cataracts from CFP and highlight the most relevant CFP features associated with cataracts. We used 17,514 CFPs from 2573 AREDS2 participants curated from the Age-Related Eye Diseases Study 2 (AREDS2) dataset, of which 8681 CFPs were labeled with cataracts. The ground truth labels were transferred from slit-lamp examination of nuclear cataracts and reading center grading of anterior segment photographs for cortical and posterior subcapsular cataracts. DeepOpacityNet was internally validated on an independent test set (20%), compared to three ophthalmologists on a subset of the test set (100 CFPs), externally validated on three datasets obtained from the Singapore Epidemiology of Eye Diseases study (SEED), and visualized to highlight important features.

RESULTS

Internally, DeepOpacityNet achieved a superior accuracy of 0.66 (95% confidence interval (CI): 0.64-0.68) and an area under the curve (AUC) of 0.72 (95% CI: 0.70-0.74), compared to that of other state-of-the-art methods. DeepOpacityNet achieved an accuracy of 0.75, compared to an accuracy of 0.67 for the ophthalmologist with the highest performance. Externally, DeepOpacityNet achieved AUC scores of 0.86, 0.88, and 0.89 on SEED datasets, demonstrating the generalizability of our proposed method. Visualizations show that the visibility of blood vessels could be characteristic of cataract absence while blurred regions could be characteristic of cataract presence.

CONCLUSIONS

DeepOpacityNet could detect cataracts from CFPs in AREDS2 with performance superior to that of ophthalmologists and generate interpretable results. The code and models are available at https://github.com/ncbi/DeepOpacityNet ( https://doi.org/10.5281/zenodo.10127002 ).

Popular large language model chatbots' accuracy, comprehensiveness, and self-awareness in answering ocular symptom queries

In light of growing interest in using emerging large language models (LLMs) for self-diagnosis, we systematically assessed the performance of ChatGPT-3.5, ChatGPT-4.0, and Google Bard in delivering proficient responses to 37 common inquiries regarding ocular symptoms. Responses were masked, randomly shuffled, and then graded by three consultant-level ophthalmologists for accuracy (poor, borderline, good) and comprehensiveness. Additionally, we evaluated the self-awareness capabilities (ability to self-check and self-correct) of the LLM-Chatbots. 89.2% of ChatGPT-4.0 responses were 'good'-rated, outperforming ChatGPT-3.5 (59.5%) and Google Bard (40.5%) significantly (all p < 0.001). All three LLM-Chatbots showed optimal mean comprehensiveness scores as well (ranging from 4.6 to 4.7 out of 5). However, they exhibited subpar to moderate self-awareness capabilities. Our study underscores the potential of ChatGPT-4.0 in delivering accurate and comprehensive responses to ocular symptom inquiries. Future rigorous validation of their performance is crucial to ensure their reliability and appropriateness for actual clinical use.

Benchmarking large language models' performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard

BACKGROUND

Large language models (LLMs) are garnering wide interest due to their human-like and contextually relevant responses. However, LLMs' accuracy across specific medical domains has yet been thoroughly evaluated. Myopia is a frequent topic which patients and parents commonly seek information online. Our study evaluated the performance of three LLMs namely ChatGPT-3.5, ChatGPT-4.0, and Google Bard, in delivering accurate responses to common myopia-related queries.

METHODS

We curated thirty-one commonly asked myopia care-related questions, which were categorised into six domains-pathogenesis, risk factors, clinical presentation, diagnosis, treatment and prevention, and prognosis. Each question was posed to the LLMs, and their responses were independently graded by three consultant-level paediatric ophthalmologists on a three-point accuracy scale (poor, borderline, good). A majority consensus approach was used to determine the final rating for each response. 'Good' rated responses were further evaluated for comprehensiveness on a five-point scale. Conversely, 'poor' rated responses were further prompted for self-correction and then re-evaluated for accuracy.

FINDINGS

ChatGPT-4.0 demonstrated superior accuracy, with 80.6% of responses rated as 'good', compared to 61.3% in ChatGPT-3.5 and 54.8% in Google Bard (Pearson's chi-squared test, all p ≤ 0.009). All three LLM-Chatbots showed high mean comprehensiveness scores (Google Bard: 4.35; ChatGPT-4.0: 4.23; ChatGPT-3.5: 4.11, out of a maximum score of 5). All LLM-Chatbots also demonstrated substantial self-correction capabilities: 66.7% (2 in 3) of ChatGPT-4.0's, 40% (2 in 5) of ChatGPT-3.5's, and 60% (3 in 5) of Google Bard's responses improved after self-correction. The LLM-Chatbots performed consistently across domains, except for 'treatment and prevention'. However, ChatGPT-4.0 still performed superiorly in this domain, receiving 70% 'good' ratings, compared to 40% in ChatGPT-3.5 and 45% in Google Bard (Pearson's chi-squared test, all p ≤ 0.001).

INTERPRETATION

Our findings underscore the potential of LLMs, particularly ChatGPT-4.0, for delivering accurate and comprehensive responses to myopia-related queries. Continuous strategies and evaluations to improve LLMs' accuracy remain crucial.

FUNDING

Dr Yih-Chung Tham was supported by the National Medical Research Council of Singapore (NMRC/MOH/HCSAINV21nov-0001).

Deep learning algorithms for automatic detection of pterygium using anterior segment photographs from slit-lamp and hand-held cameras

BACKGROUND/AIMS

To evaluate the performances of deep learning (DL) algorithms for detection of presence and extent pterygium, based on colour anterior segment photographs (ASPs) taken from slit-lamp and hand-held cameras.

METHODS

Referable pterygium was defined as having extension towards the cornea from the limbus of >2.50 mm or base width at the limbus of >5.00 mm. 2503 images from the Singapore Epidemiology of Eye Diseases (SEED) study were used as the development set. Algorithms were validated on an internal set from the SEED cohort (629 images (55.3% pterygium, 8.4% referable pterygium)), and tested on two external clinic-based sets (set 1 with 2610 images (2.8% pterygium, 0.7% referable pterygium, from slit-lamp ASP); and set 2 with 3701 images, 2.5% pterygium, 0.9% referable pterygium, from hand-held ASP).

RESULTS

The algorithm's area under the receiver operating characteristic curve (AUROC) for detection of any pterygium was 99.5%(sensitivity=98.6%; specificity=99.0%) in internal test set, 99.1% (sensitivity=95.9%, specificity=98.5%) in external test set 1 and 99.7% (sensitivity=100.0%; specificity=88.3%) in external test set 2. For referable pterygium, the algorithm's AUROC was 98.5% (sensitivity=94.0%; specificity=95.3%) in internal test set, 99.7% (sensitivity=87.2%; specificity=99.4%) in external set 1 and 99.0% (sensitivity=94.3%; specificity=98.0%) in external set 2.

CONCLUSION

DL algorithms based on ASPs can detect presence of and referable-level pterygium with optimal sensitivity and specificity. These algorithms, particularly if used with a handheld camera, may potentially be used as a simple screening tool for detection of referable pterygium. Further validation in community setting is warranted.

SYNOPSIS/PRECIS

DL algorithms based on ASPs can detect presence of and referable-level pterygium optimally, and may be used as a simple screening tool for the detection of referable pterygium in community screenings.

Detecting visually significant cataract using retinal photograph-based deep learning

Age-related cataracts are the leading cause of visual impairment among older adults. Many significant cases remain undiagnosed or neglected in communities, due to limited availability or accessibility to cataract screening. In the present study, we report the development and validation of a retinal photograph-based, deep-learning algorithm for automated detection of visually significant cataracts, using more than 25,000 images from population-based studies. In the internal test set, the area under the receiver operating characteristic curve (AUROC) was 96.6%. External testing performed across three studies showed AUROCs of 91.6-96.5%. In a separate test set of 186 eyes, we further compared the algorithm's performance with 4 ophthalmologists' evaluations. The algorithm performed comparably, if not being slightly more superior (sensitivity of 93.3% versus 51.7-96.6% by ophthalmologists and specificity of 99.0% versus 90.7-97.9% by ophthalmologists). Our findings show the potential of a retinal photograph-based screening tool for visually significant cataracts among older adults, providing more appropriate referrals to tertiary eye centers.

Deep Learning Automated Diagnosis and Quantitative Classification of Cataract Type and Severity

PURPOSE

To develop and evaluate deep learning models to perform automated diagnosis and quantitative classification of age-related cataract, including all three anatomical types, from anterior segment photographs.

DESIGN

Application of deep learning models to Age-Related Eye Disease Study (AREDS) dataset.

PARTICIPANTS

18,999 photographs (6,333 triplets) from longitudinal follow-up of 1,137 eyes (576 AREDS participants).

METHODS

Deep learning models were trained to detect and quantify nuclear cataract (NS; scale 0.9-7.1) from 45-degree slit-lamp photographs and cortical (CLO; scale 0-100%) and posterior subcapsular (PSC; scale 0-100%) cataract from retroillumination photographs. Model performance was compared with that of 14 ophthalmologists and 24 medical students. The ground truth labels were from reading center grading.

MAIN OUTCOME MEASURES

Mean squared error (MSE).

RESULTS

On the full test set, mean MSE values for the deep learning models were: 0.23 (SD 0.01) for NS, 13.1 (SD 1.6) for CLO, and 16.6 (SD 2.4) for PSC. On a subset of the test set (substantially enriched for positive cases of CLO and PSC), for NS, mean MSE for the models was 0.23 (SD 0.02), compared to 0.98 (SD 0.23; p=0.000001) for the ophthalmologists, and 1.24 (SD 0.33; p=0.000005) for the medical students. For CLO, mean MSE values were 53.5 (SD 14.8), compared to 134.9 (SD 89.9; p=0.003) and 422.0 (SD 944.4; p=0.0007), respectively. For PSC, mean MSE values were 171.9 (SD 38.9), compared to 176.8 (SD 98.0; p=0.67) and 395.2 (SD 632.5; p=0.18), respectively. In external validation on the Singapore Malay Eye Study (sampled to reflect the distribution of cataract severity in AREDS), MSE was 1.27 for NS and 25.5 for PSC.

CONCLUSIONS

A deep learning framework was able to perform automated and quantitative classification of cataract severity for all three types of age-related cataract. For the two most common types (NS and CLO), the accuracy was significantly superior to that of ophthalmologists; for the least common type (PSC), the accuracy was similar. The framework may have wide potential applications in both clinical and research domains. In the future, such approaches may increase the accessibility of cataract assessment globally. The code and models are publicly available at https://XXX.

Artificial Intelligence for Cataract Detection and Management

The rising popularity of artificial intelligence (AI) in ophthalmology is fuelled by the ever-increasing clinical "big data" that can be used for algorithm development. Cataract is one of the leading causes of visual impairment worldwide. However, compared with other major age-related eye diseases, such as diabetic retinopathy, age-related macular degeneration, and glaucoma, AI development in the domain of cataract is still relatively underexplored. In this regard, several previous studies explored algorithms for automated cataract assessment using either slit lamp of color fundus photographs. However, several other study groups proposed or derived new AI-based calculation for pre-cataract surgery intraocular lens power. Along with advancements in digitization of clinical data, data curation for future cataract-related AI developmental work is bound to undergo significant improvements in the foreseeable future. Even though most of these previous studies reported early promising performances, limitations such as lack of robust, high-quality training data, and lack of external validations remain. In the next phase of work, apart from algorithm's performance, it will also be pertinent to evaluate deployment angles, feasibility, efficiency, and cost-effectiveness of these new cataract-related AI systems.