Expert Performance in Visual Differentiation of Bacterial and Fungal Keratitis

Diagnostic performance of deep learning for infectious keratitis: a systematic review and meta-analysis

Background

Infectious keratitis (IK) is the leading cause of corneal blindness globally. Deep learning (DL) is an emerging tool for medical diagnosis, though its value in IK is unclear. We aimed to assess the diagnostic accuracy of DL for IK and its comparative accuracy with ophthalmologists.

Methods

In this systematic review and meta-analysis, we searched EMBASE, MEDLINE, and clinical registries for studies related to DL for IK published between 1974 and July 16, 2024. We performed meta-analyses using bivariate models to estimate summary sensitivities and specificities. This systematic review was registered with PROSPERO (CRD42022348596).

Findings

Of 963 studies identified, 35 studies (136,401 corneal images from >56,011 patients) were included. Most studies had low risk of bias (68.6%) and low applicability concern (91.4%) in all domains of QUADAS-2, except the index test domain. Against the reference standard of expert consensus and/or microbiological results (seven external validation studies; 10,675 images), the summary estimates (95% CI) for sensitivity and specificity of DL for IK were 86.2% (71.6-93.9) and 96.3% (91.5-98.5). From 28 internal validation studies (16,059 images), summary estimates for sensitivity and specificity were 91.6% (86.8-94.8) and 90.7% (84.8-94.5). Based on seven studies (4007 images), DL and ophthalmologists had comparable summary sensitivity [89.2% (82.2-93.6) versus 82.2% (71.5-89.5); P = 0.20] and specificity [(93.2% (85.5-97.0) versus 89.6% (78.8-95.2); P = 0.45].

Interpretation

DL models may have good diagnostic accuracy for IK and comparable performance to ophthalmologists. These findings should be interpreted with caution due to the image-based analysis that did not account for potential correlation within individuals, relatively homogeneous population studies, lack of pre-specification of DL thresholds, and limited external validation. Future studies should improve their reporting, data diversity, external validation, transparency, and explainability to increase the reliability and generalisability of DL models for clinical deployment.

Funding

NIH, Wellcome Trust, MRC, Fight for Sight, BHP, and ESCRS.

Penetrative Ionic Organic Molecular Cage Nanozyme for the Targeted Treatment of Keratomycosis

Keratomycosis, caused by pathogenic fungi, is an intractable blinding eye disease. Corneal penetration is an essential requirement for conventional antifungal medications to address keratomycosis. Due to the distinctive anatomical and physiological structure of the cornea, the therapeutic efficacy is hampered by the inadequate penetration capacity. Despite the emergence of diverse antifungal drug delivery systems and advanced antifungal nanomaterials, it has remained challenging to achieve corneal penetration over the past decade. This study fabricates a penetrative ionic organic molecular cage-based nanozyme (OMCzyme) for treating keratomycosis. The synthesis of OMCzyme involved two steps. Initially, the ionic OMC is synthesized by a [2+3] cycloimination reaction of triformylphloroglucinol and 2,3-diaminopropionic acid. Subsequently, OMCzyme is fabricated by coordination of Fe2⁺ with carboxyl anions and phenolic hydroxyls in the organic cage, and further deposition of silver nanoparticles on the surface of OMC-Fe complex. The as-prepared OMCzyme demonstrates excellent water dispersion, peroxidase-like activity, in vitro and in vivo biocompatibility, and corneal penetration. Notably, the nanozyme displays targeted antifungal activity, effectively combating Fusarium solani with negligible cytotoxicity toward human corneal epithelial cells. The hybrid mimic is further demonstrated to be effective in treating keratomycosis in mice, indicating the potential of OMCzyme for curing fungal infectious diseases.

Overview of Mycotic Keratitis

ABSTRACT

Keratomycosis is a serious corneal infection associated with high ocular morbidity that can lead to severe vision loss. It is estimated to affect more than 1 million patients annually, most commonly occurring in tropical climates, and represents a growing threat to patients worldwide. Despite aggressive medical management, fungal infections have a higher rate of perforation requiring surgical intervention compared with other infectious etiologies. Early diagnosis and appropriate treatment are keys to preserving vision and saving patients' eyes.Timely diagnosis of fungal keratitis helps minimize corneal damage and scarring and increases the likelihood of a favorable outcome. Studies have shown that correct identification of fungal infections is often delayed up to 2 to 3 weeks after initial presentation. This leads to incorrect or ineffective treatment for many patients. Diagnostic techniques explored in this study include corneal scrapings with staining and culture, visualization with in vivo confocal microscopy, molecular diagnostic techniques including polymerase chain reaction, and recently developed omics-based technologies.Treatment of fungal keratitis begins with topical antifungals. Medical management has been proven to be effective, but with limitations including poor drug penetration and low bioavailability. Cases that do not respond to topical therapy require more invasive and novel treatments to control the infection. We review the clinical trials that have shaped current practice patterns, with focus on the efficacy of topical natamycin as the primary therapy for filamentous fungal keratitis. We explore additional management strategies such as localized intrastromal and intracameral injections of antifungal medications, photodynamic therapy, and surgical intervention.

Deep learning for multi-type infectious keratitis diagnosis: A nationwide, cross-sectional, multicenter study

The main cause of corneal blindness worldwide is keratitis, especially the infectious form caused by bacteria, fungi, viruses, and Acanthamoeba. The key to effective management of infectious keratitis hinges on prompt and precise diagnosis. Nevertheless, the current gold standard, such as cultures of corneal scrapings, remains time-consuming and frequently yields false-negative results. Here, using 23,055 slit-lamp images collected from 12 clinical centers nationwide, this study constructed a clinically feasible deep learning system, DeepIK, that could emulate the diagnostic process of a human expert to identify and differentiate bacterial, fungal, viral, amebic, and noninfectious keratitis. DeepIK exhibited remarkable performance in internal, external, and prospective datasets (all areas under the receiver operating characteristic curves > 0.96) and outperformed three other state-of-the-art algorithms (DenseNet121, InceptionResNetV2, and Swin-Transformer). Our study indicates that DeepIK possesses the capability to assist ophthalmologists in accurately and swiftly identifying various infectious keratitis types from slit-lamp images, thereby facilitating timely and targeted treatment.

Locality is the strongest predictor of expert performance in image-based differentiation of bacterial and fungal corneal ulcers from India

PURPOSE

This study sought to identify the sources of differential performance and misclassification error among local (Indian) and external (non-Indian) corneal specialists in identifying bacterial and fungal keratitis based on corneal photography.

METHODS

This study is a secondary analysis of survey data assessing the ability of corneal specialists to identify acute bacterial versus fungal keratitis by using corneal photography. One-hundred images of 100 eyes from 100 patients with acute bacterial or fungal keratitis in South India were previously presented to an international cohort of cornea specialists for interpretation over the span of April to July 2021. Each expert provided a predicted probability that the ulcer was either bacterial or fungal. Using these data, we performed multivariable linear regression to identify factors predictive of expert performance, accounting for primary practice location and surrogate measures to infer local fungal ulcer prevalence, including locality, latitude, and dew point. In addition, Brier score decomposition was used to determine experts' reliability ("calibration") and resolution ("boldness") and were compared between local (Indian) and external (non-Indian) experts.

RESULTS

Sixty-six experts from 16 countries participated. Indian practice location was the only independently significant predictor of performance in multivariable linear regression. Resolution among Indian experts was significantly better (0.08) than among non-Indian experts (0.01; P < 0.001), indicating greater confidence in their predictions. There was no significant difference in reliability between the two groups ( P = 0.40).

CONCLUSION

Local cornea experts outperformed their international counterparts independent of regional variability in tropical risk factors for fungal keratitis. This may be explained by regional characteristics of infectious ulcers with which local corneal specialists are familiar.

Corneal Specialists' Confidence in Identifying Causal Organisms of Microbial Keratitis

PURPOSE

Microbial keratitis (MK) is a potentially blinding corneal disease caused by an array of microbial etiologies. However, the lack of early organism identification is a barrier to optimal care. We investigated clinician confidence in their diagnosis of organism type on initial presentation and the relationship between confidence and presenting features.

METHODS

This research presents secondary data analysis of 72 patients from the Automated Quantitative Ulcer Analysis (AQUA) study. Cornea specialists reported their confidence in organism identification. Presenting sample characteristics were recorded including patient demographics, health history, infection morphology, symptoms, and circumstances of infection. The association between confidence and presenting characteristics was investigated with 2-sample t-tests, Wilcoxon tests, and Chi-square or Fisher's exact tests.

RESULTS

Clinicians reported being "confident or very confident" in their diagnosis of the causal organism in MK infections for 39 patients (54%) and "not confident" for 33 patients (46%). Confidence was not significantly associated with patient demographics, morphologic features, or symptoms related to MK. MK cases where clinicians reported they were confident, versus not confident in their diagnosis, showed significantly smaller percentages of previous corneal disease (0% versus 15%, p = 0.017), were not seen by an outside provider first (69% versus 94%, p = 0.015), or had no prior labs drawn (8% versus 33%, p = 0.046), and a significantly larger percentage of cases wore contact lenses (54% versus 28%, p = 0.029).

CONCLUSION

In almost half of MK cases, cornea specialists reported lack of confidence in identifying the infection type. Confidence was related to ocular history and circumstances of infection but not by observable signs and symptoms or patient demographics. Tools are needed to assist clinicians with early diagnosis of MK infection type to expedite care and healing.

Validation of the C-DU(KE) Calculator as a Predictor of Outcomes in Patients Enrolled in Steroids for Corneal Ulcer and Mycotic Ulcer Treatment Trials

PURPOSE

The aim of this study was to validate the C-DU(KE) calculator as a predictor of treatment outcomes on a data set derived from patients with culture-positive ulcers.

METHODS

C-DU(KE) criteria were compiled from a data set consisting of 1063 cases of infectious keratitis from the Steroids for Corneal Ulcer Trial (SCUT) and Mycotic Ulcer Treatment Trial (MUTT) studies. These criteria include corticosteroid use after symptoms, visual acuity, ulcer area, fungal etiology, and elapsed time to organism-sensitive therapy. Univariate analysis was performed followed by multivariable logistic regressions on culture-exclusive and culture-inclusive models to assess for associations between the variables and outcome. The predictive probability of treatment failure, defined as the need for surgical intervention, was calculated for each study participant. Discrimination was assessed using the area under the curve for each model.

RESULTS

Overall, 17.9% of SCUT/MUTT participants required surgical intervention. Univariate analysis showed that decreased visual acuity, larger ulcer area, and fungal etiology had a significant association with failed medical management. The other 2 criteria did not. In the culture-exclusive model, 2 of 3 criteria, decreased vision [odds ratio (OR) = 3.13, P < 0.001] and increased ulcer area (OR = 1.03, P < 0.001), affected outcomes. In the culture-inclusive model, 3 of 5 criteria, decreased vision (OR = 4.9, P < 0.001), ulcer area (OR = 1.02, P < 0.001), and fungal etiology (OR = 9.8, P < 0.001), affected results. The area under the curves were 0.784 for the culture-exclusive model and 0.846 for the culture-inclusive model which were comparable to the original study.

CONCLUSIONS

The C-DU(KE) calculator is generalizable to a study population from large international studies primarily taking place in India. These results support its use as a risk stratification tool assisting ophthalmologists in patient management.

Antimicrobial Drug-Drug Interactions in the Treatment of Infectious Keratitis

PURPOSE

Infectious keratitis is a serious disease requiring immediate, intensive, and broad-spectrum empiric treatment to prevent vision loss. Given the diversity of organisms that can cause serious corneal disease, current guidelines recommend treatment with several antimicrobial agents simultaneously to provide adequate coverage while awaiting results of microbiology cultures. However, it is currently unknown how the use of multiple ophthalmic antimicrobial agents in combination may affect the efficacy of individual drugs.

METHODS

Using a panel of 9 ophthalmic antibiotics, 3 antifungal agents, and 2 antiacanthamoeba therapeutics, fractional inhibitory concentration testing in the standard checkerboard format was used to study 36 antibiotic-antibiotic combinations, 27 antibiotic-antifungal combinations, and 18 antibiotic-antiacanthamoeba combinations against both Staphylococcus aureus and Pseudomonas aeruginosa for synergistic, additive, neutral, or antagonistic drug-drug interactions.

RESULTS

We demonstrate that while most combinations resulted in no change in antimicrobial efficacy of individual components, the combination of erythromycin + polyhexamethylene biguanide was found to be antagonistic toward P. aeruginosa . Conversely, 18 combinations toward S. aureus and 15 combinations toward P. aeruginosa resulted in additive or synergistic activity, including 4 with improved activity toward both species.

CONCLUSIONS

Understanding how drug-drug interactions may affect drug efficacy is critical to selecting the appropriate combination therapy and improving clinical outcomes of this blinding disease.

Assessing the Impact of Image Quality on Deep Learning Classification of Infectious Keratitis

Objective

To investigate the impact of corneal photograph quality on convolutional neural network (CNN) predictions.

Design

A CNN trained to classify bacterial and fungal keratitis was evaluated using photographs of ulcers labeled according to 5 corneal image quality parameters: eccentric gaze direction, abnormal eyelid position, over/under-exposure, inadequate focus, and malpositioned light reflection.

Participants

All eligible subjects with culture and stain-proven bacterial and/or fungal ulcers presenting to Aravind Eye Hospital in Madurai, India, between January 1, 2021 and December 31, 2021.

Methods

Convolutional neural network classification performance was compared for each quality parameter, and gradient class activation heatmaps were generated to visualize regions of highest influence on CNN predictions.

Main Outcome Measures

Area under the receiver operating characteristic and precision recall curves were calculated to quantify model performance. Bootstrapped confidence intervals were used for statistical comparisons. Logistic loss was calculated to measure individual prediction accuracy.

Results

Individual presence of either light reflection or eyelids obscuring the corneal surface was associated with significantly higher CNN performance. No other quality parameter significantly influenced CNN performance. Qualitative review of gradient class activation heatmaps generally revealed the infiltrate as having the highest diagnostic relevance.

Conclusions

The CNN demonstrated expert-level performance regardless of image quality. Future studies may investigate use of smartphone cameras and image sets with greater variance in image quality to further explore the influence of these parameters on model performance.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Quantifying Clinicians' Diagnostic Uncertainty When Making Initial Treatment Decisions for Microbial Keratitis

PURPOSE

There is a need to understand physicians' diagnostic uncertainty in the initial management of microbial keratitis (MK). This study aimed to understand corneal specialists' diagnostic uncertainty by establishing risk thresholds for treatment of MK that could be used to inform a decision curve analysis for prediction modeling.

METHODS

A cross-sectional survey of corneal specialists with at least 2 years clinical experience was conducted. Clinicians provided the percentage risk at which they would always or never treat MK types (bacterial, fungal, herpetic, and amoebic) based on initial ulcer sizes and locations (<2 mm 2 central, <2 mm 2 peripheral, and >8 mm 2 central).

RESULTS

Seventy-two of 99 ophthalmologists participated who were 50% female with an average of 14.7 (SD = 10.1) years of experience, 60% in academic practices, and 38% outside the United States. Clinicians reported they would "never" and "always" treat a <2 mm 2 central MK infection if the median risk was 0% and 20% for bacterial (interquartile range, IQR = 0-5 and 5-50), 4.5% and 27.5% for herpetic (IQR = 0-10 and 10-50), 5% and 50% for fungal (IQR = 0-10 and 20-75), and 5% and 50.5% for amoebic (IQR = 0-20 and 32-80), respectively. Mixed-effects models showed lower thresholds to treat larger and central infections ( P < 0.001, respectively), and thresholds to always treat differed between MK types for the United States ( P < 0.001) but not international clinicians.

CONCLUSIONS

Risk thresholds to treat differed by practice locations and MK types, location, and size. Researchers can use these thresholds to understand when a clinician is uncertain and to create decision support tools to guide clinicians' treatment decisions.

Automatic Diagnosis of Infectious Keratitis Based on Slit Lamp Images Analysis

Infectious keratitis (IK) is a common ophthalmic emergency that requires prompt and accurate treatment. This study aimed to propose a deep learning (DL) system based on slit lamp images to automatically screen and diagnose infectious keratitis. This study established a dataset of 2757 slit lamp images from 744 patients, including normal cornea, viral keratitis (VK), fungal keratitis (FK), and bacterial keratitis (BK). Six different DL algorithms were developed and evaluated for the classification of infectious keratitis. Among all the models, the EffecientNetV2-M showed the best classification ability, with an accuracy of 0.735, a recall of 0.680, and a specificity of 0.904, which was also superior to two ophthalmologists. The area under the receiver operating characteristics curve (AUC) of the EffecientNetV2-M was 0.85; correspondingly, 1.00 for normal cornea, 0.87 for VK, 0.87 for FK, and 0.64 for BK. The findings suggested that the proposed DL system could perform well in the classification of normal corneas and different types of infectious keratitis, based on slit lamp images. This study proves the potential of the DL model to help ophthalmologists to identify infectious keratitis and improve the accuracy and efficiency of diagnosis.

Development and multi-center validation of machine learning model for early detection of fungal keratitis

BACKGROUND

Fungal keratitis (FK) is a leading cause of corneal blindness in developing countries due to poor clinical recognition and laboratory identification. Here, we aimed to identify the distinct clinical signature of FK and develop a diagnostic model to differentiate FK from other types of infectious keratitis.

METHODS

We reviewed the electronic health records (EHRs) of all patients with suspected infectious keratitis in Beijing Tongren Hospital from January 2011 to December 2021. Twelve clinical signs of slit-lamp images were assessed by Lasso regression analysis and collinear variables were excluded. Three models based on binary logistic regression, random forest classification, and decision tree classification were trained for FK diagnosis and employed for internal validation. Independent external validation of the models was performed in a cohort of 420 patients from seven different ophthalmic centers to evaluate the accuracy, specificity, and sensitivity in real world.

FINDINGS

Three diagnostic models of FK based on binary logistic regression, random forest classification, and decision tree classification were established and internal validation were achieved with the mean AUC of 0.916, 0.920, and 0.859, respectively. The models were well-calibrated by external validation using a prospective cohort including 210 FK and 210 non-FK patients from seven eye centers across China. The diagnostic model with the binary logistic regression algorithm classified the external validation dataset with a sensitivity of 0.907 (0.774, 1.000), specificity 0.899 (0.750, 1.000), accuracy 0.905 (0.805, 1.000), and AUC 0.903 (0.808, 0.998).

INTERPRETATION

Our model enables rapid identification of FK, which will help ophthalmologists to establish a preliminary diagnosis and to improve the diagnostic accuracy in clinic.

FUNDING

The Open Research Fund from the National Key Research and Development Program of China (2021YFC2301000) and the Open Research Fund from Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, Beijing Tongren Hospital, Beihang University &Capital Medical University (BHTR-KFJJ-202001) supported this study.

Deep learning-based classification of infectious keratitis on slit-lamp images

BACKGROUND

Infectious keratitis (IK) is an ocular emergency caused by a variety of microorganisms, including bacteria, fungi, viruses, and parasites. Culture-based methods were the gold standard for diagnosing IK, but difficult biopsy, delaying report, and low positive rate limited their clinical application.

OBJECTIVES

This study aims to construct a deep-learning-based auxiliary diagnostic model for early IK diagnosis.

DESIGN

A retrospective study.

METHODS

IK patients with pathological diagnosis were enrolled and their slit-lamp photos were collected. Image augmentation, normalization, and histogram equalization were applied, and five image classification networks were implemented and compared. Model blending technique was used to combine the advantages of single model. The performance of combined model was validated by 10-fold cross-validation, receiver operating characteristic curves (ROC), confusion matrix, Gradient-wright class activation mapping (Grad-CAM) visualization, and t-distributed Stochastic Neighbor Embedding (t-SNE). Three experienced cornea specialists were invited and competed with the combined model on making clinical decisions.

RESULTS

Overall, 4830 slit-lamp images were collected from patients diagnosed with IK between June 2010 and May 2021, including 1490 (30.8%) bacterial keratitis (BK), 1670 (34.6%) fungal keratitis (FK), 600 (12.4%) herpes simplex keratitis (HSK), and 1070 (22.2%) Acanthamoeba keratitis (AK). KeratitisNet, the combination of ResNext101_32x16d and DenseNet169, reached the highest accuracy 77.08%. The accuracy of KeratitisNet for diagnosing BK, FK, AK, and HSK was 70.27%, 77.71%, 83.81%, and 79.31%, and AUC was 0.86, 0.91, 0.96, and 0.98, respectively. KeratitisNet was mainly confused in distinguishing BK and FK. There were 20% of BK cases mispredicted into FK and 16% of FK cases mispredicted into BK. In diagnosing each type of IK, the accuracy of model was significantly higher than that of human ophthalmologists (p < 0.001).

CONCLUSION

KeratitisNet demonstrates a good performance on clinical IK diagnosis and classification. Deep learning could provide an auxiliary diagnostic method to help clinicians suspect IK using different corneal manifestations.

Artificial intelligence and corneal diseases

PURPOSE OF REVIEW

Artificial intelligence has advanced rapidly in recent years and has provided powerful tools to aid with the diagnosis, management, and treatment of ophthalmic diseases. This article aims to review the most current clinical artificial intelligence applications in anterior segment diseases, with an emphasis on microbial keratitis, keratoconus, dry eye syndrome, and Fuchs endothelial dystrophy.

RECENT FINDINGS

Most current artificial intelligence approaches have focused on developing deep learning algorithms based on various imaging modalities. Algorithms have been developed to detect and differentiate microbial keratitis classes and quantify microbial keratitis features. Artificial intelligence may aid with early detection and staging of keratoconus. Many advances have been made to detect, segment, and quantify features of dry eye syndrome and Fuchs. There is significant variability in the reporting of methodology, patient population, and outcome metrics.

SUMMARY

Artificial intelligence shows great promise in detecting, diagnosing, grading, and measuring diseases. There is a need for standardization of reporting to improve the transparency, validity, and comparability of algorithms.

Image-Based Differentiation of Bacterial and Fungal Keratitis Using Deep Convolutional Neural Networks

Purpose

Develop computer vision models for image-based differentiation of bacterial and fungal corneal ulcers and compare their performance against human experts.

Design

Cross-sectional comparison of diagnostic performance.

Participants

Patients with acute, culture-proven bacterial or fungal keratitis from 4 centers in South India.

Methods

Five convolutional neural networks (CNNs) were trained using images from handheld cameras collected from patients with culture-proven corneal ulcers in South India recruited as part of clinical trials conducted between 2006 and 2015. Their performance was evaluated on 2 hold-out test sets (1 single center and 1 multicenter) from South India. Twelve local expert cornea specialists performed remote interpretation of the images in the multicenter test set to enable direct comparison against CNN performance.

Main Outcome Measures

Area under the receiver operating characteristic curve (AUC) individually and for each group collectively (i.e., CNN ensemble and human ensemble).

Results

The best-performing CNN architecture was MobileNet, which attained an AUC of 0.86 on the single-center test set (other CNNs range, 0.68-0.84) and 0.83 on the multicenter test set (other CNNs range, 0.75-0.83). Expert human AUCs on the multicenter test set ranged from 0.42 to 0.79. The CNN ensemble achieved a statistically significantly higher AUC (0.84) than the human ensemble (0.76; P < 0.01). CNNs showed relatively higher accuracy for fungal (81%) versus bacterial (75%) ulcers, whereas humans showed relatively higher accuracy for bacterial (88%) versus fungal (56%) ulcers. An ensemble of the best-performing CNN and best-performing human achieved the highest AUC of 0.87, although this was not statistically significantly higher than the best CNN (0.83; P = 0.17) or best human (0.79; P = 0.09).

Conclusions

Computer vision models achieved superhuman performance in identifying the underlying infectious cause of corneal ulcers compared with cornea specialists. The best-performing model, MobileNet, attained an AUC of 0.83 to 0.86 without any additional clinical or historical information. These findings suggest the potential for future implementation of these models to enable earlier directed antimicrobial therapy in the management of infectious keratitis, which may improve visual outcomes. Additional studies are ongoing to incorporate clinical history and expert opinion into predictive models.