Development of a deep learning-based image quality control system to detect and filter out ineligible slit-lamp images: A multicenter study

Promoting smartphone-based keratitis screening using meta-learning: A multicenter study

OBJECTIVE

Keratitis is the primary cause of corneal blindness worldwide. Prompt identification and referral of patients with keratitis are fundamental measures to improve patient prognosis. Although deep learning can assist ophthalmologists in automatically detecting keratitis through a slit lamp camera, remote and underserved areas often lack this professional equipment. Smartphones, a widely available device, have recently been found to have potential in keratitis screening. However, given the limited data available from smartphones, employing traditional deep learning algorithms to construct a robust intelligent system presents a significant challenge. This study aimed to propose a meta-learning framework, cosine nearest centroid-based metric learning (CNCML), for developing a smartphone-based keratitis screening model in the case of insufficient smartphone data by leveraging the prior knowledge acquired from slit-lamp photographs.

METHODS

We developed and assessed CNCML based on 13,009 slit-lamp photographs and 4,075 smartphone photographs that were obtained from 3 independent clinical centers. To mimic real-world scenarios with various degrees of sample scarcity, we used training sets of different sizes (0 to 20 photographs per class) from the HUAWEI smartphone to train CNCML. We evaluated the performance of CNCML not only on an internal test dataset but also on two external datasets that were collected by two different brands of smartphones (VIVO and XIAOMI) in another clinical center. Furthermore, we compared the performance of CNCML with that of traditional deep learning models on these smartphone datasets. The accuracy and macro-average area under the curve (macro-AUC) were utilized to evaluate the performance of models.

RESULTS

With merely 15 smartphone photographs per class used for training, CNCML reached accuracies of 84.59%, 83.15%, and 89.99% on three smartphone datasets, with corresponding macro-AUCs of 0.96, 0.95, and 0.98, respectively. The accuracies of CNCML on these datasets were 0.56% to 9.65% higher than those of the most competitive traditional deep learning models.

CONCLUSIONS

CNCML exhibited fast learning capabilities, attaining remarkable performance with a small number of training samples. This approach presents a potential solution for transitioning intelligent keratitis detection from professional devices (e.g., slit-lamp cameras) to more ubiquitous devices (e.g., smartphones), making keratitis screening more convenient and effective.

Deep learning for multi-type infectious keratitis diagnosis: A nationwide, cross-sectional, multicenter study

The main cause of corneal blindness worldwide is keratitis, especially the infectious form caused by bacteria, fungi, viruses, and Acanthamoeba. The key to effective management of infectious keratitis hinges on prompt and precise diagnosis. Nevertheless, the current gold standard, such as cultures of corneal scrapings, remains time-consuming and frequently yields false-negative results. Here, using 23,055 slit-lamp images collected from 12 clinical centers nationwide, this study constructed a clinically feasible deep learning system, DeepIK, that could emulate the diagnostic process of a human expert to identify and differentiate bacterial, fungal, viral, amebic, and noninfectious keratitis. DeepIK exhibited remarkable performance in internal, external, and prospective datasets (all areas under the receiver operating characteristic curves > 0.96) and outperformed three other state-of-the-art algorithms (DenseNet121, InceptionResNetV2, and Swin-Transformer). Our study indicates that DeepIK possesses the capability to assist ophthalmologists in accurately and swiftly identifying various infectious keratitis types from slit-lamp images, thereby facilitating timely and targeted treatment.

An ensemble deep learning diagnostic system for determining Clinical Activity Scores in thyroid-associated ophthalmopathy: integrating multi-view multimodal images from anterior segment slit-lamp photographs and facial images

Background

Thyroid-associated ophthalmopathy (TAO) is the most prevalent autoimmune orbital condition, significantly impacting patients' appearance and quality of life. Early and accurate identification of active TAO along with timely treatment can enhance prognosis and reduce the occurrence of severe cases. Although the Clinical Activity Score (CAS) serves as an effective assessment system for TAO, it is susceptible to assessor experience bias. This study aimed to develop an ensemble deep learning system that combines anterior segment slit-lamp photographs of patients with facial images to simulate expert assessment of TAO.

Method

The study included 156 patients with TAO who underwent detailed diagnosis and treatment at Shanxi Eye Hospital Affiliated to Shanxi Medical University from May 2020 to September 2023. Anterior segment slit-lamp photographs and facial images were used as different modalities and analyzed from multiple perspectives. Two ophthalmologists with more than 10 years of clinical experience independently determined the reference CAS for each image. An ensemble deep learning model based on the residual network was constructed under supervised learning to predict five key inflammatory signs (redness of the eyelids and conjunctiva, and swelling of the eyelids, conjunctiva, and caruncle or plica) associated with TAO, and to integrate these objective signs with two subjective symptoms (spontaneous retrobulbar pain and pain on attempted upward or downward gaze) in order to assess TAO activity.

Results

The proposed model achieved 0.906 accuracy, 0.833 specificity, 0.906 precision, 0.906 recall, and 0.906 F1-score in active TAO diagnosis, demonstrating advanced performance in predicting CAS and TAO activity signs compared to conventional single-view unimodal approaches. The integration of multiple views and modalities, encompassing both anterior segment slit-lamp photographs and facial images, significantly improved the prediction accuracy of the model for TAO activity and CAS.

Conclusion

The ensemble multi-view multimodal deep learning system developed in this study can more accurately assess the clinical activity of TAO than traditional methods that solely rely on facial images. This innovative approach is intended to enhance the efficiency of TAO activity assessment, providing a novel means for its comprehensive, early, and precise evaluation.

S, Kochar P, Selvaraj S, Levine H, Amescua G, Sepulveda-Beltran PA, Niziol LM, Woodward MA. Automatic Classification of Slit-Lamp Photographs by Imaging Illumination

PURPOSE

The aim of this study was to facilitate deep learning systems in image annotations for diagnosing keratitis type by developing an automated algorithm to classify slit-lamp photographs (SLPs) based on illumination technique.

METHODS

SLPs were collected from patients with corneal ulcer at Kellogg Eye Center, Bascom Palmer Eye Institute, and Aravind Eye Care Systems. Illumination techniques were slit beam, diffuse white light, diffuse blue light with fluorescein, and sclerotic scatter (ScS). Images were manually labeled for illumination and randomly split into training, validation, and testing data sets (70%:15%:15%). Classification algorithms including MobileNetV2, ResNet50, LeNet, AlexNet, multilayer perceptron, and k-nearest neighborhood were trained to distinguish 4 type of illumination techniques. The algorithm performances on the test data set were evaluated with 95% confidence intervals (CIs) for accuracy, F1 score, and area under the receiver operator characteristics curve (AUC-ROC), overall and by class (one-vs-rest).

RESULTS

A total of 12,132 images from 409 patients were analyzed, including 41.8% (n = 5069) slit-beam photographs, 21.2% (2571) diffuse white light, 19.5% (2364) diffuse blue light, and 17.5% (2128) ScS. MobileNetV2 achieved the highest overall F1 score of 97.95% (CI, 97.94%-97.97%), AUC-ROC of 99.83% (99.72%-99.9%), and accuracy of 98.98% (98.97%-98.98%). The F1 scores for slit beam, diffuse white light, diffuse blue light, and ScS were 97.82% (97.80%-97.84%), 96.62% (96.58%-96.66%), 99.88% (99.87%-99.89%), and 97.59% (97.55%-97.62%), respectively. Slit beam and ScS were the 2 most frequently misclassified illumination.

CONCLUSIONS

MobileNetV2 accurately labeled illumination of SLPs using a large data set of corneal images. Effective, automatic classification of SLPs is key to integrating deep learning systems for clinical decision support into practice workflows.

Trustworthy in silico cell labeling via ensemble-based image translation

Artificial intelligence (AI) image translation has been a valuable tool for processing image data in biological and medical research. To apply such a tool in mission-critical applications, including drug screening, toxicity study, and clinical diagnostics, it is essential to ensure that the AI prediction is trustworthy. Here, we demonstrate that an ensemble learning method can quantify the uncertainty of AI image translation. We tested the uncertainty evaluation using experimentally acquired images of mesenchymal stromal cells. We find that the ensemble method reports a prediction standard deviation that correlates with the prediction error, estimating the prediction uncertainty. We show that this uncertainty is in agreement with the prediction error and Pearson correlation coefficient. We further show that the ensemble method can detect out-of-distribution input images by reporting increased uncertainty. Altogether, these results suggest that the ensemble-estimated uncertainty can be a useful indicator for identifying erroneous AI image translations.

Artificial intelligence in ophthalmology: The path to the real-world clinic

Artificial intelligence (AI) has great potential to transform healthcare by enhancing the workflow and productivity of clinicians, enabling existing staff to serve more patients, improving patient outcomes, and reducing health disparities. In the field of ophthalmology, AI systems have shown performance comparable with or even better than experienced ophthalmologists in tasks such as diabetic retinopathy detection and grading. However, despite these quite good results, very few AI systems have been deployed in real-world clinical settings, challenging the true value of these systems. This review provides an overview of the current main AI applications in ophthalmology, describes the challenges that need to be overcome prior to clinical implementation of the AI systems, and discusses the strategies that may pave the way to the clinical translation of these systems.

Comparisons of artificial intelligence algorithms in automatic segmentation for fungal keratitis diagnosis by anterior segment images

Purpose

This study combines automatic segmentation and manual fine-tuning with an early fusion method to provide efficient clinical auxiliary diagnostic efficiency for fungal keratitis.

Methods

First, 423 high-quality anterior segment images of keratitis were collected in the Department of Ophthalmology of the Jiangxi Provincial People's Hospital (China). The images were divided into fungal keratitis and non-fungal keratitis by a senior ophthalmologist, and all images were divided randomly into training and testing sets at a ratio of 8:2. Then, two deep learning models were constructed for diagnosing fungal keratitis. Model 1 included a deep learning model composed of the DenseNet 121, mobienet_v2, and squeezentet1_0 models, the least absolute shrinkage and selection operator (LASSO) model, and the multi-layer perception (MLP) classifier. Model 2 included an automatic segmentation program and the deep learning model already described. Finally, the performance of Model 1 and Model 2 was compared.

Results

In the testing set, the accuracy, sensitivity, specificity, F1-score, and the area under the receiver operating characteristic (ROC) curve (AUC) of Model 1 reached 77.65, 86.05, 76.19, 81.42%, and 0.839, respectively. For Model 2, accuracy improved by 6.87%, sensitivity by 4.43%, specificity by 9.52%, F1-score by 7.38%, and AUC by 0.086, respectively.

Conclusion

The models in our study could provide efficient clinical auxiliary diagnostic efficiency for fungal keratitis.

Evaluation of a computer-aided diagnostic model for corneal diseases by analyzing in vivo confocal microscopy images

Objective

In order to automatically and rapidly recognize the layers of corneal images using in vivo confocal microscopy (IVCM) and classify them into normal and abnormal images, a computer-aided diagnostic model was developed and tested based on deep learning to reduce physicians' workload.

Methods

A total of 19,612 corneal images were retrospectively collected from 423 patients who underwent IVCM between January 2021 and August 2022 from Renmin Hospital of Wuhan University (Wuhan, China) and Zhongnan Hospital of Wuhan University (Wuhan, China). Images were then reviewed and categorized by three corneal specialists before training and testing the models, including the layer recognition model (epithelium, bowman's membrane, stroma, and endothelium) and diagnostic model, to identify the layers of corneal images and distinguish normal images from abnormal images. Totally, 580 database-independent IVCM images were used in a human-machine competition to assess the speed and accuracy of image recognition by 4 ophthalmologists and artificial intelligence (AI). To evaluate the efficacy of the model, 8 trainees were employed to recognize these 580 images both with and without model assistance, and the results of the two evaluations were analyzed to explore the effects of model assistance.

Results

The accuracy of the model reached 0.914, 0.957, 0.967, and 0.950 for the recognition of 4 layers of epithelium, bowman's membrane, stroma, and endothelium in the internal test dataset, respectively, and it was 0.961, 0.932, 0.945, and 0.959 for the recognition of normal/abnormal images at each layer, respectively. In the external test dataset, the accuracy of the recognition of corneal layers was 0.960, 0.965, 0.966, and 0.964, respectively, and the accuracy of normal/abnormal image recognition was 0.983, 0.972, 0.940, and 0.982, respectively. In the human-machine competition, the model achieved an accuracy of 0.929, which was similar to that of specialists and higher than that of senior physicians, and the recognition speed was 237 times faster than that of specialists. With model assistance, the accuracy of trainees increased from 0.712 to 0.886.

Conclusion

A computer-aided diagnostic model was developed for IVCM images based on deep learning, which rapidly recognized the layers of corneal images and classified them as normal and abnormal. This model can increase the efficacy of clinical diagnosis and assist physicians in training and learning for clinical purposes.

Comparison of deep learning systems and cornea specialists in detecting corneal diseases from low-quality images

The performance of deep learning in disease detection from high-quality clinical images is identical to and even greater than that of human doctors. However, in low-quality images, deep learning performs poorly. Whether human doctors also have poor performance in low-quality images is unknown. Here, we compared the performance of deep learning systems with that of cornea specialists in detecting corneal diseases from low-quality slit lamp images. The results showed that the cornea specialists performed better than our previously established deep learning system (PEDLS) trained on only high-quality images. The performance of the system trained on both high- and low-quality images was superior to that of the PEDLS while inferior to that of a senior corneal specialist. This study highlights that cornea specialists perform better in low-quality images than the system trained on high-quality images. Adding low-quality images with sufficient diagnostic certainty to the training set can reduce this performance gap.

•

Deep learning performs poorly in low-quality images for detecting corneal diseases

•

Corneal specialists perform better than the PEDLS in low-quality images

•

The performance of the NDLS is better than that of the PEDLS in low-quality images

•

Adding low-quality images to the training set can improve the system's performance

Deep learning approach for guiding three-dimensional computed tomography reconstruction of lower limbs for robotically-assisted total knee arthroplasty

BACKGROUND

Robotic-assisted total knee arthroplasty (TKA) was performed to promote the accuracy of bone resection and mechanical alignment. Among these TKA system procedures, 3D reconstruction of CT data of lower limbs consumes significant manpower. Artificial intelligence (AI) algorithms applying deep learning has been proved efficient in automated identification and visual processing.

METHODS

CT data of a total of 200 lower limbs scanning were used for AI-based 3D model construction and CT data of 20 lower limbs scanning were utilised for verification.

RESULTS

We showed that the performance of an AI-guided 3D reconstruction of CT data of lower limbs for robotic-assisted TKA was similar to that of the operator-based approach. The time of 3D lower limb model construction using AI was 4.7 min. AI-based 3D models can be used for surgical planning.

CONCLUSION

AI was used for the first time to guide the 3D reconstruction of CT data of lower limbs for facilitating robotic-assisted TKA. Incorporation of AI in 3D model reconstruction before TKA might reduce the workload of radiologists.