Segmentation of Corneal Nerves Using a U-Net-Based Convolutional Neural Network

Segmentation and Classification Approaches of Clinically Relevant Curvilinear Structures: A Review

Detection of curvilinear structures from microscopic images, which help the clinicians to make an unambiguous diagnosis is assuming paramount importance in recent clinical practice. Appearance and size of dermatophytic hyphae, keratitic fungi, corneal and retinal vessels vary widely making their automated detection cumbersome. Automated deep learning methods, endowed with superior self-learning capacity, have superseded the traditional machine learning methods, especially in complex images with challenging background. Automatic feature learning ability using large input data with better generalization and recognition capability, but devoid of human interference and excessive pre-processing, is highly beneficial in the above context. Varied attempts have been made by researchers to overcome challenges such as thin vessels, bifurcations and obstructive lesions in retinal vessel detection as revealed through several publications reviewed here. Revelations of diabetic neuropathic complications such as tortuosity, changes in the density and angles of the corneal fibers have been successfully sorted in many publications reviewed here. Since artifacts complicate the images and affect the quality of analysis, methods addressing these challenges have been described. Traditional and deep learning methods, that have been adapted and published between 2015 and 2021 covering retinal vessels, corneal nerves and filamentous fungi have been summarized in this review. We find several novel and meritorious ideas and techniques being put to use in the case of retinal vessel segmentation and classification, which by way of cross-domain adaptation can be utilized in the case of corneal and filamentous fungi also, making suitable adaptations to the challenges to be addressed.

Centerline extraction by neighborhood-statistics thinning for quantitative analysis of corneal nerve fibers

Objective. Corneal nerve fiber (CNF) has been found to exhibit morphological changes associated with various diseases, which can therefore be utilized to aid in the early diagnosis of those diseases. CNF is usually visualized under corneal confocal microscopy (CCM) in clinic. To obtain the diagnostic biomarkers from CNF image produced from CCM, image processing and quantitative analysis are needed. Usually, CNF is segmented first and then CNF’s centerline is extracted, allowing for measuring geometrical and topological biomarkers of CNF, such as density, tortuosity, and length. Consequently, the accuracy of the segmentation and centerline extraction can make a big impact on the biomarker measurement. Thus, this study is aimed to improve the accuracy and universality of centerline extraction. Approach. We developed a new thinning algorithm based on neighborhood statistics, called neighborhood-statistics thinning (NST), to extract the centerline of CNF. Compared with traditional thinning and skeletonization techniques, NST exhibits a better capability to preserve the fine structure of CNF which can effectively benefit the biomarkers measurement above. Moreover, NST incorporates a fitting process, which can make centerline extraction be less influenced by image segmentation. Main results. This new method is evaluated on three datasets which are segmented with five different deep learning networks. The results show that NST is superior to thinning and skeletonization on all the CNF-segmented datasets with a precision rate above 0.82. Last, NST is attempted to be applied for the diagnosis of keratitis with the quantitative biomarkers measured from the extracted centerlines. Longer length and higher density but lower tortuosity were found on the CNF of keratitis patients as compared to healthy patients. Significance. This demonstrates that NST has a good potential to aid in the diagnostics of eye diseases in clinic.

Segmentation and Evaluation of Corneal Nerves and Dendritic Cells From In Vivo Confocal Microscopy Images Using Deep Learning

Purpose

Segmentation and evaluation of in vivo confocal microscopy (IVCM) images requires manual intervention, which is time consuming, laborious, and non-reproducible. The aim of this research was to develop and validate deep learning–based methods that could automatically segment and evaluate corneal nerve fibers (CNFs) and dendritic cells (DCs) in IVCM images, thereby reducing processing time to analyze larger volumes of clinical images.

Methods

CNF and DC segmentation models were developed based on U-Net and Mask R-CNN architectures, respectively; 10-fold cross-validation was used to evaluate both models. The CNF model was trained and tested using 1097 and 122 images, and the DC model was trained and tested using 679 and 75 images, respectively, at each fold. The CNF morphology, number of nerves, number of branching points, nerve length, and tortuosity were analyzed; for DCs, number, size, and immature–mature cells were analyzed. Python-based software was written for model training, testing, and automatic morphometric parameters evaluation.

Results

The CNF model achieved on average 86.1% sensitivity and 90.1% specificity, and the DC model achieved on average 89.37% precision, 94.43% recall, and 91.83% F₁ score. The interclass correlation coefficient (ICC) between manual annotation and automatic segmentation were 0.85, 0.87, 0.95, and 0.88 for CNF number, length, branching points, and tortuosity, respectively, and the ICC for DC number and size were 0.95 and 0.92, respectively.

Conclusions

Our proposed methods demonstrated reliable consistency between manual annotation and automatic segmentation of CNF and DC with rapid speed. The results showed that these approaches have the potential to be implemented into clinical practice in IVCM images.

Translational Relevance

The deep learning–based automatic segmentation and quantification algorithm significantly increases the efficiency of evaluating IVCM images, thereby supporting and potentially improving the diagnosis and treatment of ocular surface disease associated with corneal nerves and dendritic cells.

SS-net: split and spatial attention network for vessel segmentation of retinal OCT angiography

Optical coherence tomography angiography (OCTA) has been widely used in clinical fields because of its noninvasive, high-resolution qualities. Accurate vessel segmentation on OCTA images plays an important role in disease diagnosis. Most deep learning methods are based on region segmentation, which may lead to inaccurate segmentation for the extremely complex curve structure of retinal vessels. We propose a U-shaped network called SS-Net that is based on the attention mechanism to solve the problem of continuous segmentation of discontinuous vessels of a retinal OCTA. In this SS-Net, the improved SRes Block combines the residual structure and split attention to prevent the disappearance of gradient and gives greater weight to capillary features to form a backbone with an encoder and decoder architecture. In addition, spatial attention is applied to extract key information from spatial dimensions. To enhance the credibility, we use several indicators to evaluate the function of the SS-Net. In two datasets, the important indicators of accuracy reach 0.9258/0.9377, respectively, and a Dice coefficient is achieved, with an improvement of around 3% compared to state-of-the-art models in segmentation.

Artificial intelligence utilising corneal confocal microscopy for the diagnosis of peripheral neuropathy in diabetes mellitus and prediabetes

AIMS/HYPOTHESIS

We aimed to develop an artificial intelligence (AI)-based deep learning algorithm (DLA) applying attribution methods without image segmentation to corneal confocal microscopy images and to accurately classify peripheral neuropathy (or lack of).

METHODS

The AI-based DLA utilised convolutional neural networks with data augmentation to increase the algorithm's generalisability. The algorithm was trained using a high-end graphics processor for 300 epochs on 329 corneal nerve images and tested on 40 images (1 image/participant). Participants consisted of healthy volunteer (HV) participants (n = 90) and participants with type 1 diabetes (n = 88), type 2 diabetes (n = 141) and prediabetes (n = 50) (defined as impaired fasting glucose, impaired glucose tolerance or a combination of both), and were classified into HV, those without neuropathy (PN-) (n = 149) and those with neuropathy (PN+) (n = 130). For the AI-based DLA, a modified residual neural network called ResNet-50 was developed and used to extract features from images and perform classification. The algorithm was tested on 40 participants (15 HV, 13 PN-, 12 PN+). Attribution methods gradient-weighted class activation mapping (Grad-CAM), Guided Grad-CAM and occlusion sensitivity displayed the areas within the image that had the greatest impact on the decision of the algorithm.

RESULTS

The results were as follows: HV: recall of 1.0 (95% CI 1.0, 1.0), precision of 0.83 (95% CI 0.65, 1.0), F₁-score of 0.91 (95% CI 0.79, 1.0); PN-: recall of 0.85 (95% CI 0.62, 1.0), precision of 0.92 (95% CI 0.73, 1.0), F₁-score of 0.88 (95% CI 0.71, 1.0); PN+: recall of 0.83 (95% CI 0.58, 1.0), precision of 1.0 (95% CI 1.0, 1.0), F₁-score of 0.91 (95% CI 0.74, 1.0). The features displayed by the attribution methods demonstrated more corneal nerves in HV, a reduction in corneal nerves for PN- and an absence of corneal nerves for PN+ images.

CONCLUSIONS/INTERPRETATION

We demonstrate promising results in the rapid classification of peripheral neuropathy using a single corneal image. A large-scale multicentre validation study is required to assess the utility of AI-based DLA in screening and diagnostic programmes for diabetic neuropathy.

Corneal and Epidermal Nerve Quantification in Chemotherapy Induced Peripheral Neuropathy

Chemotherapy-induced neurotoxicity is an increasingly recognized clinical issue in oncology. in vivo confocal microscopy (IVCM) of corneal nerves has been successfully used to diagnose peripheral neuropathies, including diabetic neuropathy. The purpose of this study was to test if the combination of corneal nerve density and morphology assessed by IVCM is useful to monitor the neurotoxic effects of chemotherapy compared to epidermal nerve quantification. Overall, 95 adult patients with different cancer types were recruited from the oncology and hematology departments of the San Raffaele Hospital. Neurological examination, including clinical Total Neuropathy Score, and in vivo corneal confocal microscopy (IVCM), were performed before and after chemotherapy. In a group of 14 patients, skin biopsy was performed at the first and last visit. In the group of 14 patients who underwent both skin biopsy and corneal nerve imaging, clinical worsening (+69%, p = 0.0018) was paralleled by corneal nerve fiber (CNF) density reduction (−22%, p = 0.0457). Clinical Total neuropathy score significantly worsened from the first to the last visit (+62%, p < 0.0001). CNF length was not significantly reduced overall. However, CNF density/tortuosity ratio significantly decreased after therapy. Correlation analysis showed that the CNF density/tortuosity ratio was also correlated with the number of chemotherapy cycles (r = −0.04790, P = 0.0009). Our data confirm that in vivo corneal confocal microscopy is a helpful, non-invasive tool which shows promise for the diagnosis of chemotherapy-induced peripheral neuropathies. IVCM could allow a rapid, reproducible and non-invasive quantification of peripheral nerve pathology in chemotherapy-associated neuropathy.

Deep learning-based analysis of macaque corneal sub-basal nerve fibers in confocal microscopy images

BACKGROUND

To develop and validate a deep learning-based approach to the fully-automated analysis of macaque corneal sub-basal nerves using in vivo confocal microscopy (IVCM).

METHODS

IVCM was used to collect 108 images from 35 macaques. 58 of the images from 22 macaques were used to evaluate different deep convolutional neural network (CNN) architectures for the automatic analysis of sub-basal nerves relative to manual tracings. The remaining images were used to independently assess correlations and inter-observer performance relative to three readers.

RESULTS

Correlation scores using the coefficient of determination between readers and the best CNN averaged 0.80. For inter-observer comparison, inter-correlation coefficients (ICCs) between the three expert readers and the automated approach were 0.75, 0.85 and 0.92. The ICC between all four observers was 0.84, the same as the average between the CNN and individual readers.

CONCLUSIONS

Deep learning-based segmentation of sub-basal nerves in IVCM images shows high to very high correlation to manual segmentations in macaque data and is indistinguishable across readers. As quantitative measurements of corneal sub-basal nerves are important biomarkers for disease screening and management, the reported work offers utility to a variety of research and clinical studies using IVCM.