Retinal vessel centerline extraction using multiscale matched filters, confidence and edge measures

Opening a window to skin biomarkers for diabetes stage with optoacoustic mesoscopy

Being the largest and most accessible organ of the human body, the skin could offer a window to diabetes-related complications on the microvasculature. However, skin microvasculature is typically assessed by histological analysis, which is not suited for applications to large populations or longitudinal studies. We introduce ultra-wideband raster-scan optoacoustic mesoscopy (RSOM) for precise, non-invasive assessment of diabetes-related changes in the dermal microvasculature and skin micro-anatomy, resolved with unprecedented sensitivity and detail without the need for contrast agents. Providing unique imaging contrast, we explored a possible role for RSOM as an investigational tool in diabetes healthcare and offer the first comprehensive study investigating the relationship between different diabetes complications and microvascular features in vivo. We applied RSOM to scan the pretibial area of 95 participants with diabetes mellitus and 48 age-matched volunteers without diabetes, grouped according to disease complications, and extracted six label-free optoacoustic biomarkers of human skin, including dermal microvasculature density and epidermal parameters, based on a novel image-processing pipeline. We then correlated these biomarkers to disease severity and found statistically significant effects on microvasculature parameters as a function of diabetes complications. We discuss how label-free RSOM biomarkers can lead to a quantitative assessment of the systemic effects of diabetes and its complications, complementing the qualitative assessment allowed by current clinical metrics, possibly leading to a precise scoring system that captures the gradual evolution of the disease.

A lightweight network guided with differential matched filtering for retinal vessel segmentation

The geometric morphology of retinal vessels reflects the state of cardiovascular health, and fundus images are important reference materials for ophthalmologists. Great progress has been made in automated vessel segmentation, but few studies have focused on thin vessel breakage and false-positives in areas with lesions or low contrast. In this work, we propose a new network, differential matched filtering guided attention UNet (DMF-AU), to address these issues, incorporating a differential matched filtering layer, feature anisotropic attention, and a multiscale consistency constrained backbone to perform thin vessel segmentation. The differential matched filtering is used for the early identification of locally linear vessels, and the resulting rough vessel map guides the backbone to learn vascular details. Feature anisotropic attention reinforces the vessel features of spatial linearity at each stage of the model. Multiscale constraints reduce the loss of vessel information while pooling within large receptive fields. In tests on multiple classical datasets, the proposed model performed well compared with other algorithms on several specially designed criteria for vessel segmentation. DMF-AU is a high-performance, lightweight vessel segmentation model. The source code is at https://github.com/tyb311/DMF-AU.

Neuron tracing from light microscopy images: automation, deep learning and bench testing

MOTIVATION

Large-scale neuronal morphologies are essential to neuronal typing, connectivity characterization and brain modeling. It is widely accepted that automation is critical to the production of neuronal morphology. Despite previous survey papers about neuron tracing from light microscopy data in the last decade, thanks to the rapid development of the field, there is a need to update recent progress in a review focusing on new methods and remarkable applications.

RESULTS

This review outlines neuron tracing in various scenarios with the goal to help the community understand and navigate tools and resources. We describe the status, examples and accessibility of automatic neuron tracing. We survey recent advances of the increasingly popular deep-learning enhanced methods. We highlight the semi-automatic methods for single neuron tracing of mammalian whole brains as well as the resulting datasets, each containing thousands of full neuron morphologies. Finally, we exemplify the commonly used datasets and metrics for neuron tracing bench testing.

Deep Learning of the Retina Enables Phenome- and Genome-Wide Analyses of the Microvasculature

BACKGROUND

The microvasculature, the smallest blood vessels in the body, has key roles in maintenance of organ health and tumorigenesis. The retinal fundus is a window for human in vivo noninvasive assessment of the microvasculature. Large-scale complementary machine learning-based assessment of the retinal vasculature with phenome-wide and genome-wide analyses may yield new insights into human health and disease.

METHODS

We used 97 895 retinal fundus images from 54 813 UK Biobank participants. Using convolutional neural networks to segment the retinal microvasculature, we calculated vascular density and fractal dimension as a measure of vascular branching complexity. We associated these indices with 1866 incident International Classification of Diseases-based conditions (median 10-year follow-up) and 88 quantitative traits, adjusting for age, sex, smoking status, and ethnicity.

RESULTS

Low retinal vascular fractal dimension and density were significantly associated with higher risks for incident mortality, hypertension, congestive heart failure, renal failure, type 2 diabetes, sleep apnea, anemia, and multiple ocular conditions, as well as corresponding quantitative traits. Genome-wide association of vascular fractal dimension and density identified 7 and 13 novel loci, respectively, that were enriched for pathways linked to angiogenesis (eg, vascular endothelial growth factor, platelet-derived growth factor receptor, angiopoietin, and WNT signaling pathways) and inflammation (eg, interleukin, cytokine signaling).

CONCLUSIONS

Our results indicate that the retinal vasculature may serve as a biomarker for future cardiometabolic and ocular disease and provide insights into genes and biological pathways influencing microvascular indices. Moreover, such a framework highlights how deep learning of images can quantify an interpretable phenotype for integration with electronic health record, biomarker, and genetic data to inform risk prediction and risk modification.

Computational modeling of PET tracer distribution in solid tumors integrating microvasculature

BACKGROUND

We present computational modeling of positron emission tomography radiotracer uptake with consideration of blood flow and interstitial fluid flow, performing spatiotemporally-coupled modeling of uptake and integrating the microvasculature. In our mathematical modeling, the uptake of fluorodeoxyglucose F-18 (FDG) was simulated based on the Convection-Diffusion-Reaction equation given its high accuracy and reliability in modeling of transport phenomena. In the proposed model, blood flow and interstitial flow are solved simultaneously to calculate interstitial pressure and velocity distribution inside cancer and normal tissues. As a result, the spatiotemporal distribution of the FDG tracer is calculated based on velocity and pressure distributions in both kinds of tissues.

RESULTS

Interstitial pressure has maximum value in the tumor region compared to surrounding tissue. In addition, interstitial fluid velocity is extremely low in the entire computational domain indicating that convection can be neglected without effecting results noticeably. Furthermore, our results illustrate that the total concentration of FDG in the tumor region is an order of magnitude larger than in surrounding normal tissue, due to lack of functional lymphatic drainage system and also highly-permeable microvessels in tumors. The magnitude of the free tracer and metabolized (phosphorylated) radiotracer concentrations followed very different trends over the entire time period, regardless of tissue type (tumor vs. normal).

CONCLUSION

Our spatiotemporally-coupled modeling provides helpful tools towards improved understanding and quantification of in vivo preclinical and clinical studies.

Segmentation and Automatic Identification of Vasculature in Coronary Angiograms

Coronary angiography is the “gold standard” for the diagnosis of coronary heart disease, of which vessel segmentation and identification technologies are paid much attention to. However, because of the characteristics of coronary angiograms, such as the complex and variable morphology of coronary artery structure and the noise caused by various factors, there are many difficulties in these studies. To conquer these problems, we design a preprocessing scheme including block-matching and 3D filtering, unsharp masking, contrast-limited adaptive histogram equalization, and multiscale image enhancement to improve the quality of the image and enhance the vascular structure. To achieve vessel segmentation, we use the C-V model to extract the vascular contour. Finally, we propose an improved adaptive tracking algorithm to realize automatic identification of the vascular skeleton. According to our experiments, the vascular structures can be successfully highlighted and the background is restrained by the preprocessing scheme, the continuous contour of the vessel is extracted accurately by the C-V model, and it is verified that the proposed tracking method has higher accuracy and stronger robustness compared with the existing adaptive tracking method.

HDC-Net: A hierarchical dilation convolutional network for retinal vessel segmentation

The cardinal symptoms of some ophthalmic diseases observed through exceptional retinal blood vessels, such as retinal vein occlusion, diabetic retinopathy, etc. The advanced deep learning models used to obtain morphological and structural information of blood vessels automatically are conducive to the early treatment and initiative prevention of ophthalmic diseases. In our work, we propose a hierarchical dilation convolutional network (HDC-Net) to extract retinal vessels in a pixel-to-pixel manner. It utilizes the hierarchical dilation convolution (HDC) module to capture the fragile retinal blood vessels usually neglected by other methods. An improved residual dual efficient channel attention (RDECA) module can infer more delicate channel information to reinforce the discriminative capability of the model. The structured Dropblock can help our HDC-Net model to solve the network overfitting effectively. From a holistic perspective, the segmentation results obtained by HDC-Net are superior to other deep learning methods on three acknowledged datasets (DRIVE, CHASE-DB1, STARE), the sensitivity, specificity, accuracy, f1-score and AUC score are {0.8252, 0.9829, 0.9692, 0.8239, 0.9871}, {0.8227, 0.9853, 0.9745, 0.8113, 0.9884}, and {0.8369, 0.9866, 0.9751, 0.8385, 0.9913}, respectively. It surpasses most other advanced retinal vessel segmentation models. Qualitative and quantitative analysis demonstrates that HDC-Net can fulfill the task of retinal vessel segmentation efficiently and accurately.

Joint Extraction of Retinal Vessels and Centerlines Based on Deep Semantics and Multi-Scaled Cross-Task Aggregation

Retinal vessel segmentation and centerline extraction are crucial steps in building a computer-aided diagnosis system on retinal images. Previous works treat them as two isolated tasks, while ignoring their tight association. In this paper, we propose a deep semantics and multi-scaled cross-task aggregation network that takes advantage of the association to jointly improve their performances. Our network is featured by two sub-networks. The forepart is a deep semantics aggregation sub-network that aggregates strong semantic information to produce more powerful features for both tasks, and the tail is a multi-scaled cross-task aggregation sub-network that explores complementary information to refine the results. We evaluate the proposed method on three public databases, which are DRIVE, STARE and CHASE_DB1. Experimental results show that our method can not only simultaneously extract retinal vessels and their centerlines but also achieve the state-of-the-art performances on both tasks.

Ordered multi-path propagation for vessel centerline extraction

Vessel centerline extraction from x-ray angiography images is essential for vessel structure analysis in the diagnosis of coronary artery disease. However, complete and continuous centerline extraction remains a challenging task due to image noise, poor contrast, and complexity of vessel structure. Thus, an iterative multi-path search framework for automatic vessel centerline extraction is proposed. First, the seed points of the vessel structure are detected and sorted by confidence. With the ordered seed points, multi-bifurcation centerline is searched through multi-path propagation of wavefront and accumulated voting. Finally, the centerline is further extended piecewise by wavefront propagation on the basis of keypoint detection. The latter two steps are performed alternately to obtain the final centerline result. The proposed method is qualitatively and quantitatively evaluated on 1260 synthetic images and 50 clinical angiography images. The results demonstrate that our method has a highF1score of 87.8% ± 2.7% for the angiography images and achieves accurate and continuous results of vessel centerline extraction.

A review of machine learning methods for retinal blood vessel segmentation and artery/vein classification

The eye affords a unique opportunity to inspect a rich part of the human microvasculature non-invasively via retinal imaging. Retinal blood vessel segmentation and classification are prime steps for the diagnosis and risk assessment of microvascular and systemic diseases. A high volume of techniques based on deep learning have been published in recent years. In this context, we review 158 papers published between 2012 and 2020, focussing on methods based on machine and deep learning (DL) for automatic vessel segmentation and classification for fundus camera images. We divide the methods into various classes by task (segmentation or artery-vein classification), technique (supervised or unsupervised, deep and non-deep learning, hand-crafted methods) and more specific algorithms (e.g. multiscale, morphology). We discuss advantages and limitations, and include tables summarising results at-a-glance. Finally, we attempt to assess the quantitative merit of DL methods in terms of accuracy improvement compared to other methods. The results allow us to offer our views on the outlook for vessel segmentation and classification for fundus camera images.

Visible Vessels of Vocal Folds: Can they have a Diagnostic Role?

BACKGROUND

Challenges in visual identification of laryngeal disorders lead researchers to investigate new opportunities to help clinical examination. This paper presents an efficient and simple method which extracts and assesses blood vessels on vocal fold tissue in order to serve medical diagnosis.

METHODS

The proposed vessel segmentation approach has been designed in order to overcome difficulties raised by design specifications of videolaryngostroboscopy and anatomic structure of vocal fold vasculature. The limited number of medical studies on vocal fold vasculature point out that the direction of blood vessels and amount of vasculature are discriminative features for vocal fold disorders. Therefore, we extracted the features of vessels on the basis of these studies. We represent vessels as vascular vectors and suggest a vector field based measurement that quantifies the orientation pattern of blood vessels towards vocal fold pathologies.

RESULTS

In order to demonstrate the relationship between vessel structure and vocal fold disorders, we performed classification of vocal fold disorders by using only vessel features. A binary tree of Support Vector Machine (SVM) has been exploited for classification. Average recall of proposed vessel extraction method was calculated as 0.82 while healthy, sulcus vocalis, laryngitis classification accuracy of 0.75 was achieved.

CONCLUSION

Obtained success rates showed the efficiency of vocal fold vessels in serving as an indicator of laryngeal diseases.

G, Naik G. A supervised blood vessel segmentation technique for digital Fundus images using Zernike Moment based features

This paper proposes a new supervised method for blood vessel segmentation using Zernike moment-based shape descriptors. The method implements a pixel wise classification by computing a 11-D feature vector comprising of both statistical (gray-level) features and shape-based (Zernike moment) features. Also the feature set contains optimal coefficients of the Zernike Moments which were derived based on the maximum differentiability between the blood vessel and background pixels. A manually selected training points obtained from the training set of the DRIVE dataset, covering all possible manifestations were used for training the ANN-based binary classifier. The method was evaluated on unknown test samples of DRIVE and STARE databases and returned accuracies of 0.945 and 0.9486 respectively, outperforming other existing supervised learning methods. Further, the segmented outputs were able to cover thinner blood vessels better than previous methods, aiding in early detection of pathologies.

A Multi-Scale Directional Line Detector for Retinal Vessel Segmentation

The assessment of transformations in the retinal vascular structure has a strong potential in indicating a wide range of underlying ocular pathologies. Correctly identifying the retinal vessel map is a crucial step in disease identification, severity progression assessment, and appropriate treatment. Marking the vessels manually by a human expert is a tedious and time-consuming task, thereby reinforcing the need for automated algorithms capable of quick segmentation of retinal features and any possible anomalies. Techniques based on unsupervised learning methods utilize vessel morphology to classify vessel pixels. This study proposes a directional multi-scale line detector technique for the segmentation of retinal vessels with the prime focus on the tiny vessels that are most difficult to segment out. Constructing a directional line-detector, and using it on images having only the features oriented along the detector's direction, significantly improves the detection accuracy of the algorithm. The finishing step involves a binarization operation, which is again directional in nature, helps in achieving further performance improvements in terms of key performance indicators. The proposed method is observed to obtain a sensitivity of 0.8043, 0.8011, and 0.7974 for the Digital Retinal Images for Vessel Extraction (DRIVE), STructured Analysis of the Retina (STARE), and Child Heart And health Study in England (CHASE_DB1) datasets, respectively. These results, along with other performance enhancements demonstrated by the conducted experimental evaluation, establish the validity and applicability of directional multi-scale line detectors as a competitive framework for retinal image segmentation.

Ex vivo evaluation of an atherosclerotic human coronary artery via histology and high-resolution hard X-ray tomography

Atherosclerotic arteries exhibit characteristic constrictions and substantial deviations from cylindrical shape. Therefore, determining the artery's cross-section along the centerline is challenging, although high-resolution isotropic three-dimensional data are available. Herein, we apply high-resolution computed tomography in absorption and phase to a plaque-containing human artery post-mortem, through the course of the preparation stages for histology. We identify the impact of paraffin embedding and decalcification on the artery lumen. For automatic extraction of lumen's cross-section along centerline we present a dedicated pipeline. Comparing fixated tissue before and after paraffin embedding gives rise to shape changes with lumen reduction to 50-80%. The histological slicing induces further deformations with respect to tomography. Data acquired after decalcification show debris unintentionally distributed within the vessel preventing the reliable automatic lumen segmentation. Comparing tomography of laboratory- and synchrotron-radiation-based X rays by means of joint histogram analysis leads us to conclude that advanced desktop tomography is capable of quantifying the artery's lumen as an essential input for blood flow simulations. The results indicate that the most reliable lumen quantification is achieved by imaging the non-decalcified specimen fixed in formalin, using phase contrast modality and a dedicated processing pipeline. This study focusses on a methodology to quantitatively evaluate diseased artery segments post-mortem and provides unique structural parameters on the treatment-induced local shrinkage, which will be the basis of future studies on the flow in vessels affected by constrictions.

Scale-space approximated convolutional neural networks for retinal vessel segmentation

BACKGROUND AND OBJECTIVE

Retinal fundus images are widely used to diagnose retinal diseases and can potentially be used for early diagnosis and prevention of chronic vascular diseases and diabetes. While various automatic retinal vessel segmentation methods using deep learning have been proposed, they are mostly based on common CNN structures developed for other tasks such as classification.

METHODS

We present a novel and simple multi-scale convolutional neural network (CNN) structure for retinal vessel segmentation. We first provide a theoretical analysis of existing multi-scale structures based on signal processing. In previous structures, multi-scale representations are achieved through downsampling by subsampling and decimation. By incorporating scale-space theory, we propose a simple yet effective multi-scale structure for CNNs using upsampling, which we term scale-space approximated CNN (SSANet). Based on further analysis of the effects of the SSA structure within a CNN, we also incorporate residual blocks, resulting in a multi-scale CNN that outperforms current state-of-the-art methods.

RESULTS

Quantitative evaluations are presented as the area-under-curve (AUC) of the receiver operating characteristic (ROC) curve and the precision-recall curve, as well as accuracy, for four publicly available datasets, namely DRIVE, STARE, CHASE_DB1, and HRF. For the CHASE_DB1 set, the SSANet achieves state-of-the-art AUC value of 0.9916 for the ROC curve. An ablative analysis is presented to analyze the contribution of different components of the SSANet to the performance improvement.

CONCLUSIONS

The proposed retinal SSANet achieves state-of-the-art or comparable accuracy across publicly available datasets, especially improving segmentation for thin vessels, vessel junctions, and central vessel reflexes.

RetinaMatch: Efficient Template Matching of Retina Images for Teleophthalmology

Retinal template matching and registration is an important challenge in teleophthalmology with low-cost imaging devices. However, the images from such devices generally have a small field of view (FOV) and image quality degradations, making matching difficult. In this paper, we develop an efficient and accurate retinal matching technique that combines dimension reduction and mutual information (MI), called RetinaMatch. The dimension reduction initializes the MI optimization as a coarse localization process, which narrows the optimization domain and avoids local optima. The effectiveness of RetinaMatch is demonstrated on the open fundus image database STARE with simulated reduced FOV and anticipated degradations, and on retinal images acquired by adapter-based optics attached to a smartphone. RetinaMatch achieves a success rate over 94% on human retinal images with the matched target registration errors below 2 pixels on average, excluding the observer variability, outperforming standard template matching solutions. In the application of measuring vessel diameter repeatedly, single pixel errors are expected. In addition, our method can be used in the process of image mosaicking with area-based registration, providing a robust approach when feature-based methods fail. To the best of our knowledge, this is the first template matching algorithm for retina images with small template images from unconstrained retinal areas. In the context of the emerging mixed reality market, we envision automated retinal image matching and registration methods as transformative for advanced teleophthalmology and long-term retinal monitoring.

Multilevel and Multiscale Deep Neural Network for Retinal Blood Vessel Segmentation

Retinal blood vessel segmentation influences a lot of blood vessel-related disorders such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular disorders, etc. It is found that vessel segmentation using a convolutional neural network (CNN) showed increased accuracy in feature extraction and vessel segmentation compared to the classical segmentation algorithms. CNN does not need any artificial handcrafted features to train the network. In the proposed deep neural network (DNN), a better pre-processing technique and multilevel/multiscale deep supervision (DS) layers are being incorporated for proper segmentation of retinal blood vessels. From the first four layers of the VGG-16 model, multilevel/multiscale deep supervision layers are formed by convolving vessel-specific Gaussian convolutions with two different scale initializations. These layers output the activation maps that are capable to learn vessel-specific features at multiple scales, levels, and depth. Furthermore, the receptive field of these maps is increased to obtain the symmetric feature maps that provide the refined blood vessel probability map. This map is completely free from the optic disc, boundaries, and non-vessel background. The segmented results are tested on Digital Retinal Images for Vessel Extraction (DRIVE), STructured Analysis of the Retina (STARE), High-Resolution Fundus (HRF), and real-world retinal datasets to evaluate its performance. This proposed model achieves better sensitivity values of 0.8282, 0.8979 and 0.8655 in DRIVE, STARE and HRF datasets with acceptable specificity and accuracy performance metrics.

Content-Aware Enhancement of Images With Filamentous Structures

In this paper, we describe a novel enhancement method for images containing filamentous structures. Our method combines a gradient sparsity constraint with a filamentous structure constraint for the effective removal of clutter and noise from the background. The method is applied and evaluated on three types of data: 1) confocal microscopy images of neurons; 2) calcium imaging data; and 3) images of road pavement. We found that the images enhanced by our method preserve both the structure and the intensity details of the original object. In the case of neuron microscopy, we find that the neurons enhanced by our method are better correlated with the original structure intensities than the neurons enhanced by well-known vessel enhancement methods. Experiments on simulated calcium imaging data indicate that both the number of detected neurons and the accuracy of the derived calcium activity are improved. Applying our method to real calcium data, more regions exhibiting calcium activity in the full field of view were found. In road pavement crack detection, smaller or milder cracks were detected after using our enhancement method.

Artery-vein segmentation in fundus images using a fully convolutional network

Epidemiological studies demonstrate that dimensions of retinal vessels change with ocular diseases, coronary heart disease and stroke. Different metrics have been described to quantify these changes in fundus images, with arteriolar and venular calibers among the most widely used. The analysis often includes a manual procedure during which a trained grader differentiates between arterioles and venules. This step can be time-consuming and can introduce variability, especially when large volumes of images need to be analyzed. In light of the recent successes of fully convolutional networks (FCNs) applied to biomedical image segmentation, we assess its potential in the context of retinal artery-vein (A/V) discrimination. To the best of our knowledge, a deep learning (DL) architecture for simultaneous vessel extraction and A/V discrimination has not been previously employed. With the aim of improving the automation of vessel analysis, a novel application of the U-Net semantic segmentation architecture (based on FCNs) on the discrimination of arteries and veins in fundus images is presented. By utilizing DL, results are obtained that exceed accuracies reported in the literature. Our model was trained and tested on the public DRIVE and HRF datasets. For DRIVE, measuring performance on vessels wider than two pixels, the FCN achieved accuracies of 94.42% and 94.11% on arteries and veins, respectively. This represents a decrease in error of 25% over the previous state of the art reported by Xu et al. (2017). Additionally, we introduce the HRF A/V ground truth, on which our model achieves 96.98% accuracy on all discovered centerline pixels. HRF A/V ground truth validated by an ophthalmologist, predicted A/V annotations and evaluation code are available at https://github.com/rubenhx/av-segmentation.

Optimizing the trainable B-COSFIRE filter for retinal blood vessel segmentation

Segmentation of the retinal blood vessels using filtering techniques is a widely used step in the development of an automated system for diagnostic retinal image analysis. This paper optimized the blood vessel segmentation, by extending the trainable B-COSFIRE filter via identification of more optimal parameters. The filter parameters are introduced using an optimization procedure to three public datasets (STARE, DRIVE, and CHASE-DB1). The suggested approach considers analyzing thresholding parameters selection followed by application of background artifacts removal techniques. The approach results are better than the other state of the art methods used for vessel segmentation. ANOVA analysis technique is also used to identify the most significant parameters that are impacting the performance results (p-value ¡ 0.05). The proposed enhancement has improved the vessel segmentation accuracy in DRIVE, STARE and CHASE-DB1 to 95.47, 95.30 and 95.30, respectively.

A Skeletal Similarity Metric for Quality Evaluation of Retinal Vessel Segmentation

The most commonly used evaluation metrics for quality assessment of retinal vessel segmentation are sensitivity, specificity, and accuracy, which are based on pixel-to-pixel matching. However, due to the inter-observer problem that vessels annotated by different observers vary in both thickness and location, pixel-to-pixel matching is too restrictive to fairly evaluate the results of vessel segmentation. In this paper, the proposed skeletal similarity metric is constructed by comparing the skeleton maps generated from the reference and the source vessel segmentation maps. To address the inter-observer problem, instead of using a pixel-to-pixel matching strategy, each skeleton segment in the reference skeleton map is adaptively assigned with a searching range whose radius is determined based on its vessel thickness. Pixels in the source skeleton map located within the searching range are then selected for similarity calculation. The skeletal similarity consists of a curve similarity, which measures the structural similarity between the reference and the source skeleton maps and a thickness similarity, which measures the thickness consistency between the reference and the source vessel segmentation maps. In contrast to other metrics that provide a global score for the overall performance, we modify the definitions of true positive, false negative, true negative, and false positive based on the skeletal similarity, based on which sensitivity, specificity, accuracy, and other objective measurements can be constructed. More importantly, the skeletal similarity metric has better potential to be used as a pixelwise loss function for training deep learning models for retinal vessel segmentation. Through comparison of a set of examples, we demonstrate that the redefined metrics based on the skeletal similarity are more effective for quality evaluation, especially with greater tolerance to the inter-observer problem.

Supervised retinal vessel segmentation from color fundus images based on matched filtering and AdaBoost classifier

The structure and appearance of the blood vessel network in retinal fundus images is an essential part of diagnosing various problems associated with the eyes, such as diabetes and hypertension. In this paper, an automatic retinal vessel segmentation method utilizing matched filter techniques coupled with an AdaBoost classifier is proposed. The fundus image is enhanced using morphological operations, the contrast is increased using contrast limited adaptive histogram equalization (CLAHE) method and the inhomogeneity is corrected using Retinex approach. Then, the blood vessels are enhanced using a combination of B-COSFIRE and Frangi matched filters. From this preprocessed image, different statistical features are computed on a pixel-wise basis and used in an AdaBoost classifier to extract the blood vessel network inside the image. Finally, the segmented images are postprocessed to remove the misclassified pixels and regions. The proposed method was validated using publicly accessible Digital Retinal Images for Vessel Extraction (DRIVE), Structured Analysis of the Retina (STARE) and Child Heart and Health Study in England (CHASE_DB1) datasets commonly used for determining the accuracy of retinal vessel segmentation methods. The accuracy of the proposed segmentation method was comparable to other state of the art methods while being very close to the manual segmentation provided by the second human observer with an average accuracy of 0.972, 0.951 and 0.948 in DRIVE, STARE and CHASE_DB1 datasets, respectively.

An Automatic Detection System of Lung Nodule Based on Multigroup Patch-Based Deep Learning Network

High-efficiency lung nodule detection dramatically contributes to the risk assessment of lung cancer. It is a significant and challenging task to quickly locate the exact positions of lung nodules. Extensive work has been done by researchers around this domain for approximately two decades. However, previous computer-aided detection (CADe) schemes are mostly intricate and time-consuming since they may require more image processing modules, such as the computed tomography image transformation, the lung nodule segmentation, and the feature extraction, to construct a whole CADe system. It is difficult for these schemes to process and analyze enormous data when the medical images continue to increase. Besides, some state of the art deep learning schemes may be strict in the standard of database. This study proposes an effective lung nodule detection scheme based on multigroup patches cut out from the lung images, which are enhanced by the Frangi filter. Through combining two groups of images, a four-channel convolution neural networks model is designed to learn the knowledge of radiologists for detecting nodules of four levels. This CADe scheme can acquire the sensitivity of 80.06% with 4.7 false positives per scan and the sensitivity of 94% with 15.1 false positives per scan. The results demonstrate that the multigroup patch-based learning system is efficient to improve the performance of lung nodule detection and greatly reduce the false positives under a huge amount of image data.

Automated Quantification of Nonperfusion in Three Retinal Plexuses Using Projection-Resolved Optical Coherence Tomography Angiography in Diabetic Retinopathy

Purpose

The purpose of this study was to evaluate an automated algorithm for detecting avascular area (AA) in optical coherence tomography angiograms (OCTAs) separated into three individual plexuses using a projection-resolved technique.

Methods

A 3 × 3 mm macular OCTA was obtained in 13 healthy and 13 mild nonproliferative diabetic retinopathy (NPDR) participants. A projection-resolved algorithm segmented OCTA into three vascular plexuses: superficial, intermediate, and deep. An automated algorithm detected AA in each of the three plexuses that were segmented and in the combined inner-retinal angiograms. We assessed the diagnostic accuracy of extrafoveal and total AA using segmented and combined angiograms, the agreement between automated and manual detection of AA, and the within-visit repeatability.

Results

The sum of extrafoveal AA from the segmented angiograms was larger in the NPDR group by 0.17 mm² (P < 0.001) and detected NPDR with 94.6% sensitivity (area under the receiver operating characteristic curve [AROC] = 0.99). In the combined inner-retinal angiograms, the extrafoveal AA was larger in the NPDR group by 0.01 mm² (P = 0.168) and detected NPDR with 26.9% sensitivity (AROC = 0.62). The total AA, inclusive of the foveal avascular zone, in the segmented and combined angiograms, detected NPDR with 23.1% and 7.7% sensitivity, respectively. The agreement between the manual and automated detection of AA had a Jaccard index of >0.8. The pooled SDs of AA were small compared with the difference in mean for control and NPDR groups.

Conclusions

An algorithm to detect AA in OCTA separated into three individual plexuses using a projection-resolved algorithm accurately distinguishes mild NPDR from control eyes. Automatically detected AA agrees with manual delineation and is highly repeatable.

Retinal Image Denoising via Bilateral Filter with a Spatial Kernel of Optimally Oriented Line Spread Function

Filtering belongs to the most fundamental operations of retinal image processing and for which the value of the filtered image at a given location is a function of the values in a local window centered at this location. However, preserving thin retinal vessels during the filtering process is challenging due to vessels' small area and weak contrast compared to background, caused by the limited resolution of imaging and less blood flow in the vessel. In this paper, we present a novel retinal image denoising approach which is able to preserve the details of retinal vessels while effectively eliminating image noise. Specifically, our approach is carried out by determining an optimal spatial kernel for the bilateral filter, which is represented by a line spread function with an orientation and scale adjusted adaptively to the local vessel structure. Moreover, this approach can also be served as a preprocessing tool for improving the accuracy of the vessel detection technique. Experimental results show the superiority of our approach over state-of-the-art image denoising techniques such as the bilateral filter.

Joint volumetric extraction and enhancement of vasculature from low-SNR 3-D fluorescence microscopy images

To simultaneously overcome the challenges imposed by the nature of optical imaging characterized by a range of artifacts including space-varying signal to noise ratio (SNR), scattered light, and non-uniform illumination, we developed a novel method that segments the 3-D vasculature directly from original fluorescence microscopy images eliminating the need for employing pre- and post-processing steps such as noise removal and segmentation refinement as used with the majority of segmentation techniques. Our method comprises two initialization and constrained recovery and enhancement stages. The initialization approach is fully automated using features derived from bi-scale statistical measures and produces seed points robust to non-uniform illumination, low SNR, and local structural variations. This algorithm achieves the goal of segmentation via design of an iterative approach that extracts the structure through voting of feature vectors formed by distance, local intensity gradient, and median measures. Qualitative and quantitative analysis of the experimental results obtained from synthetic and real data prove the effcacy of this method in comparison to the state-of-the-art enhancing-segmenting methods. The algorithmic simplicity, freedom from having a priori probabilistic information about the noise, and structural definition gives this algorithm a wide potential range of applications where i.e. structural complexity significantly complicates the segmentation problem.

Three-Dimensional Segmentation of the Ex-Vivo Anterior Lamina Cribrosa From Second-Harmonic Imaging Microscopy

The lamina cribrosa (LC) is a connective tissue in the posterior eye with a complex mesh-like trabecular microstructure, through which all the retinal ganglion cell axons and central retinal vessels pass. Recent studies have demonstrated that changes in the structure of the LC correlate with glaucomatous damage. Thus, accurate segmentation and reconstruction of the LC is of utmost importance. This paper presents a new automated method for segmenting the microstructure of the anterior LC in the images obtained via multiphoton microscopy using a combination of ideas. In order to reduce noise, we first smooth the input image using a 4-D collaborative filtering scheme. Next, we enhance the beam-like trabecular microstructure of the LC using wavelet multiresolution analysis. The enhanced LC microstructure is then automatically extracted using a combination of histogram thresholding and graph-cut binarization. Finally, we use morphological area opening as a postprocessing step to remove the small and unconnected 3-D regions in the binarized images. The performance of the proposed method is evaluated using mutual overlap accuracy, Tanimoto index, F-score, and Rand index. Quantitative and qualitative results show that the proposed algorithm provides improved segmentation accuracy and computational efficiency compared to the other recent algorithms.

Blood vessel extraction of diabetic retinopathy using optimized enhanced images and matched filter

Accurate extraction of structural changes in the blood vessels of the retina is an essential task in diagnosis of retinopathy. Matched filter (MF) technique is the effective way to extract blood vessels, but the effectiveness is reduced due to noisy images. The concept of MF and MF with first-order derivative of Gaussian (MF-FDOG) has been implemented for retina images of the DRIVE database. The optimized particle swarm optimization (PSO) algorithm is used for enhancing the images by edgels to improve the performance of filters. The vessels were detected by the response of thresholding to the MF, whereas the threshold is adjusted in response to the FDOG. The PSO-based enhanced MF response significantly improved the performances of filters to extract fine blood vessels structures. Experimental results show that the proposed method based on enhanced images improved the accuracy to 91.1%, which is higher than that of MF and MF-FDOG, respectively. The peak signal-to-noise ratio was also found to be higher with low mean square error values in enhanced MF response. The accuracy, sensitivity, and specificity values are significantly improved among MF, MF-FDOG, and PSO-enhanced images ([Formula: see text]).

Determination of Statistical Properties of Microtubule Populations

Microtubules are structures within the cell that form a transportation network along which motor proteins tow cargo to destinations. To establish and maintain a structure capable of serving the cell’s tasks, microtubules undergo deconstruction and reconstruction regularly. This change in structure is critical to tasks like wound repair and cell motility. Images of fluorescing microtubule networks are captured in grayscale at different wavelengths, displaying different tagged proteins. The analysis of these polymeric structures involves identifying the presence of the protein and the direction of the structure in which it resides. This study considers the problem of finding statistical properties of sections of microtubules. We consider the research done on directional filters and utilize a basic solution to find the center of a ridge. The method processes the captured image by centering a circle around pre-determined pixel locations so that the highest possible average pixel intensity is found within the circle, thus marking the center of the microtubule. The location of these centers allows us to estimate angular direction and curvature of the microtubules, statistically estimate the direction of microtubules in a region of the cell, and compare properties of different types of microtubule networks in the same region. To verify accuracy, we study the results of the method on a test image.

Multiscale Centerline Detection

Finding the centerline and estimating the radius of linear structures is a critical first step in many applications, ranging from road delineation in 2D aerial images to modeling blood vessels, lung bronchi, and dendritic arbors in 3D biomedical image stacks. Existing techniques rely either on filters designed to respond to ideal cylindrical structures or on classification techniques. The former tend to become unreliable when the linear structures are very irregular while the latter often has difficulties distinguishing centerline locations from neighboring ones, thus losing accuracy. We solve this problem by reformulating centerline detection in terms of a regression problem. We first train regressors to return the distances to the closest centerline in scale-space, and we apply them to the input images or volumes. The centerlines and the corresponding scale then correspond to the regressors local maxima, which can be easily identified. We show that our method outperforms state-of-the-art techniques for various 2D and 3D datasets. Moreover, our approach is very generic and also performs well on contour detection. We show an improvement above recent contour detection algorithms on the BSDS500 dataset.

Retinal image registration via feature-guided Gaussian mixture model

Registration of retinal images taken at different times, from different perspectives, or with different modalities is a critical prerequisite for the diagnoses and treatments of various eye diseases. This problem can be formulated as registration of two sets of sparse feature points extracted from the given images, and it is typically solved by first creating a set of putative correspondences and then removing the false matches as well as estimating the spatial transformation between the image pairs or solved by estimating the correspondence and transformation jointly involving an iteration process. However, the former strategy suffers from missing true correspondences, and the latter strategy does not make full use of local appearance information, which may be problematic for low-quality retinal images due to a lack of reliable features. In this paper, we propose a feature-guided Gaussian mixture model (GMM) to address these issues. We formulate point registration as the estimation of a feature-guided mixture of densities: A GMM is fitted to one point set, such that both the centers and local features of the Gaussian densities are constrained to coincide with the other point set. The problem is solved under a unified maximum-likelihood framework together with an iterative expectation-maximization algorithm initialized by the confident feature correspondences, where the image transformation is modeled by an affine function. Extensive experiments on various retinal images show the robustness of our approach, which consistently outperforms other state-of-the-art methods, especially when the data is badly degraded.