CcNet: A cross-connected convolutional network for segmenting retinal vessels using multi-scale features

LiViT-Net: A U-Net-like, lightweight Transformer network for retinal vessel segmentation

The intricate task of precisely segmenting retinal vessels from images, which is critical for diagnosing various eye diseases, presents significant challenges for models due to factors such as scale variation, complex anatomical patterns, low contrast, and limitations in training data. Building on these challenges, we offer novel contributions spanning model architecture, loss function design, robustness, and real-time efficacy. To comprehensively address these challenges, a new U-Net-like, lightweight Transformer network for retinal vessel segmentation is presented. By integrating MobileViT+ and a novel local representation in the encoder, our design emphasizes lightweight processing while capturing intricate image structures, enhancing vessel edge precision. A novel joint loss is designed, leveraging the characteristics of weighted cross-entropy and Dice loss to effectively guide the model through the task's challenges, such as foreground-background imbalance and intricate vascular structures. Exhaustive experiments were performed on three prominent retinal image databases. The results underscore the robustness and generalizability of the proposed LiViT-Net, which outperforms other methods in complex scenarios, especially in intricate environments with fine vessels or vessel edges. Importantly, optimized for efficiency, LiViT-Net excels on devices with constrained computational power, as evidenced by its fast performance. To demonstrate the model proposed in this study, a freely accessible and interactive website was established (https://hz-t3.matpool.com:28765?token=aQjYR4hqMI), revealing real-time performance with no login requirements.

TLTNet: A novel transscale cascade layered transformer network for enhanced retinal blood vessel segmentation

Extracting global and local feature information is still challenging due to the problems of retinal blood vessel medical images like fuzzy edge features, noise, difficulty in distinguishing between lesion regions and background information, and loss of low-level feature information, which leads to insufficient extraction of feature information. To better solve these problems and fully extract the global and local feature information of the image, we propose a novel transscale cascade layered transformer network for enhanced retinal blood vessel segmentation, which consists of an encoder and a decoder and is connected between the encoder and decoder by a transscale transformer cascade module. Among them, the encoder consists of a local-global transscale transformer module, a multi-head layered transscale adaptive embedding module, and a local context(LCNet) module. The transscale transformer cascade module learns local and global feature information from the first three layers of the encoder, and multi-scale dependent features, fuses the hierarchical feature information from the skip connection block and the channel-token interaction fusion block, respectively, and inputs it to the decoder. The decoder includes a decoding module for the local context network and a transscale position transformer module to input the local and global feature information extracted from the encoder with retained key position information into the decoding module and the position embedding transformer module for recovery and output of the prediction results that are consistent with the input feature information. In addition, we propose an improved cross-entropy loss function based on the difference between the deterministic observation samples and the prediction results with the deviation distance, which is validated on the DRIVE and STARE datasets combined with the proposed network model based on the dual transformer structure in this paper, and the segmentation accuracies are 97.26% and 97.87%, respectively. Compared with other state-of-the-art networks, the results show that the proposed network model has a significant competitive advantage in improving the segmentation performance of retinal blood vessel images.

LMBiS-Net: A lightweight bidirectional skip connection based multipath CNN for retinal blood vessel segmentation

Blinding eye diseases are often related to changes in retinal structure, which can be detected by analysing retinal blood vessels in fundus images. However, existing techniques struggle to accurately segment these delicate vessels. Although deep learning has shown promise in medical image segmentation, its reliance on specific operations can limit its ability to capture crucial details such as the edges of the vessel. This paper introduces LMBiS-Net, a lightweight convolutional neural network designed for the segmentation of retinal vessels. LMBiS-Net achieves exceptional performance with a remarkably low number of learnable parameters (only 0.172 million). The network used multipath feature extraction blocks and incorporates bidirectional skip connections for the information flow between the encoder and decoder. In addition, we have optimised the efficiency of the model by carefully selecting the number of filters to avoid filter overlap. This optimisation significantly reduces training time and improves computational efficiency. To assess LMBiS-Net's robustness and ability to generalise to unseen data, we conducted comprehensive evaluations on four publicly available datasets: DRIVE, STARE, CHASE_DB1, and HRF The proposed LMBiS-Net achieves significant performance metrics in various datasets. It obtains sensitivity values of 83.60%, 84.37%, 86.05%, and 83.48%, specificity values of 98.83%, 98.77%, 98.96%, and 98.77%, accuracy (acc) scores of 97.08%, 97.69%, 97.75%, and 96.90%, and AUC values of 98.80%, 98.82%, 98.71%, and 88.77% on the DRIVE, STARE, CHEASE_DB, and HRF datasets, respectively. In addition, it records F1 scores of 83.43%, 84.44%, 83.54%, and 78.73% on the same datasets. Our evaluations demonstrate that LMBiS-Net achieves high segmentation accuracy (acc) while exhibiting both robustness and generalisability across various retinal image datasets. This combination of qualities makes LMBiS-Net a promising tool for various clinical applications.

Foreground segmentation network using transposed convolutional neural networks and up sampling for multiscale feature encoding

Foreground segmentation algorithm aims to precisely separate moving objects from the background in various environments. However, the interference from darkness, dynamic background information, and camera jitter makes it still challenging to build a decent detection network. To solve these issues, a triplet CNN and Transposed Convolutional Neural Network (TCNN) are created by attaching a Features Pooling Module (FPM). TCNN process reduces the amount of multi-scale inputs to the network by fusing features into the Foreground Segmentation Network (FgSegNet) based FPM, which extracts multi-scale features from images and builds a strong feature pooling. Additionally, the up-sampling network is added to the proposed technique, which is used to up-sample the abstract image representation, so that its spatial dimensions match with the input image. The large context and long-range dependencies among pixels are acquired by TCNN and segmentation mask, in multiple scales using triplet CNN, to enhance the foreground segmentation of FgSegNet. The results, clearly show that FgSegNet surpasses other state-of-the-art algorithms on the CDnet2014 datasets, with an average F-Measure of 0.9804, precision of 0.9801, PWC as (0.0461), and recall as (0.9896). Moreover, the FgSegNet with up-sampling achieves the F-measure of 0.9804 which is higher when compared to the FgSegNet without up-sampling.

An improved supervised and attention mechanism-based U-Net algorithm for retinal vessel segmentation

The segmentation results of retinal blood vessels are crucial for automatically diagnosing ophthalmic diseases such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular diseases. To improve the accuracy of vessel segmentation and better extract information about small vessels and edges, we introduce the U-Net algorithm with a supervised attention mechanism for retinal vessel segmentation. We achieve this by introducing a decoder fusion module (DFM) in the encoding part, effectively combining different convolutional blocks to extract features comprehensively. Additionally, in the decoding part of U-Net, we propose the context squeeze and excitation (CSE) decoding module to enhance important contextual feature information and the detection of tiny blood vessels. For the final output, we introduce the supervised fusion mechanism (SFM), which combines multiple branches from shallow to deep layers, effectively fusing multi-scale features and capturing information from different levels, fully integrating low-level and high-level features to improve segmentation performance. Our experimental results on the public datasets of DRIVE, STARE, and CHASED_B1 demonstrate the excellent performance of our proposed network.

Multi-level spatial-temporal and attentional information deep fusion network for retinal vessel segmentation

Objective.Automatic segmentation of fundus vessels has the potential to enhance the judgment ability of intelligent disease diagnosis systems. Even though various methods have been proposed, it is still a demanding task to accurately segment the fundus vessels. The purpose of our study is to develop a robust and effective method to segment the vessels in human color retinal fundus images.Approach.We present a novel multi-level spatial-temporal and attentional information deep fusion network for the segmentation of retinal vessels, called MSAFNet, which enhances segmentation performance and robustness. Our method utilizes the multi-level spatial-temporal encoding module to obtain spatial-temporal information and the Self-Attention module to capture feature correlations in different levels of our network. Based on the encoder and decoder structure, we combine these features to get the final segmentation results.Main results.Through abundant experiments on four public datasets, our method achieves preferable performance compared with other SOTA retinal vessel segmentation methods. Our Accuracy and Area Under Curve achieve the highest scores of 96.96%, 96.57%, 96.48% and 98.78%, 98.54%, 98.27% on DRIVE, CHASE_DB1, and HRF datasets. Our Specificity achieves the highest score of 98.58% and 99.08% on DRIVE and STARE datasets.Significance.The experimental results demonstrate that our method has strong learning and representation capabilities and can accurately detect retinal blood vessels, thereby serving as a potential tool for assisting in diagnosis.

CCS-UNet: a cross-channel spatial attention model for accurate retinal vessel segmentation

Precise segmentation of retinal vessels plays an important role in computer-assisted diagnosis. Deep learning models have been applied to retinal vessel segmentation, but the efficacy is limited by the significant scale variation of vascular structures and the intricate background of retinal images. This paper supposes a cross-channel spatial attention U-Net (CCS-UNet) for accurate retinal vessel segmentation. In comparison to other models based on U-Net, our model employes a ResNeSt block for the encoder-decoder architecture. The block has a multi-branch structure that enables the model to extract more diverse vascular features. It facilitates weight distribution across channels through the incorporation of soft attention, which effectively aggregates contextual information in vascular images. Furthermore, we suppose an attention mechanism within the skip connection. This mechanism serves to enhance feature integration across various layers, thereby mitigating the degradation of effective information. It helps acquire cross-channel information and enhance the localization of regions of interest, ultimately leading to improved recognition of vascular structures. In addition, the feature fusion module (FFM) module is used to provide semantic information for a more refined vascular segmentation map. We evaluated CCS-UNet based on five benchmark retinal image datasets, DRIVE, CHASEDB1, STARE, IOSTAR and HRF. Our proposed method exhibits superior segmentation efficacy compared to other state-of-the-art techniques with a global accuracy of 0.9617/0.9806/0.9766/0.9786/0.9834 and AUC of 0.9863/0.9894/0.9938/0.9902/0.9855 on DRIVE, CHASEDB1, STARE, IOSTAR and HRF respectively. Ablation studies are also performed to evaluate the the relative contributions of different architectural components. Our proposed model is potential for diagnostic aid of retinal diseases.

Global Transformer and Dual Local Attention Network via Deep-Shallow Hierarchical Feature Fusion for Retinal Vessel Segmentation

Clinically, retinal vessel segmentation is a significant step in the diagnosis of fundus diseases. However, recent methods generally neglect the difference of semantic information between deep and shallow features, which fail to capture the global and local characterizations in fundus images simultaneously, resulting in the limited segmentation performance for fine vessels. In this article, a global transformer (GT) and dual local attention (DLA) network via deep-shallow hierarchical feature fusion (GT-DLA-dsHFF) are investigated to solve the above limitations. First, the GT is developed to integrate the global information in the retinal image, which effectively captures the long-distance dependence between pixels, alleviating the discontinuity of blood vessels in the segmentation results. Second, DLA, which is constructed using dilated convolutions with varied dilation rates, unsupervised edge detection, and squeeze-excitation block, is proposed to extract local vessel information, consolidating the edge details in the segmentation result. Finally, a novel deep-shallow hierarchical feature fusion (dsHFF) algorithm is studied to fuse the features in different scales in the deep learning framework, respectively, which can mitigate the attenuation of valid information in the process of feature fusion. We verified the GT-DLA-dsHFF on four typical fundus image datasets. The experimental results demonstrate our GT-DLA-dsHFF achieves superior performance against the current methods and detailed discussions verify the efficacy of the proposed three modules. Segmentation results of diseased images show the robustness of our proposed GT-DLA-dsHFF. Implementation codes will be available on https://github.com/YangLibuaa/GT-DLA-dsHFF.

SCANet: A Unified Semi-Supervised Learning Framework for Vessel Segmentation

Automatic subcutaneous vessel imaging with near-infrared (NIR) optical apparatus can promote the accuracy of locating blood vessels, thus significantly contributing to clinical venipuncture research. Though deep learning models have achieved remarkable success in medical image segmentation, they still struggle in the subfield of subcutaneous vessel segmentation due to the scarcity and low-quality of annotated data. To relieve it, this work presents a novel semi-supervised learning framework, SCANet, that achieves accurate vessel segmentation through an alternate training strategy. The SCANet is composed of a multi-scale recurrent neural network that embeds coarse-to-fine features and two auxiliary branches, a consistency decoder and an adversarial learning branch, responsible for strengthening fine-grained details and eliminating differences between ground-truths and predictions, respectively. Equipped with a novel semi-supervised alternate training strategy, the three components work collaboratively, enabling SCANet to accurately segment vessel regions with only a handful of labeled data and abounding unlabeled data. Moreover, to mitigate the shortage of annotated data in this field, we provide a new subcutaneous vessel dataset, VESSEL-NIR. Extensive experiments on a wide variety of tasks, including the segmentation of subcutaneous vessels, retinal vessels, and skin lesions, well demonstrate the superiority and generality of our approach.

Separable Paravector Orientation Tensors for Enhancing Retinal Vessels

Robust detection of retinal vessels remains an unsolved research problem, particularly in handling the intrinsic real-world challenges of highly imbalanced contrast between thick vessels and thin ones, inhomogeneous background regions, uneven illumination, and complex geometries of crossing/bifurcations. This paper presents a new separable paravector orientation tensor that addresses these difficulties by characterizing the enhancement of retinal vessels to be dependent on a nonlinear scale representation, invariant to changes in contrast and lighting, responsive for symmetric patterns, and fitted with elliptical cross-sections. The proposed method is built on projecting vessels as a 3D paravector valued function rotated in an alpha quarter domain, providing geometrical, structural, symmetric, and energetic features. We introduce an innovative symmetrical inhibitory scheme that incorporates paravector features for producing a set of directional contrast-independent elongated-like patterns reconstructing vessel tree in orientation tensors. By fitting constraint elliptical volumes via eigensystem analysis, the final vessel tree is produced with a strong and uniform response preserving various vessel features. The validation of proposed method on clinically relevant retinal images with high-quality results, shows its excellent performance compared to the state-of-the-art benchmarks and the second human observers.

Attention aware fully convolutional deep learning model for retinal blood vessel segmentation

Recent reports indicate a rise in retinal issues, and automatic artery vein categorization offers data that is particularly instructive for the medical evaluation of serious retinal disorders including glaucoma and diabetic retinopathy. This work presents a competent and precise deep-learning model designed for vessel segmentation in retinal fundus imaging. This article aims to segment the retinal images using an attention-based dense fully convolutional neural network (A-DFCNN) after removing uncertainty. The artery extraction layers encompass vessel-specific convolutional blocks to focus the tiny blood vessels and dense layers with skip connections for feature propagation. Segmentation is associated with artery extraction layers via individual loss function. Blood vessel maps produced from individual loss functions are authenticated for performance. The proposed technique attains improved outcomes in terms of Accuracy (0.9834), Sensitivity (0.8553), and Specificity (0.9835) from DRIVE, STARE, and CHASE-DB1 datasets. The result demonstrates that the proposed A-DFCNN is capable of segmenting minute vessel bifurcation breakdowns during the training and testing phases.

Wave-Net: A lightweight deep network for retinal vessel segmentation from fundus images

Accurate segmentation of retinal vessels from fundus images is fundamental for the diagnosis of numerous diseases of eye, and an automated vessel segmentation method can effectively help clinicians to make accurate diagnosis for the patients and provide the appropriate treatment schemes. It is important to note that both thick and thin vessels play the key role for disease judgements. Because of complex factors, the precise segmentation of thin vessels is still a great challenge, such as the presence of various lesions, image noise, complex backgrounds and poor contrast in the fundus images. Recently, because of the advantage of context feature representation learning capabilities, deep learning has shown a remarkable segmentation performance on retinal vessels. However, it still has some shortcomings on high-precision retinal vessel extraction due to some factors, such as semantic information loss caused by pooling operations, limited receptive field, etc. To address these problems, this paper proposes a new lightweight segmentation network for precise retinal vessel segmentation, which is called as Wave-Net model on account of the whole shape. To alleviate the influence of semantic information loss problem to thin vessels, to acquire more contexts about micro structures and details, a detail enhancement and denoising block (DED) is proposed to improve the segmentation precision on thin vessels, which replaces the simple skip connections of original U-Net. On the other hand, it could well alleviate the influence of the semantic gap problem. Further, faced with limited receptive field, for multi-scale vessel detection, a multi-scale feature fusion block (MFF) is proposed to fuse cross-scale contexts to achieve higher segmentation accuracy and realize effective processing of local feature maps. Experiments indicate that proposed Wave-Net achieves an excellent performance on retinal vessel segmentation while maintaining a lightweight network design compared to other advanced segmentation methods, and it also has shown a better segmentation ability to thin vessels.

Recent trends and advances in fundus image analysis: A review

Automated retinal image analysis holds prime significance in the accurate diagnosis of various critical eye diseases that include diabetic retinopathy (DR), age-related macular degeneration (AMD), atherosclerosis, and glaucoma. Manual diagnosis of retinal diseases by ophthalmologists takes time, effort, and financial resources, and is prone to error, in comparison to computer-aided diagnosis systems. In this context, robust classification and segmentation of retinal images are primary operations that aid clinicians in the early screening of patients to ensure the prevention and/or treatment of these diseases. This paper conducts an extensive review of the state-of-the-art methods for the detection and segmentation of retinal image features. Existing notable techniques for the detection of retinal features are categorized into essential groups and compared in depth. Additionally, a summary of quantifiable performance measures for various important stages of retinal image analysis, such as image acquisition and preprocessing, is provided. Finally, the widely used in the literature datasets for analyzing retinal images are described and their significance is emphasized.

Retinal Vessel Segmentation, a Review of Classic and Deep Methods

Retinal illnesses such as diabetic retinopathy (DR) are the main causes of vision loss. In the early recognition of eye diseases, the segmentation of blood vessels in retina images plays an important role. Different symptoms of ocular diseases can be identified by the geometric features of ocular arteries. However, due to the complex construction of the blood vessels and their different thicknesses, segmenting the retina image is a challenging task. There are a number of algorithms that helped the detection of retinal diseases. This paper presents an overview of papers from 2016 to 2022 that discuss machine learning and deep learning methods for automatic vessel segmentation. The methods are divided into two groups: Deep learning-based, and classic methods. Algorithms, classifiers, pre-processing and specific techniques of each group is described, comprehensively. The performances of recent works are compared based on their achieved accuracy in different datasets in inclusive tables. A survey of most popular datasets like DRIVE, STARE, HRF and CHASE_DB1 is also given in this paper. Finally, a list of findings from this review is presented in the conclusion section.

Robust Detection Model of Vascular Landmarks for Retinal Image Registration: A Two-Stage Convolutional Neural Network

Registration is useful for image processing in computer vision. It can be applied to retinal images and provide support for ophthalmologists in tracking disease progression and monitoring therapeutic responses. This study proposed a robust detection model of vascular landmarks to improve the performance of retinal image registration. The proposed model consists of a two-stage convolutional neural network, in which one segments the retinal vessels on a pair of images, and the other detects junction points from the vessel segmentation image. Information obtained from the model was utilized for the registration. The keypoints were extracted based on the acquired vascular landmark points, and the orientation features were calculated as descriptors. Then, the reference and sensed images were registered by matching keypoints using a homography matrix and random sample consensus algorithm. The proposed method was evaluated on five databases and seven evaluation metrics to verify both clinical effectiveness and robustness. The results established that the proposed method showed outstanding performance for registration compared with other state-of-the-art methods. In particular, the high and significantly improved registration results were identified on FIRE database with area under the curve (AUC) of 0.988, 0.511, and 0.803 in S, P, and A classes. Furthermore, the proposed method worked well on poor quality and multimodal datasets demonstrating an ability to achieve high AUC above 0.8.

Image Segmentation of Retinal Blood Vessels Based on Dual-Attention Multiscale Feature Fusion

Aiming at the problem of insufficient details of retinal blood vessel segmentation in current research methods, this paper proposes a multiscale feature fusion residual network based on dual attention. Specifically, a feature fusion residual module with adaptive calibration weight features is designed, which avoids gradient dispersion and network degradation while effectively extracting image details. The SA module and ECA module are used many times in the backbone feature extraction network to adaptively select the focus position to generate more discriminative feature representations; at the same time, the information of different levels of the network is fused, and long-range and short-range features are used. This method aggregates low-level and high-level feature information, which effectively improves the segmentation performance. The experimental results show that the method in this paper achieves the classification accuracy of 0.9795 and 0.9785 on the STARE and DRIVE datasets, respectively, and the classification performance is better than the current mainstream methods.

Which Color Channel Is Better for Diagnosing Retinal Diseases Automatically in Color Fundus Photographs?

Color fundus photographs are the most common type of image used for automatic diagnosis of retinal diseases and abnormalities. As all color photographs, these images contain information about three primary colors, i.e., red, green, and blue, in three separate color channels. This work aims to understand the impact of each channel in the automatic diagnosis of retinal diseases and abnormalities. To this end, the existing works are surveyed extensively to explore which color channel is used most commonly for automatically detecting four leading causes of blindness and one retinal abnormality along with segmenting three retinal landmarks. From this survey, it is clear that all channels together are typically used for neural network-based systems, whereas for non-neural network-based systems, the green channel is most commonly used. However, from the previous works, no conclusion can be drawn regarding the importance of the different channels. Therefore, systematic experiments are conducted to analyse this. A well-known U-shaped deep neural network (U-Net) is used to investigate which color channel is best for segmenting one retinal abnormality and three retinal landmarks.

U-shaped Retinal Vessel Segmentation Based on Adaptive Aggregation of Feature Information

Detection and analysis of retinal blood vessels contribute to the clinical diagnosis of many ophthalmic diseases. In this paper, aiming on achieving more accurate segmentation of retinal vessels and enhance the ability of the algorithm to identify microvessels, we propose a U-shaped network based on adaptive aggregation of feature information. The introduced feature selection module, which could strengthen feature transmission and selectively emphasize feature information. To effectively capture the characteristics of vessels at different scales, generate richer and denser context information, and DenseASPP is embedded at the bottom of the network. Meanwhile, we propose an adaptive aggregation module to aggregate the semantic information in each layer of the encoder part and transmit it to subsequent layers, which is beneficial to the spatial reconstruction of retinal vessels. A joint loss function is also introduced to facilitate network training. The proposed network is evaluated on three public datasets. The sensitivity, accuracy, and area under curve(AUC) are 83.48%/83.16/85.86, 95.67%/96.67%/96.52%, and 98.11%/98.69%/98.60% on DRIVE, STARE and CHASE_DB1, respectively. In order to achieve more accurate retinal blood vessel segmentation and improve the ability of the algorithm to identify microvessels. We propose a U-shaped network based on adaptive aggregation of feature information. The introduction of the adaptive aggregation module aggregates the semantic information of each level of the encoder part, which improves the robustness of the model to segment blood vessels.

Segmenting Retinal Vessels Using a Shallow Segmentation Network to Aid Ophthalmic Analysis

Retinal blood vessels possess a complex structure in the retina and are considered an important biomarker for several retinal diseases. Ophthalmic diseases result in specific changes in the retinal vasculature; for example, diabetic retinopathy causes the retinal vessels to swell, and depending upon disease severity, fluid or blood can leak. Similarly, hypertensive retinopathy causes a change in the retinal vasculature due to the thinning of these vessels. Central retinal vein occlusion (CRVO) is a phenomenon in which the main vein causes drainage of the blood from the retina and this main vein can close completely or partially with symptoms of blurred vision and similar eye problems. Considering the importance of the retinal vasculature as an ophthalmic disease biomarker, ophthalmologists manually analyze retinal vascular changes. Manual analysis is a tedious task that requires constant observation to detect changes. The deep learning-based methods can ease the problem by learning from the annotations provided by an expert ophthalmologist. However, current deep learning-based methods are relatively inaccurate, computationally expensive, complex, and require image preprocessing for final detection. Moreover, existing methods are unable to provide a better true positive rate (sensitivity), which shows that the model can predict most of the vessel pixels. Therefore, this study presents the so-called vessel segmentation ultra-lite network (VSUL-Net) to accurately extract the retinal vasculature from the background. The proposed VSUL-Net comprises only 0.37 million trainable parameters and uses an original image as input without preprocessing. The VSUL-Net uses a retention block that specifically maintains the larger feature map size and low-level spatial information transfer. This retention block results in better sensitivity of the proposed VSUL-Net without using expensive preprocessing schemes. The proposed method was tested on three publicly available datasets: digital retinal images for vessel extraction (DRIVE), structured analysis of retina (STARE), and children’s heart health study in England database (CHASE-DB1) for retinal vasculature segmentation. The experimental results demonstrated that VSUL-Net provides robust segmentation of retinal vasculature with sensitivity (Sen), specificity (Spe), accuracy (Acc), and area under the curve (AUC) values of 83.80%, 98.21%, 96.95%, and 98.54%, respectively, for DRIVE, 81.73%, 98.35%, 97.17%, and 98.69%, respectively, for CHASE-DB1, and 86.64%, 98.13%, 97.27%, and 99.01%, respectively, for STARE datasets. The proposed method provides an accurate segmentation mask for deep ophthalmic analysis.

Robust Detection and Modeling of the Major Temporal Arcade in Retinal Fundus Images

The Major Temporal Arcade (MTA) is a critical component of the retinal structure that facilitates clinical diagnosis and monitoring of various ocular pathologies. Although recent works have addressed the quantitative analysis of the MTA through parametric modeling, their efforts are strongly based on an assumption of symmetry in the MTA shape. This work presents a robust method for the detection and piecewise parametric modeling of the MTA in fundus images. The model consists of a piecewise parametric curve with the ability to consider both symmetric and asymmetric scenarios. In an initial stage, multiple models are built from random blood vessel points taken from the blood-vessel segmented retinal image, following a weighted-RANSAC strategy. To choose the final model, the algorithm extracts blood-vessel width and grayscale-intensity features and merges them to obtain a coarse MTA probability function, which is used to weight the percentage of inlier points for each model. This procedure promotes selecting a model based on points with high MTA probability. Experimental results in the public benchmark dataset Digital Retinal Images for Vessel Extraction (DRIVE), for which manual MTA delineations have been prepared, indicate that the proposed method outperforms existing approaches with a balanced Accuracy of 0.7067, Mean Distance to Closest Point of 7.40 pixels, and Hausdorff Distance of 27.96 pixels, while demonstrating competitive results in terms of execution time (9.93 s per image).

Novel convolutional neural networks for efficient classification of rotated and scaled images

This paper presents a novel method for improving the invariance of convolutional neural networks (CNNs) to selected geometric transformations in order to obtain more efficient image classifiers. A common strategy employed to achieve this aim is to train the network using data augmentation. Such a method alone, however, increases the complexity of the neural network model, as any change in the rotation or size of the input image results in the activation of different CNN feature maps. This problem can be resolved by the proposed novel convolutional neural network models with geometric transformations embedded into the network architecture. The evaluation of the proposed CNN model is performed on the image classification task with the use of diverse representative data sets. The CNN models with embedded geometric transformations are compared to those without the transformations, using different data augmentation setups. As the compared approaches use the same amount of memory to store the parameters, the improved classification score means that the proposed architecture is more optimal.

Diabetic and Hypertensive Retinopathy Screening in Fundus Images Using Artificially Intelligent Shallow Architectures

Retinal blood vessels are considered valuable biomarkers for the detection of diabetic retinopathy, hypertensive retinopathy, and other retinal disorders. Ophthalmologists analyze retinal vasculature by manual segmentation, which is a tedious task. Numerous studies have focused on automatic retinal vasculature segmentation using different methods for ophthalmic disease analysis. However, most of these methods are computationally expensive and lack robustness. This paper proposes two new shallow deep learning architectures: dual-stream fusion network (DSF-Net) and dual-stream aggregation network (DSA-Net) to accurately detect retinal vasculature. The proposed method uses semantic segmentation in raw color fundus images for the screening of diabetic and hypertensive retinopathies. The proposed method's performance is assessed using three publicly available fundus image datasets: Digital Retinal Images for Vessel Extraction (DRIVE), Structured Analysis of Retina (STARE), and Children Heart Health Study in England Database (CHASE-DB1). The experimental results revealed that the proposed method provided superior segmentation performance with accuracy (Acc), sensitivity (SE), specificity (SP), and area under the curve (AUC) of 96.93%, 82.68%, 98.30%, and 98.42% for DRIVE, 97.25%, 82.22%, 98.38%, and 98.15% for CHASE-DB1, and 97.00%, 86.07%, 98.00%, and 98.65% for STARE datasets, respectively. The experimental results also show that the proposed DSA-Net provides higher SE compared to the existing approaches. It means that the proposed method detected the minor vessels and provided the least false negatives, which is extremely important for diagnosis. The proposed method provides an automatic and accurate segmentation mask that can be used to highlight the vessel pixels. This detected vasculature can be utilized to compute the ratio between the vessel and the non-vessel pixels and distinguish between diabetic and hypertensive retinopathies, and morphology can be analyzed for related retinal disorders.

Bi-SANet—Bilateral Network with Scale Attention for Retinal Vessel Segmentation

The segmentation of retinal vessels is critical for the diagnosis of some fundus diseases. Retinal vessel segmentation requires abundant spatial information and receptive fields with different sizes while existing methods usually sacrifice spatial resolution to achieve real-time reasoning speed, resulting in inadequate vessel segmentation of low-contrast regions and weak anti-noise interference ability. The asymmetry of capillaries in fundus images also increases the difficulty of segmentation. In this paper, we proposed a two-branch network based on multi-scale attention to alleviate the above problem. First, a coarse network with multi-scale U-Net as the backbone is designed to capture more semantic information and to generate high-resolution features. A multi-scale attention module is used to obtain enough receptive fields. The other branch is a fine network, which uses the residual block of a small convolution kernel to make up for the deficiency of spatial information. Finally, we use the feature fusion module to aggregate the information of the coarse and fine networks. The experiments were performed on the DRIVE, CHASE, and STARE datasets. Respectively, the accuracy reached 96.93%, 97.58%, and 97.70%. The specificity reached 97.72%, 98.52%, and 98.94%. The F-measure reached 83.82%, 81.39%, and 84.36%. Experimental results show that compared with some state-of-art methods such as Sine-Net, SA-Net, our proposed method has better performance on three datasets.

Gated Skip-Connection Network with Adaptive Upsampling for Retinal Vessel Segmentation

Segmentation of retinal vessels is a critical step for the diagnosis of some fundus diseases. Methods: To further enhance the performance of vessel segmentation, we propose a method based on a gated skip-connection network with adaptive upsampling (GSAU-Net). In GSAU-Net, a novel skip-connection with gating is first utilized in the extension path, which facilitates the flow of information from the encoder to the decoder. Specifically, we used the gated skip-connection between the encoder and decoder to gate the lower-level information from the encoder. In the decoding phase, we used an adaptive upsampling to replace the bilinear interpolation, which recovers feature maps from the decoder to obtain the pixelwise prediction. Finally, we validated our method on the DRIVE, CHASE, and STARE datasets. Results: The experimental results showed that our proposed method outperformed some existing methods, such as DeepVessel, AG-Net, and IterNet, in terms of accuracy, F-measure, and AUCROC. The proposed method achieved a vessel segmentation F-measure of 83.13%, 81.40%, and 84.84% on the DRIVE, CHASE, and STARE datasets, respectively.

TA-Net: Triple attention network for medical image segmentation

The automatic segmentation of medical images has made continuous progress due to the development of convolutional neural networks (CNNs) and attention mechanism. However, previous works usually explore the attention features of a certain dimension in the image, thus may ignore the correlation between feature maps in other dimensions. Therefore, how to capture the global features of various dimensions is still facing challenges. To deal with this problem, we propose a triple attention network (TA-Net) by exploring the ability of the attention mechanism to simultaneously recognize global contextual information in the channel domain, spatial domain, and feature internal domain. Specifically, during the encoder step, we propose a channel with self-attention encoder (CSE) block to learn the long-range dependencies of pixels. The CSE effectively increases the receptive field and enhances the representation of target features. In the decoder step, we propose a spatial attention up-sampling (SU) block that makes the network pay more attention to the position of the useful pixels when fusing the low-level and high-level features. Extensive experiments were tested on four public datasets and one local dataset. The datasets include the following types: retinal blood vessels (DRIVE and STARE), cells (ISBI 2012), cutaneous melanoma (ISIC 2017), and intracranial blood vessels. Experimental results demonstrate that the proposed TA-Net is overall superior to previous state-of-the-art methods in different medical image segmentation tasks with high accuracy, promising robustness, and relatively low redundancy.

Simultaneous segmentation and classification of the retinal arteries and veins from color fundus images

BACKGROUND AND OBJECTIVES

The study of the retinal vasculature represents a fundamental stage in the screening and diagnosis of many high-incidence diseases, both systemic and ophthalmic. A complete retinal vascular analysis requires the segmentation of the vascular tree along with the classification of the blood vessels into arteries and veins. Early automatic methods approach these complementary segmentation and classification tasks in two sequential stages. However, currently, these two tasks are approached as a joint semantic segmentation, because the classification results highly depend on the effectiveness of the vessel segmentation. In that regard, we propose a novel approach for the simultaneous segmentation and classification of the retinal arteries and veins from eye fundus images.

METHODS

We propose a novel method that, unlike previous approaches, and thanks to the proposal of a novel loss, decomposes the joint task into three segmentation problems targeting arteries, veins and the whole vascular tree. This configuration allows to handle vessel crossings intuitively and directly provides accurate segmentation masks of the different target vascular trees.

RESULTS

The provided ablation study on the public Retinal Images vessel Tree Extraction (RITE) dataset demonstrates that the proposed method provides a satisfactory performance, particularly in the segmentation of the different structures. Furthermore, the comparison with the state of the art shows that our method achieves highly competitive results in the artery/vein classification, while significantly improving the vascular segmentation.

CONCLUSIONS

The proposed multi-segmentation method allows to detect more vessels and better segment the different structures, while achieving a competitive classification performance. Also, in these terms, our approach outperforms the approaches of various reference works. Moreover, in contrast with previous approaches, the proposed method allows to directly detect the vessel crossings, as well as preserving the continuity of both arteries and veins at these complex locations.

A new deep learning method for blood vessel segmentation in retinal images based on convolutional kernels and modified U-Net model

BACKGROUND AND OBJECTIVE

Automatic monitoring of retinal blood vessels proves very useful for the clinical assessment of ocular vascular anomalies or retinopathies. This paper presents an efficient and accurate deep learning-based method for vessel segmentation in eye fundus images.

METHODS

The approach consists of a convolutional neural network based on a simplified version of the U-Net architecture that combines residual blocks and batch normalization in the up- and downscaling phases. The network receives patches extracted from the original image as input and is trained with a novel loss function that considers the distance of each pixel to the vascular tree. At its output, it generates the probability of each pixel of the input patch belonging to the vascular structure. The application of the network to the patches in which a retinal image can be divided allows obtaining the pixel-wise probability map of the complete image. This probability map is then binarized with a certain threshold to generate the blood vessel segmentation provided by the method.

RESULTS

The method has been developed and evaluated in the DRIVE, STARE and CHASE_Db1 databases, which offer a manual segmentation of the vascular tree by each of its images. Using this set of images as ground truth, the accuracy of the vessel segmentations obtained for an operating point proposal (established by a single threshold value for each database) was quantified. The overall performance was measured using the area of its receiver operating characteristic curve. The method demonstrated robustness in the face of the variability of the fundus images of diverse origin, being capable of working with the highest level of accuracy in the entire set of possible points of operation, compared to those provided by the most accurate methods found in literature.

CONCLUSIONS

The analysis of results concludes that the proposed method reaches better performance than the rest of state-of-art methods and can be considered the most promising for integration into a real tool for vascular structure segmentation.

Multichannel Retinal Blood Vessel Segmentation Based on the Combination of Matched Filter and U-Net Network

Aiming at the current problem of insufficient extraction of small retinal blood vessels, we propose a retinal blood vessel segmentation algorithm that combines supervised learning and unsupervised learning algorithms. In this study, we use a multiscale matched filter with vessel enhancement capability and a U-Net model with a coding and decoding network structure. Three channels are used to extract vessel features separately, and finally, the segmentation results of the three channels are merged. The algorithm proposed in this paper has been verified and evaluated on the DRIVE, STARE, and CHASE_DB1 datasets. The experimental results show that the proposed algorithm can segment small blood vessels better than most other methods. We conclude that our algorithm has reached 0.8745, 0.8903, and 0.8916 on the three datasets in the sensitivity metric, respectively, which is nearly 0.1 higher than other existing methods.

Applications of deep learning in fundus images: A review

The use of fundus images for the early screening of eye diseases is of great clinical importance. Due to its powerful performance, deep learning is becoming more and more popular in related applications, such as lesion segmentation, biomarkers segmentation, disease diagnosis and image synthesis. Therefore, it is very necessary to summarize the recent developments in deep learning for fundus images with a review paper. In this review, we introduce 143 application papers with a carefully designed hierarchy. Moreover, 33 publicly available datasets are presented. Summaries and analyses are provided for each task. Finally, limitations common to all tasks are revealed and possible solutions are given. We will also release and regularly update the state-of-the-art results and newly-released datasets at https://github.com/nkicsl/Fundus_Review to adapt to the rapid development of this field.

VSSC Net: Vessel Specific Skip chain Convolutional Network for blood vessel segmentation

BACKGROUND AND OBJECTIVE

Deep learning techniques are instrumental in developing network models that aid in the early diagnosis of life-threatening diseases. To screen and diagnose the retinal fundus and coronary blood vessel disorders, the most important step is the proper segmentation of the blood vessels.

METHODS

This paper aims to segment the blood vessels from both the coronary angiogram and the retinal fundus images using a single VSSC Net after performing the image-specific preprocessing. The VSSC Net uses two-vessel extraction layers with added supervision on top of the base VGG-16 network. The vessel extraction layers comprise of the vessel-specific convolutional blocks to localize the blood vessels, skip chain convolutional layers to enable rich feature propagation, and a unique feature map summation. Supervision is associated with the two-vessel extraction layers using separate loss/sigmoid function. Finally, the weighted fusion of the individual loss/sigmoid function produces the desired blood vessel probability map. It is then binary segmented and validated for performance.

RESULTS

The VSSC Net shows improved accuracy values on the standard retinal and coronary angiogram datasets respectively. The computational time required to segment the blood vessels is 0.2 seconds using GPU. Moreover, the vessel extraction layer uses a lesser parameter count of 0.4 million parameters to accurately segment the blood vessels.

CONCLUSION

The proposed VSSC Net that segments blood vessels from both the retinal fundus images and coronary angiogram can be used for the early diagnosis of vessel disorders. Moreover, it could aid the physician to analyze the blood vessel structure of images obtained from multiple imaging sources.

MRU-NET: A U-Shaped Network for Retinal Vessel Segmentation

Fundus blood vessel image segmentation plays an important role in the diagnosis and treatment of diseases and is the basis of computer-aided diagnosis. Feature information from the retinal blood vessel image is relatively complicated, and the existing algorithms are sometimes difficult to perform effective segmentation with. Aiming at the problems of low accuracy and low sensitivity of the existing segmentation methods, an improved U-shaped neural network (MRU-NET) segmentation method for retinal vessels was proposed. Firstly, the image enhancement algorithm and random segmentation method are used to solve the problems of low contrast and insufficient image data of the original image. Moreover, smaller image blocks after random segmentation are helpful to reduce the complexity of the U-shaped neural network model; secondly, the residual learning is introduced into the encoder and decoder to improve the efficiency of feature use and to reduce information loss, and a feature fusion module is introduced between the encoder and decoder to extract image features with different granularities; and finally, a feature balancing module is added to the skip connections to resolve the semantic gap between low-dimensional features in the encoder and high-dimensional features in decoder. Experimental results show that our method has better accuracy and sensitivity on the DRIVE and STARE datasets (accuracy (ACC) = 0.9611, sensitivity (SE) = 0.8613; STARE: ACC = 0.9662, SE = 0.7887) than some of the state-of-the-art methods.

Aiding the Diagnosis of Diabetic and Hypertensive Retinopathy Using Artificial Intelligence-Based Semantic Segmentation

Automatic segmentation of retinal images is an important task in computer-assisted medical image analysis for the diagnosis of diseases such as hypertension, diabetic and hypertensive retinopathy, and arteriosclerosis. Among the diseases, diabetic retinopathy, which is the leading cause of vision detachment, can be diagnosed early through the detection of retinal vessels. The manual detection of these retinal vessels is a time-consuming process that can be automated with the help of artificial intelligence with deep learning. The detection of vessels is difficult due to intensity variation and noise from non-ideal imaging. Although there are deep learning approaches for vessel segmentation, these methods require many trainable parameters, which increase the network complexity. To address these issues, this paper presents a dual-residual-stream-based vessel segmentation network (Vess-Net), which is not as deep as conventional semantic segmentation networks, but provides good segmentation with few trainable parameters and layers. The method takes advantage of artificial intelligence for semantic segmentation to aid the diagnosis of retinopathy. To evaluate the proposed Vess-Net method, experiments were conducted with three publicly available datasets for vessel segmentation: digital retinal images for vessel extraction (DRIVE), the Child Heart Health Study in England (CHASE-DB1), and structured analysis of retina (STARE). Experimental results show that Vess-Net achieved superior performance for all datasets with sensitivity (Se), specificity (Sp), area under the curve (AUC), and accuracy (Acc) of 80.22%, 98.1%, 98.2%, and 96.55% for DRVIE; 82.06%, 98.41%, 98.0%, and 97.26% for CHASE-DB1; and 85.26%, 97.91%, 98.83%, and 96.97% for STARE dataset.