Automated polyp segmentation in colonoscopy images via deep network with lesion-aware feature selection and refinement

Deep Learning-Based Semantic Segmentation for Objective Colonoscopy Quality Assessment

Background: This study aims to objectively evaluate the overall quality of colonoscopies using a specially trained deep learning-based semantic segmentation neural network. This represents a modern and valuable approach for the analysis of colonoscopy frames. Methods: We collected thousands of colonoscopy frames extracted from a set of video colonoscopy files. A color-based image processing method was used to extract color features from specific regions of each colonoscopy frame, namely, the intestinal mucosa, residues, artifacts, and lumen. With these features, we automatically annotated all the colonoscopy frames and then selected the best of them to train a semantic segmentation network. This trained network was used to classify the four region types in a different set of test colonoscopy frames and extract pixel statistics that are relevant to quality evaluation. The test colonoscopies were also evaluated by colonoscopy experts using the Boston scale. Results: The deep learning semantic segmentation method obtained good results, in terms of classifying the four key regions in colonoscopy frames, and produced pixel statistics that are efficient in terms of objective quality assessment. The Spearman correlation results were as follows: BBPS vs. pixel scores: 0.69; BBPS vs. mucosa pixel percentage: 0.63; BBPS vs. residue pixel percentage: -0.47; BBPS vs. Artifact Pixel Percentage: -0.65. The agreement analysis using Cohen's Kappa yielded a value of 0.28. The colonoscopy evaluation based on the extracted pixel statistics showed a fair level of compatibility with the experts' evaluations. Conclusions: Our proposed deep learning semantic segmentation approach is shown to be a promising tool for evaluating the overall quality of colonoscopies and goes beyond the Boston Bowel Preparation Scale in terms of assessing colonoscopy quality. In particular, while the Boston scale focuses solely on the amount of residual content, our method can identify and quantify the percentage of colonic mucosa, residues, and artifacts, providing a more comprehensive and objective evaluation.

PFPRNet: A Phase-Wise Feature Pyramid With Retention Network for Polyp Segmentation

Early detection of colonic polyps is crucial for the prevention and diagnosis of colorectal cancer. Currently, deep learning-based polyp segmentation methods have become mainstream and achieved remarkable results. Acquiring a large number of labeled data is time-consuming and labor-intensive, and meanwhile the presence of numerous similar wrinkles in polyp images also hampers model prediction performance. In this paper, we propose a novel approach called Phase-wise Feature Pyramid with Retention Network (PFPRNet), which leverages a pre-trained Transformer-based Encoder to obtain multi-scale feature maps. A Phase-wise Feature Pyramid with Retention Decoder is designed to gradually integrate global features into local features and guide the model's attention towards key regions. Additionally, our custom Enhance Perception module enables capturing image information from a broader perspective. Finally, we introduce an innovative Low-layer Retention module as an alternative to Transformer for more efficient global attention modeling. Evaluation results on several widely-used polyp segmentation datasets demonstrate that our proposed method has strong learning ability and generalization capability, and outperforms the state-of-the-art approaches.

An Edge-Enhanced Network for Polyp Segmentation

Colorectal cancer remains a leading cause of cancer-related deaths worldwide, with early detection and removal of polyps being critical in preventing disease progression. Automated polyp segmentation, particularly in colonoscopy images, is a challenging task due to the variability in polyp appearance and the low contrast between polyps and surrounding tissues. In this work, we propose an edge-enhanced network (EENet) designed to address these challenges by integrating two novel modules: the covariance edge-enhanced attention (CEEA) and cross-scale edge enhancement (CSEE) modules. The CEEA module leverages covariance-based attention to enhance boundary detection, while the CSEE module bridges multi-scale features to preserve fine-grained edge details. To further improve the accuracy of polyp segmentation, we introduce a hybrid loss function that combines cross-entropy loss with edge-aware loss. Extensive experiments show that the EENet achieves a Dice score of 0.9208 and an IoU of 0.8664 on the Kvasir-SEG dataset, surpassing state-of-the-art models such as Polyp-PVT and PraNet. Furthermore, it records a Dice score of 0.9316 and an IoU of 0.8817 on the CVC-ClinicDB dataset, demonstrating its strong potential for clinical application in polyp segmentation. Ablation studies further validate the contribution of the CEEA and CSEE modules.

SR-AttNet: An Interpretable Stretch-Relax Attention based Deep Neural Network for Polyp Segmentation in Colonoscopy Images

BACKGROUND

Colorectal polyp is a common structural gastrointestinal (GI) anomaly, which can in certain cases turn malignant. Colonoscopic image inspection is, thereby, an important step for isolating the polyps as well as removing them if necessary. However, the process is around 30-60 min long and inspecting each image for polyps can prove to be a tedious task. Hence, an automatic computerized process for efficient and accurate polyp isolation can be a useful tool.

METHODS

In this study, a deep learning network is introduced for colorectal polyp segmentation. The network is based on an encoder-decoder architecture, however, having both un-dilated and dilated filtering in order to extract both near and far local information as well as perceive image depth. Four-fold skip-connections exist between each spatial encoder-decoder due to both type of filtering and a 'Feature-to-Mask' pipeline processes the decoded dilated and un-dilated features for final prediction. The proposed network implements a 'Stretch-Relax' based attention system, SR-Attention, to generate high variance spatial features in order to obtain useful attention masks for cognitive feature selection. From this 'Stretch-Relax' attention based operation, the network is termed as 'SR-AttNet'.

RESULTS

Training and optimization is performed on four different datasets, and inference has been done on five (Kvasir-SEG, CVC-ClinicDB, CVC-Colon, ETIS-Larib, EndoCV2020); all of which output higher Dice-score compared to state-of-the-art and existing networks. The efficacy and interpretability of SR-Attention is also demonstrated based on quantitative variance.

CONCLUSION

In consequence, the proposed SR-AttNet can be considered for an automated and general approach for polyp segmentation during colonoscopy.