PHF3 Technique: A Pyramid Hybrid Feature Fusion Framework for Severity Classification of Ulcerative Colitis Using Endoscopic Images

Machine Learning for Biomedical Applications

Machine learning (ML) is a field of artificial intelligence that uses algorithms capable of extracting knowledge directly from data that could support decisions in multiple fields of engineering [...].

UCFNNet: Ulcerative colitis evaluation based on fine-grained lesion learner and noise suppression gating

BACKGROUND AND OBJECTIVE

Ulcerative colitis (UC) is a chronic disease characterized by recurrent symptoms and significant morbidity. The exact cause of the disease remains unknown. The selection of current treatment options for ulcerative colitis depends on the severity and location of the disease in each patient. Therefore, developing a fully automated endoscopic images for evaluating UC is crucial for guiding treatment plans and facilitating early prevention efforts.

METHODS

We propose a network called ulcerative colitis evaluation based on fine-grained lesion learner and noise suppression gating (UCFNNet). UCFNNet contains three novel modules. Firstly, a fine-grained lesion feature learner (FG-LF Learner) is proposed by integrating local features and a Softmax category prediction (SCP) module to improve the feature accuracy in small lesion areas. Subsequently, a graph convolutional feature combiner (GCFC) is developed to connect features across adjacent convolutional layers and to incorporate short connections between input and output, thereby mitigating feature loss during transmission. Thereafter, a noise suppression gating (NS gating) technique is designed by implementing a grid attention mechanism and a feature gating (FG) module to prioritize significant lesion features and suppress irrelevant and noisy regions in the input feature map.

RESULTS

We evaluate the performance of the proposed network on both privately-collected and publicly-available datasets. The evaluation of UC achieves excellent results on privately-collected dataset, with an accuracy (ACC) of 89.57 %, Matthews correlation coefficient (MCC) of 85.52 %, precision of 89.26 %, recall of 89.48 %, and F1-score of 89.78 %. The results are also impressive on publicly-available dataset, with ACC of 85.47 %, MCC of 80.42 %, precision of 85.62 %, recall of 84.00 %, and F1-score of 84.53 %, surpassing the performance of state-of-the-art techniques.

CONCLUSION

Our proposed model introduces three innovative algorithm modules, which outperform the current state-of-the-art methods and achieve high ACC and F1-score. This indicates that our method has superior performance compared to traditional machine learning and existing deep methods, which means that our method has good application prospects. Meanwhile, it has been verified that the proposed model demonstrates good interpretability. The source code is available at github.com/YinLeRenNB/UCFNNet.

Development and validation of a deep learning-based approach to predict the Mayo endoscopic score of ulcerative colitis

Background

The ulcerative colitis (UC) Mayo endoscopy score is a useful tool for evaluating the severity of UC in patients in clinical practice.

Objectives

We aimed to develop and validate a deep learning-based approach to automatically predict the Mayo endoscopic score using UC endoscopic images.

Design

A multicenter, diagnostic retrospective study.

Methods

We collected 15120 colonoscopy images of 768 UC patients from two hospitals in China and developed a deep model based on a vision transformer named the UC-former. The performance of the UC-former was compared with that of six endoscopists on the internal test set. Furthermore, multicenter validation from three hospitals was also carried out to evaluate UC-former's generalization performance.

Results

On the internal test set, the areas under the curve of Mayo 0, Mayo 1, Mayo 2, and Mayo 3 achieved by the UC-former were 0.998, 0.984, 0.973, and 0.990, respectively. The accuracy (ACC) achieved by the UC-former was 90.8%, which is higher than that achieved by the best senior endoscopist. For three multicenter external validations, the ACC was 82.4%, 85.0%, and 83.6%, respectively.

Conclusions

The developed UC-former could achieve high ACC, fidelity, and stability to evaluate the severity of UC, which may provide potential application in clinical practice.

Registration

This clinical trial was registered at the ClinicalTrials.gov (trial registration number: NCT05336773).