1
|
Huang X, Wang L, Jiang S, Xu L. DHAFormer: Dual-channel hybrid attention network with transformer for polyp segmentation. PLoS One 2024; 19:e0306596. [PMID: 38985710 PMCID: PMC11236112 DOI: 10.1371/journal.pone.0306596] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 06/17/2024] [Indexed: 07/12/2024] Open
Abstract
The accurate early diagnosis of colorectal cancer significantly relies on the precise segmentation of polyps in medical images. Current convolution-based and transformer-based segmentation methods show promise but still struggle with the varied sizes and shapes of polyps and the often low contrast between polyps and their background. This research introduces an innovative approach to confronting the aforementioned challenges by proposing a Dual-Channel Hybrid Attention Network with Transformer (DHAFormer). Our proposed framework features a multi-scale channel fusion module, which excels at recognizing polyps across a spectrum of sizes and shapes. Additionally, the framework's dual-channel hybrid attention mechanism is innovatively conceived to reduce background interference and improve the foreground representation of polyp features by integrating local and global information. The DHAFormer demonstrates significant improvements in the task of polyp segmentation compared to currently established methodologies.
Collapse
Affiliation(s)
- Xuejie Huang
- School of Computer Science and Technology, Xinjiang University, Urumqi, China
| | - Liejun Wang
- School of Computer Science and Technology, Xinjiang University, Urumqi, China
| | - Shaochen Jiang
- School of Computer Science and Technology, Xinjiang University, Urumqi, China
| | - Lianghui Xu
- School of Computer Science and Technology, Xinjiang University, Urumqi, China
| |
Collapse
|
2
|
K P AG, D RR, N MS, P LB. Gastrointestinal tract disease detection via deep learning based structural and statistical features optimized hexa-classification model. Technol Health Care 2024:THC240603. [PMID: 39031411 DOI: 10.3233/thc-240603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/22/2024]
Abstract
BACKGROUND Gastrointestinal tract (GIT) diseases impact the entire digestive system, spanning from the mouth to the anus. Wireless Capsule Endoscopy (WCE) stands out as an effective analytic instrument for Gastrointestinal tract diseases. Nevertheless, accurately identifying various lesion features, such as irregular sizes, shapes, colors, and textures, remains challenging in this field. OBJECTIVE Several computer vision algorithms have been introduced to tackle these challenges, but many relied on handcrafted features, resulting in inaccuracies in various instances. METHODS In this work, a novel Deep SS-Hexa model is proposed which is a combination two different deep learning structures for extracting two different features from the WCE images to detect various GIT ailment. The gathered images are denoised by weighted median filter to remove the noisy distortions and augment the images for enhancing the training data. The structural and statistical (SS) feature extraction process is sectioned into two phases for the analysis of distinct regions of gastrointestinal. In the first stage, statistical features of the image are retrieved using MobileNet with the support of SiLU activation function to retrieve the relevant features. In the second phase, the segmented intestine images are transformed into structural features to learn the local information. These SS features are parallelly fused for selecting the best relevant features with walrus optimization algorithm. Finally, Deep belief network (DBN) is used classified the GIT diseases into hexa classes namely normal, ulcer, pylorus, cecum, esophagitis and polyps on the basis of the selected features. RESULTS The proposed Deep SS-Hexa model attains an overall average accuracy of 99.16% in GIT disease detection based on KVASIR and KID datasets. The proposed Deep SS-Hexa model achieves high level of accuracy with minimal computational cost in the recognition of GIT illness. CONCLUSIONS The proposed Deep SS-Hexa Model progresses the overall accuracy range of 0.04%, 0.80% better than GastroVision, Genetic algorithm based on KVASIR dataset and 0.60%, 1.21% better than Modified U-Net, WCENet based on KID dataset respectively.
Collapse
Affiliation(s)
- Ajitha Gladis K P
- Department of Information Technology, CSI Institute of Technology, Thovalai, India
| | - Roja Ramani D
- Department of Computer Science and Engineering, New Horizon College of Engineering, Bengaluru, India
| | - Mohana Suganthi N
- Department of Computer Science and Engineering, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Chennai, India
| | - Linu Babu P
- Department of Electronics and Communication Engineering, IES College of Engineering, Thrissur, India
| |
Collapse
|
3
|
Oukdach Y, Garbaz A, Kerkaou Z, El Ansari M, Koutti L, El Ouafdi AF, Salihoun M. UViT-Seg: An Efficient ViT and U-Net-Based Framework for Accurate Colorectal Polyp Segmentation in Colonoscopy and WCE Images. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01124-8. [PMID: 38671336 DOI: 10.1007/s10278-024-01124-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 04/01/2024] [Accepted: 04/13/2024] [Indexed: 04/28/2024]
Abstract
Colorectal cancer (CRC) stands out as one of the most prevalent global cancers. The accurate localization of colorectal polyps in endoscopy images is pivotal for timely detection and removal, contributing significantly to CRC prevention. The manual analysis of images generated by gastrointestinal screening technologies poses a tedious task for doctors. Therefore, computer vision-assisted cancer detection could serve as an efficient tool for polyp segmentation. Numerous efforts have been dedicated to automating polyp localization, with the majority of studies relying on convolutional neural networks (CNNs) to learn features from polyp images. Despite their success in polyp segmentation tasks, CNNs exhibit significant limitations in precisely determining polyp location and shape due to their sole reliance on learning local features from images. While gastrointestinal images manifest significant variation in their features, encompassing both high- and low-level ones, a framework that combines the ability to learn both features of polyps is desired. This paper introduces UViT-Seg, a framework designed for polyp segmentation in gastrointestinal images. Operating on an encoder-decoder architecture, UViT-Seg employs two distinct feature extraction methods. A vision transformer in the encoder section captures long-range semantic information, while a CNN module, integrating squeeze-excitation and dual attention mechanisms, captures low-level features, focusing on critical image regions. Experimental evaluations conducted on five public datasets, including CVC clinic, ColonDB, Kvasir-SEG, ETIS LaribDB, and Kvasir Capsule-SEG, demonstrate UViT-Seg's effectiveness in polyp localization. To confirm its generalization performance, the model is tested on datasets not used in training. Benchmarking against common segmentation methods and state-of-the-art polyp segmentation approaches, the proposed model yields promising results. For instance, it achieves a mean Dice coefficient of 0.915 and a mean intersection over union of 0.902 on the CVC Colon dataset. Furthermore, UViT-Seg has the advantage of being efficient, requiring fewer computational resources for both training and testing. This feature positions it as an optimal choice for real-world deployment scenarios.
Collapse
Affiliation(s)
- Yassine Oukdach
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco.
| | - Anass Garbaz
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Zakaria Kerkaou
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Mohamed El Ansari
- Informatics and Applications Laboratory, Department of Computer Sciences, Faculty of Science, Moulay Ismail University, B.P 11201, Meknès, 52000, Morocco
| | - Lahcen Koutti
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Ahmed Fouad El Ouafdi
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Mouna Salihoun
- Faculty of Medicine and Pharmacy of Rabat, Mohammed V University of Rabat, Rabat, 10000, Morocco
| |
Collapse
|
4
|
Guo X, Xu L, Li S, Xu M, Chu Y, Jiang Q. Cascade-EC Network: Recognition of Gastrointestinal Multiple Lesions Based on EfficientNet and CA_stm_Retinanet. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01096-9. [PMID: 38587768 DOI: 10.1007/s10278-024-01096-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 03/17/2024] [Accepted: 03/20/2024] [Indexed: 04/09/2024]
Abstract
Capsule endoscopy (CE) is non-invasive and painless during gastrointestinal examination. However, capsule endoscopy can increase the workload of image reviewing for clinicians, making it prone to missed and misdiagnosed diagnoses. Current researches primarily concentrated on binary classifiers, multiple classifiers targeting fewer than four abnormality types and detectors within a specific segment of the digestive tract, and segmenters for a single type of anomaly. Due to intra-class variations, the task of creating a unified scheme for detecting multiple gastrointestinal diseases is particularly challenging. A cascade neural network designed in this study, Cascade-EC, can automatically identify and localize four types of gastrointestinal lesions in CE images: angiectasis, bleeding, erosion, and polyp. Cascade-EC consists of EfficientNet for image classification and CA_stm_Retinanet for lesion detection and location. As the first layer of Cascade-EC, the EfficientNet network classifies CE images. CA_stm_Retinanet, as the second layer, performs the target detection and location task on the classified image. CA_stm_Retinanet adopts the general architecture of Retinanet. Its feature extraction module is the CA_stm_Backbone from the stack of CA_stm Block. CA_stm Block adopts the split-transform-merge strategy and introduces the coordinate attention. The dataset in this study is from Shanghai East Hospital, collected by PillCam SB3 and AnKon capsule endoscopes, which contains a total of 7936 images of 317 patients from the years 2017 to 2021. In the testing set, the average precision of Cascade-EC in the multi-lesions classification task was 94.55%, the average recall was 90.60%, and the average F1 score was 92.26%. The mean mAP@ 0.5 of Cascade-EC for detecting the four types of diseases is 85.88%. The experimental results show that compared with a single target detection network, Cascade-EC has better performance and can effectively assist clinicians to classify and detect multiple lesions in CE images.
Collapse
Affiliation(s)
- Xudong Guo
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China.
| | - Lei Xu
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Shengnan Li
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Meidong Xu
- Endoscopy Center, Department of Gastroenterology, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, China.
| | - Yuan Chu
- Endoscopy Center, Department of Gastroenterology, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, China
| | - Qinfen Jiang
- Department of Information Management, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, China
| |
Collapse
|
5
|
Radhachandran A, Kinzel A, Chen J, Sant V, Patel M, Masamed R, Arnold CW, Speier W. A multitask approach for automated detection and segmentation of thyroid nodules in ultrasound images. Comput Biol Med 2024; 170:107974. [PMID: 38244471 PMCID: PMC11354840 DOI: 10.1016/j.compbiomed.2024.107974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/06/2023] [Accepted: 01/02/2024] [Indexed: 01/22/2024]
Abstract
An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9888 images, and annotated by collaborating radiologists. Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any UNet architecture variant to improve image-level nodule detection. Of the evaluated multitask models, a UNet with a ImageNet pretrained encoder and AD achieved the highest F1 score of 0.839 and image-wide Dice similarity coefficient of 0.808 on the hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.
Collapse
Affiliation(s)
- Ashwath Radhachandran
- Computational Diagnostics Lab, University of California, Los Angeles, Los Angeles, CA, USA; Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA
| | - Adam Kinzel
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Joseph Chen
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Vivek Sant
- Division of Endocrine Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Maitraya Patel
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Rinat Masamed
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Corey W Arnold
- Computational Diagnostics Lab, University of California, Los Angeles, Los Angeles, CA, USA; Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA; Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA; Department of Pathology and Laboratory Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - William Speier
- Computational Diagnostics Lab, University of California, Los Angeles, Los Angeles, CA, USA; Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|
6
|
Wang H, Hu T, Zhang Y, Zhang H, Qi Y, Wang L, Ma J, Du M. Unveiling camouflaged and partially occluded colorectal polyps: Introducing CPSNet for accurate colon polyp segmentation. Comput Biol Med 2024; 171:108186. [PMID: 38394804 DOI: 10.1016/j.compbiomed.2024.108186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 02/02/2024] [Accepted: 02/18/2024] [Indexed: 02/25/2024]
Abstract
BACKGROUND Segmenting colorectal polyps presents a significant challenge due to the diverse variations in their size, shape, texture, and intricate backgrounds. Particularly demanding are the so-called "camouflaged" polyps, which are partially concealed by surrounding tissues or fluids, adding complexity to their detection. METHODS We present CPSNet, an innovative model designed for camouflaged polyp segmentation. CPSNet incorporates three key modules: the Deep Multi-Scale-Feature Fusion Module, the Camouflaged Object Detection Module, and the Multi-Scale Feature Enhancement Module. These modules work collaboratively to improve the segmentation process, enhancing both robustness and accuracy. RESULTS Our experiments confirm the effectiveness of CPSNet. When compared to state-of-the-art methods in colon polyp segmentation, CPSNet consistently outperforms the competition. Particularly noteworthy is its performance on the ETIS-LaribPolypDB dataset, where CPSNet achieved a remarkable 2.3% increase in the Dice coefficient compared to the Polyp-PVT model. CONCLUSION In summary, CPSNet marks a significant advancement in the field of colorectal polyp segmentation. Its innovative approach, encompassing multi-scale feature fusion, camouflaged object detection, and feature enhancement, holds considerable promise for clinical applications.
Collapse
Affiliation(s)
- Huafeng Wang
- School of Information Technology, North China University of Technology, Beijing 100041, China.
| | - Tianyu Hu
- School of Information Technology, North China University of Technology, Beijing 100041, China.
| | - Yanan Zhang
- School of Information Technology, North China University of Technology, Beijing 100041, China.
| | - Haodu Zhang
- School of Intelligent Systems Engineering, Sun Yat-sen University, Guangzhou 510335, China.
| | - Yong Qi
- School of Information Technology, North China University of Technology, Beijing 100041, China.
| | - Longzhen Wang
- Department of Gastroenterology, Second People's Hospital, Changzhi, Shanxi 046000, China.
| | - Jianhua Ma
- School of Biomedical Engineering, Southern Medical University, Guangzhou 510335, China.
| | - Minghua Du
- Department of Emergency, PLA General Hospital, Beijing 100853, China.
| |
Collapse
|
7
|
Wang G, Bai L, Wu Y, Chen T, Ren H. Rethinking exemplars for continual semantic segmentation in endoscopy scenes: Entropy-based mini-batch pseudo-replay. Comput Biol Med 2023; 165:107412. [PMID: 37696180 DOI: 10.1016/j.compbiomed.2023.107412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 08/05/2023] [Accepted: 08/27/2023] [Indexed: 09/13/2023]
Abstract
Endoscopy is a widely used technique for the early detection of diseases or robotic-assisted minimally invasive surgery (RMIS). Numerous deep learning (DL)-based research works have been developed for automated diagnosis or processing of endoscopic view. However, existing DL models may suffer from catastrophic forgetting. When new target classes are introduced over time or cross institutions, the performance of old classes may suffer severe degradation. More seriously, data privacy and storage issues may lead to the unavailability of old data when updating the model. Therefore, it is necessary to develop a continual learning (CL) methodology to solve the problem of catastrophic forgetting in endoscopic image segmentation. To tackle this, we propose a Endoscopy Continual Semantic Segmentation (EndoCSS) framework that does not involve the storage and privacy issues of exemplar data. The framework includes a mini-batch pseudo-replay (MB-PR) mechanism and a self-adaptive noisy cross-entropy (SAN-CE) loss. The MB-PR strategy circumvents privacy and storage issues by generating pseudo-replay images through a generative model. Meanwhile, the MB-PR strategy can also correct the model deviation to the replay data and current training data, which is aroused by the significant difference in the amount of current and replay images. Therefore, the model can perform effective representation learning on both new and old tasks. SAN-CE loss can help model fitting by adjusting the model's output logits, and also improve the robustness of training. Extensive continual semantic segmentation (CSS) experiments on public datasets demonstrate that our method can robustly and effectively address the catastrophic forgetting brought by class increment in endoscopy scenes. The results show that our framework holds excellent potential for real-world deployment in a streaming learning manner.
Collapse
Affiliation(s)
- Guankun Wang
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China.
| | - Long Bai
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China.
| | - Yanan Wu
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China; College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China.
| | - Tong Chen
- School of Electrical and Information Engineering, The University of Sydney, Sydney, Australia.
| | - Hongliang Ren
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China; Department of Biomedical Engineering, National University of Singapore, Singapore; Suzhou Research Institute, National University of Singapore, Suzhou, China; Shun Hing Institute of Advanced Engineering, The Chinese University of Hong Kong, Hong Kong, China; Shenzhen Research Institute, The Chinese University of, Hong Kong, Shenzhen, China.
| |
Collapse
|
8
|
Quindós A, Laiz P, Vitrià J, Seguí S. Self-supervised out-of-distribution detection in wireless capsule endoscopy images. Artif Intell Med 2023; 143:102606. [PMID: 37673575 DOI: 10.1016/j.artmed.2023.102606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 06/02/2023] [Accepted: 06/05/2023] [Indexed: 09/08/2023]
Abstract
While deep learning has displayed excellent performance in a broad spectrum of application areas, neural networks still struggle to recognize what they have not seen, i.e., out-of-distribution (OOD) inputs. In the medical field, building robust models that are able to detect OOD images is highly critical, as these rare images could show diseases or anomalies that should be detected. In this study, we use wireless capsule endoscopy (WCE) images to present a novel patch-based self-supervised approach comprising three stages. First, we train a triplet network to learn vector representations of WCE image patches. Second, we cluster the patch embeddings to group patches in terms of visual similarity. Third, we use the cluster assignments as pseudolabels to train a patch classifier and use the Out-of-Distribution Detector for Neural Networks (ODIN) for OOD detection. The system has been tested on the Kvasir-capsule, a publicly released WCE dataset. Empirical results show an OOD detection improvement compared to baseline methods. Our method can detect unseen pathologies and anomalies such as lymphangiectasia, foreign bodies and blood with AUROC>0.6. This work presents an effective solution for OOD detection models without needing labeled images.
Collapse
Affiliation(s)
- Arnau Quindós
- Departament de Matemàtiques i Informàtica, Universitat de Barcelona (UB), Barcelona, Spain
| | - Pablo Laiz
- Departament de Matemàtiques i Informàtica, Universitat de Barcelona (UB), Barcelona, Spain
| | - Jordi Vitrià
- Departament de Matemàtiques i Informàtica, Universitat de Barcelona (UB), Barcelona, Spain
| | - Santi Seguí
- Departament de Matemàtiques i Informàtica, Universitat de Barcelona (UB), Barcelona, Spain.
| |
Collapse
|
9
|
Lin CH, Hsu PI, Tseng CD, Chao PJ, Wu IT, Ghose S, Shih CA, Lee SH, Ren JH, Shie CB, Lee TF. Application of artificial intelligence in endoscopic image analysis for the diagnosis of a gastric cancer pathogen-Helicobacter pylori infection. Sci Rep 2023; 13:13380. [PMID: 37592004 PMCID: PMC10435453 DOI: 10.1038/s41598-023-40179-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 08/06/2023] [Indexed: 08/19/2023] Open
Abstract
Helicobacter pylori (H. pylori) infection is the principal cause of chronic gastritis, gastric ulcers, duodenal ulcers, and gastric cancer. In clinical practice, diagnosis of H. pylori infection by a gastroenterologists' impression of endoscopic images is inaccurate and cannot be used for the management of gastrointestinal diseases. The aim of this study was to develop an artificial intelligence classification system for the diagnosis of H. pylori infection by pre-processing endoscopic images and machine learning methods. Endoscopic images of the gastric body and antrum from 302 patients receiving endoscopy with confirmation of H. pylori status by a rapid urease test at An Nan Hospital were obtained for the derivation and validation of an artificial intelligence classification system. The H. pylori status was interpreted as positive or negative by Convolutional Neural Network (CNN) and Concurrent Spatial and Channel Squeeze and Excitation (scSE) network, combined with different classification models for deep learning of gastric images. The comprehensive assessment for H. pylori status by scSE-CatBoost classification models for both body and antrum images from same patients achieved an accuracy of 0.90, sensitivity of 1.00, specificity of 0.81, positive predictive value of 0.82, negative predicted value of 1.00, and area under the curve of 0.88. The data suggest that an artificial intelligence classification model using scSE-CatBoost deep learning for gastric endoscopic images can distinguish H. pylori status with good performance and is useful for the survey or diagnosis of H. pylori infection in clinical practice.
Collapse
Affiliation(s)
- Chih-Hsueh Lin
- Department of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
- Medical Physics and Informatics Laboratory of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
| | - Ping-I Hsu
- Division of Gastroenterology, Department of Medicine, An Nan Hospital, China Medical University, Tainan, Taiwan
| | - Chin-Dar Tseng
- Department of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan.
- Medical Physics and Informatics Laboratory of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan.
| | - Pei-Ju Chao
- Department of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
- Medical Physics and Informatics Laboratory of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
| | - I-Ting Wu
- Division of Gastroenterology, Department of Medicine, An Nan Hospital, China Medical University, Tainan, Taiwan
| | - Supratip Ghose
- Department of Education and Research, An Nan Hospital, China Medical University, Tainan, Taiwan
| | - Chih-An Shih
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Antai Medical Care Corporation, Antai Tian-Sheng Memorial Hospital, Donggan, Pingtung County, Taiwan
- Department of Nursing, Meiho University, Neipu, Pingtung County, Taiwan
| | - Shen-Hao Lee
- Department of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
- Medical Physics and Informatics Laboratory of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
- Department of Radiation Oncology, Linkou Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Linkou, Taiwan
| | - Jia-Hong Ren
- Department of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
- Medical Physics and Informatics Laboratory of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan
| | - Chang-Bih Shie
- Division of Gastroenterology, Department of Medicine, An Nan Hospital, China Medical University, Tainan, Taiwan
| | - Tsair-Fwu Lee
- Department of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan.
- Medical Physics and Informatics Laboratory of Electronic Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, 80778, Taiwan.
- Department of Medical Imaging and Radiological Sciences, Kaohsiung Medical University, Kaohsiung, 80708, Taiwan.
- PhD Program in Biomedical Engineering, Kaohsiung Medical University, Kaohsiung, 80708, Taiwan.
- School of Dentistry, College of Dental Medicine, Kaohsiung Medical University, Kaohsiung, 80708, Taiwan.
| |
Collapse
|
10
|
Musha A, Hasnat R, Mamun AA, Ping EP, Ghosh T. Computer-Aided Bleeding Detection Algorithms for Capsule Endoscopy: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:7170. [PMID: 37631707 PMCID: PMC10459126 DOI: 10.3390/s23167170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2023] [Revised: 08/08/2023] [Accepted: 08/10/2023] [Indexed: 08/27/2023]
Abstract
Capsule endoscopy (CE) is a widely used medical imaging tool for the diagnosis of gastrointestinal tract abnormalities like bleeding. However, CE captures a huge number of image frames, constituting a time-consuming and tedious task for medical experts to manually inspect. To address this issue, researchers have focused on computer-aided bleeding detection systems to automatically identify bleeding in real time. This paper presents a systematic review of the available state-of-the-art computer-aided bleeding detection algorithms for capsule endoscopy. The review was carried out by searching five different repositories (Scopus, PubMed, IEEE Xplore, ACM Digital Library, and ScienceDirect) for all original publications on computer-aided bleeding detection published between 2001 and 2023. The Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) methodology was used to perform the review, and 147 full texts of scientific papers were reviewed. The contributions of this paper are: (I) a taxonomy for computer-aided bleeding detection algorithms for capsule endoscopy is identified; (II) the available state-of-the-art computer-aided bleeding detection algorithms, including various color spaces (RGB, HSV, etc.), feature extraction techniques, and classifiers, are discussed; and (III) the most effective algorithms for practical use are identified. Finally, the paper is concluded by providing future direction for computer-aided bleeding detection research.
Collapse
Affiliation(s)
- Ahmmad Musha
- Department of Electrical and Electronic Engineering, Pabna University of Science and Technology, Pabna 6600, Bangladesh; (A.M.); (R.H.)
| | - Rehnuma Hasnat
- Department of Electrical and Electronic Engineering, Pabna University of Science and Technology, Pabna 6600, Bangladesh; (A.M.); (R.H.)
| | - Abdullah Al Mamun
- Faculty of Engineering and Technology, Multimedia University, Melaka 75450, Malaysia;
| | - Em Poh Ping
- Faculty of Engineering and Technology, Multimedia University, Melaka 75450, Malaysia;
| | - Tonmoy Ghosh
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA;
| |
Collapse
|
11
|
Alam MJ, Fattah SA. SR-AttNet: An Interpretable Stretch-Relax Attention based Deep Neural Network for Polyp Segmentation in Colonoscopy Images. Comput Biol Med 2023; 160:106945. [PMID: 37163966 DOI: 10.1016/j.compbiomed.2023.106945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 04/07/2023] [Accepted: 04/14/2023] [Indexed: 05/12/2023]
Abstract
BACKGROUND Colorectal polyp is a common structural gastrointestinal (GI) anomaly, which can in certain cases turn malignant. Colonoscopic image inspection is, thereby, an important step for isolating the polyps as well as removing them if necessary. However, the process is around 30-60 min long and inspecting each image for polyps can prove to be a tedious task. Hence, an automatic computerized process for efficient and accurate polyp isolation can be a useful tool. METHODS In this study, a deep learning network is introduced for colorectal polyp segmentation. The network is based on an encoder-decoder architecture, however, having both un-dilated and dilated filtering in order to extract both near and far local information as well as perceive image depth. Four-fold skip-connections exist between each spatial encoder-decoder due to both type of filtering and a 'Feature-to-Mask' pipeline processes the decoded dilated and un-dilated features for final prediction. The proposed network implements a 'Stretch-Relax' based attention system, SR-Attention, to generate high variance spatial features in order to obtain useful attention masks for cognitive feature selection. From this 'Stretch-Relax' attention based operation, the network is termed as 'SR-AttNet'. RESULTS Training and optimization is performed on four different datasets, and inference has been done on five (Kvasir-SEG, CVC-ClinicDB, CVC-Colon, ETIS-Larib, EndoCV2020); all of which output higher Dice-score compared to state-of-the-art and existing networks. The efficacy and interpretability of SR-Attention is also demonstrated based on quantitative variance. CONCLUSION In consequence, the proposed SR-AttNet can be considered for an automated and general approach for polyp segmentation during colonoscopy.
Collapse
Affiliation(s)
- Md Jahin Alam
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1205, Bangladesh.
| | - Shaikh Anowarul Fattah
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1205, Bangladesh.
| |
Collapse
|
12
|
Horovistiz A, Oliveira M, Araújo H. Computer vision-based solutions to overcome the limitations of wireless capsule endoscopy. J Med Eng Technol 2023; 47:242-261. [PMID: 38231042 DOI: 10.1080/03091902.2024.2302025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 12/28/2023] [Indexed: 01/18/2024]
Abstract
Endoscopic investigation plays a critical role in the diagnosis of gastrointestinal (GI) diseases. Since 2001, Wireless Capsule Endoscopy (WCE) has been available for small bowel exploration and is in continuous development. Over the last decade, WCE has achieved impressive improvements in areas such as miniaturisation, image quality and battery life. As a result, WCE is currently a very useful alternative to wired enteroscopy in the investigation of various small bowel abnormalities and has the potential to become the leading screening technique for the entire gastrointestinal tract. However, commercial solutions still have several limitations, namely incomplete examination and limited diagnostic capacity. These deficiencies are related to technical issues, such as image quality, motion estimation and power consumption management. Computational methods, based on image processing and analysis, can help to overcome these challenges and reduce both the time required by reviewers and human interpretation errors. Research groups have proposed a series of methods including algorithms for locating the capsule or lesion, assessing intestinal motility and improving image quality.In this work, we provide a critical review of computational vision-based methods for WCE image analysis aimed at overcoming the technological challenges of capsules. This article also reviews several representative public datasets used to evaluate the performance of WCE techniques and methods. Finally, some promising solutions of computational methods based on the analysis of multiple-camera endoscopic images are presented.
Collapse
Affiliation(s)
- Ana Horovistiz
- Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal
| | - Marina Oliveira
- Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal
- Department of Electrical and Computer Engineering (DEEC), Faculty of Sciences and Technology, University of Coimbra, Coimbra, Portugal
| | - Helder Araújo
- Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal
- Department of Electrical and Computer Engineering (DEEC), Faculty of Sciences and Technology, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
13
|
Radhachandran A, Kinzel A, Chen J, Sant V, Patel M, Masamed R, Arnold CW, Speier W. A Multitask Approach for Automated Detection and Segmentation of Thyroid Nodules in Ultrasound Images. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.01.31.23285223. [PMID: 36778410 PMCID: PMC9915831 DOI: 10.1101/2023.01.31.23285223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification.Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any U-Net architecture variant to improve image-level nodule detection. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9,888 images, and annotated by collaborating radiologists. Of the evaluated models, a multi-scale UNet (MSUNet) with AD achieved the highest F1 score of 0.829 and image-wide Dice similarity coefficient of 0.782 on our hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.
Collapse
Affiliation(s)
- Ashwath Radhachandran
- Computational Diagnostics Lab and Department of Bioengineering at the University of California, Los Angeles. The Computational Diagnostics Lab is located at 924 Westwood Blvd, Ste 420, Los Angeles, CA 90024, USA
| | - Adam Kinzel
- Department of Radiology at the University of California, Los Angeles
| | - Joseph Chen
- Department of Radiology at the University of California, Los Angeles
| | - Vivek Sant
- Section of Endocrine Surgery in the Department of Surgery at the University of California, Los Angeles
| | - Maitraya Patel
- Department of Radiology at the University of California, Los Angeles
| | - Rinat Masamed
- Department of Radiology at the University of California, Los Angeles
| | - Corey W Arnold
- Computational Diagnostics Lab, Department of Bioengineering, Department of Radiology and Department of Pathology and Laboratory Medicine at the University of California, Los Angeles
| | - William Speier
- Computational Diagnostics Lab and Department of Bioengineering at the University of California, Los Angeles. The Computational Diagnostics Lab is located at 924 Westwood Blvd, Ste 420, Los Angeles, CA 90024, USA
| |
Collapse
|
14
|
Falt P, Uricová D, Fejfar T, Šembera Š, Tachecí I. News in gastroenterology, hepatology and digestive endoscopy. VNITRNI LEKARSTVI 2023; 69:198-206. [PMID: 37468316 DOI: 10.36290/vnl.2023.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/21/2023]
Abstract
Gastroenterology, hepatology and digestive endoscopy are rapidly evolving disciplines with significant advances in the diagnostics and treatment in the entire gastrointestinal tract. The aim of our article was to summarize new perspectives on relevant situations in gastroenterology and hepatology like acute pancreatitis, functional dyspepsia, rational indication of proton pump inhibitors, inflammatory bowel diseases (IBD), cholestatic liver diseases, alcohol induced hepatitis, non-alcoholic fatty live disease (NAFLD) and patophysiology of bilirubin and bile acids. Digestive endoscopy represents an interventional part of gastroenterology and key recent topics are mentioned like pancreatic cancer screening, arteficial intelligence, resection of low-risk neoplastic lesions, enteroscopy techniques, cholangio- and pancreatiscopy and extraluminal expansion of endoscopy techniques by means of endoscopic submucosal and transmural dissection, endoscopic myotomy and lumen apposing stents.
Collapse
|
15
|
Liu F, Hua Z, Li J, Fan L. DBMF: Dual Branch Multiscale Feature Fusion Network for polyp segmentation. Comput Biol Med 2022; 151:106304. [PMID: 36401969 DOI: 10.1016/j.compbiomed.2022.106304] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 10/18/2022] [Accepted: 11/06/2022] [Indexed: 11/10/2022]
Abstract
Accurate and reliable segmentation of colorectal polyps is important for the diagnosis and treatment of colorectal cancer. Most of the existing polyp segmentation methods innovatively combine CNN with Transformer. Due to the single combination approach, there are limitations in establishing connections between local feature information and utilizing global contextual information captured by Transformer. Still not a better solution to the problems in polyp segmentation. In this paper, we propose a Dual Branch Multiscale Feature Fusion Network for Polyp Segmentation, abbreviated as DBMF, for polyp segmentation to achieve accurate segmentation of polyps. DBMF uses CNN and Transformer in parallel to extract multi-scale local information and global contextual information respectively, with different regions and levels of information to make the network more accurate in identifying polyps and their surrounding tissues. Feature Super Decoder (FSD) fuses multi-level local features and global contextual information in dual branches to fully exploit the potential of combining CNN and Transformer to improve the network's ability to parse complex scenes and the detection rate of tiny polyps. The FSD generates an initial segmentation map to guide the second parallel decoder (SPD) to refine the segmentation boundary layer by layer. SPD consists of a multi-scale feature aggregation module (MFA) and parallel polarized self-attention (PSA) and reverse attention fusion modules (RAF). MFA aggregates multi-level local feature information extracted by CNN Brach to find consensus regions between multiple scales and improve the network's ability to identify polyp regions. PSA uses dual attention to enhance the fine-grained nature of segmented regions and reduce the redundancy introduced by MFA and interference information. RAF mines boundary cues and establishes relationships between regions and boundary cues. The three RAFs guide the network to explore lost targets and boundaries in a bottom-up manner. We used the CVC-ClinicDB, Kvasir, CVC-300, CVC-ColonDB, and ETIS datasets to conduct comparison experiments and ablation experiments between DBMF and mainstream polyp segmentation networks. The results showed that DBMF outperformed the current mainstream networks on five benchmark datasets.
Collapse
Affiliation(s)
- Fangjin Liu
- Co-Innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, School of Information and Electronic Engineering, Shandong Technology and Business University, Laishan District, Yantai, 264005, China
| | - Zhen Hua
- Co-Innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, School of Information and Electronic Engineering, Shandong Technology and Business University, Laishan District, Yantai, 264005, China.
| | - Jinjiang Li
- Co-Innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, School of Information and Electronic Engineering, Shandong Technology and Business University, Laishan District, Yantai, 264005, China
| | - Linwei Fan
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, Shandong, 250014, China
| |
Collapse
|
16
|
Su Q, Wang F, Chen D, Chen G, Li C, Wei L. Deep convolutional neural networks with ensemble learning and transfer learning for automated detection of gastrointestinal diseases. Comput Biol Med 2022; 150:106054. [PMID: 36244302 DOI: 10.1016/j.compbiomed.2022.106054] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 08/12/2022] [Accepted: 08/27/2022] [Indexed: 11/22/2022]
Abstract
Gastrointestinal (GI) diseases are serious health threats to human health, and the related detection and treatment of gastrointestinal diseases place a huge burden on medical institutions. Imaging-based methods are one of the most important approaches for automated detection of gastrointestinal diseases. Although deep neural networks have shown impressive performance in a number of imaging tasks, its application to detection of gastrointestinal diseases has not been sufficiently explored. In this study, we propose a novel and practical method to detect gastrointestinal disease from wireless capsule endoscopy (WCE) images by convolutional neural networks. The proposed method utilizes three backbone networks modified and fine-tuned by transfer learning as the feature extractors, and an integrated classifier using ensemble learning is trained to detection of gastrointestinal diseases. The proposed method outperforms existing computational methods on the benchmark dataset. The case study results show that the proposed method captures discriminative information of wireless capsule endoscopy images. This work shows the potential of using deep learning-based computer vision models for effective GI disease screening.
Collapse
Affiliation(s)
- Qiaosen Su
- School of Software, Shandong University, Jinan, China; Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan, China
| | - Fengsheng Wang
- School of Software, Shandong University, Jinan, China; Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan, China
| | | | | | - Chao Li
- Beidahuang Industry Group General Hospital, Harbin, China.
| | - Leyi Wei
- School of Software, Shandong University, Jinan, China; Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan, China.
| |
Collapse
|
17
|
Time-based self-supervised learning for Wireless Capsule Endoscopy. Comput Biol Med 2022; 146:105631. [DOI: 10.1016/j.compbiomed.2022.105631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 04/17/2022] [Accepted: 04/17/2022] [Indexed: 11/18/2022]
|