1
|
K P AG, D RR, N MS, P LB. Gastrointestinal tract disease detection via deep learning based structural and statistical features optimized hexa-classification model. Technol Health Care 2024:THC240603. [PMID: 39031411 DOI: 10.3233/thc-240603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/22/2024]
Abstract
BACKGROUND Gastrointestinal tract (GIT) diseases impact the entire digestive system, spanning from the mouth to the anus. Wireless Capsule Endoscopy (WCE) stands out as an effective analytic instrument for Gastrointestinal tract diseases. Nevertheless, accurately identifying various lesion features, such as irregular sizes, shapes, colors, and textures, remains challenging in this field. OBJECTIVE Several computer vision algorithms have been introduced to tackle these challenges, but many relied on handcrafted features, resulting in inaccuracies in various instances. METHODS In this work, a novel Deep SS-Hexa model is proposed which is a combination two different deep learning structures for extracting two different features from the WCE images to detect various GIT ailment. The gathered images are denoised by weighted median filter to remove the noisy distortions and augment the images for enhancing the training data. The structural and statistical (SS) feature extraction process is sectioned into two phases for the analysis of distinct regions of gastrointestinal. In the first stage, statistical features of the image are retrieved using MobileNet with the support of SiLU activation function to retrieve the relevant features. In the second phase, the segmented intestine images are transformed into structural features to learn the local information. These SS features are parallelly fused for selecting the best relevant features with walrus optimization algorithm. Finally, Deep belief network (DBN) is used classified the GIT diseases into hexa classes namely normal, ulcer, pylorus, cecum, esophagitis and polyps on the basis of the selected features. RESULTS The proposed Deep SS-Hexa model attains an overall average accuracy of 99.16% in GIT disease detection based on KVASIR and KID datasets. The proposed Deep SS-Hexa model achieves high level of accuracy with minimal computational cost in the recognition of GIT illness. CONCLUSIONS The proposed Deep SS-Hexa Model progresses the overall accuracy range of 0.04%, 0.80% better than GastroVision, Genetic algorithm based on KVASIR dataset and 0.60%, 1.21% better than Modified U-Net, WCENet based on KID dataset respectively.
Collapse
Affiliation(s)
- Ajitha Gladis K P
- Department of Information Technology, CSI Institute of Technology, Thovalai, India
| | - Roja Ramani D
- Department of Computer Science and Engineering, New Horizon College of Engineering, Bengaluru, India
| | - Mohana Suganthi N
- Department of Computer Science and Engineering, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Chennai, India
| | - Linu Babu P
- Department of Electronics and Communication Engineering, IES College of Engineering, Thrissur, India
| |
Collapse
|
2
|
Yokote A, Umeno J, Kawasaki K, Fujioka S, Fuyuno Y, Matsuno Y, Yoshida Y, Imazu N, Miyazono S, Moriyama T, Kitazono T, Torisu T. Small bowel capsule endoscopy examination and open access database with artificial intelligence: The SEE-artificial intelligence project. DEN OPEN 2024; 4:e258. [PMID: 37359150 PMCID: PMC10288072 DOI: 10.1002/deo2.258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 05/31/2023] [Accepted: 06/05/2023] [Indexed: 06/28/2023]
Abstract
OBJECTIVES Artificial intelligence (AI) may be practical for image classification of small bowel capsule endoscopy (CE). However, creating a functional AI model is challenging. We attempted to create a dataset and an object detection CE AI model to explore modeling problems to assist in reading small bowel CE. METHODS We extracted 18,481 images from 523 small bowel CE procedures performed at Kyushu University Hospital from September 2014 to June 2021. We annotated 12,320 images with 23,033 disease lesions, combined them with 6161 normal images as the dataset, and examined the characteristics. Based on the dataset, we created an object detection AI model using YOLO v5 and we tested validation. RESULTS We annotated the dataset with 12 types of annotations, and multiple annotation types were observed in the same image. We test validated our AI model with 1396 images, and sensitivity for all 12 types of annotations was about 91%, with 1375 true positives, 659 false positives, and 120 false negatives detected. The highest sensitivity for individual annotations was 97%, and the highest area under the receiver operating characteristic curve was 0.98, but the quality of detection varied depending on the specific annotation. CONCLUSIONS Object detection AI model in small bowel CE using YOLO v5 may provide effective and easy-to-understand reading assistance. In this SEE-AI project, we open our dataset, the weights of the AI model, and a demonstration to experience our AI. We look forward to further improving the AI model in the future.
Collapse
Affiliation(s)
- Akihito Yokote
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Junji Umeno
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Keisuke Kawasaki
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Shin Fujioka
- Department of Endoscopic Diagnostics and Therapeutics Kyushu University Hospital Fukuoka Japan
| | - Yuta Fuyuno
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Yuichi Matsuno
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Yuichiro Yoshida
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Noriyuki Imazu
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Satoshi Miyazono
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Tomohiko Moriyama
- International Medical Department Kyushu University Hospital Fukuoka Japan
| | - Takanari Kitazono
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| | - Takehiro Torisu
- Department of Medicine and Clinical Science Graduate School of Medical Science Kyushu University Fukuoka Japan
| |
Collapse
|
3
|
Sahafi A, Koulaouzidis A, Lalinia M. Polypoid Lesion Segmentation Using YOLO-V8 Network in Wireless Video Capsule Endoscopy Images. Diagnostics (Basel) 2024; 14:474. [PMID: 38472946 DOI: 10.3390/diagnostics14050474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Revised: 01/26/2024] [Accepted: 02/19/2024] [Indexed: 03/14/2024] Open
Abstract
Gastrointestinal (GI) tract disorders are a significant public health issue. They are becoming more common and can cause serious health problems and high healthcare costs. Small bowel tumours (SBTs) and colorectal cancer (CRC) are both becoming more prevalent, especially among younger adults. Early detection and removal of polyps (precursors of malignancy) is essential for prevention. Wireless Capsule Endoscopy (WCE) is a procedure that utilises swallowable camera devices that capture images of the GI tract. Because WCE generates a large number of images, automated polyp segmentation is crucial. This paper reviews computer-aided approaches to polyp detection using WCE imagery and evaluates them using a dataset of labelled anomalies and findings. The study focuses on YOLO-V8, an improved deep learning model, for polyp segmentation and finds that it performs better than existing methods, achieving high precision and recall. The present study underscores the potential of automated detection systems in improving GI polyp identification.
Collapse
Affiliation(s)
- Ali Sahafi
- Department of Mechanical and Electrical Engineering, Digital and High-Frequency Electronics Section, University of Southern Denmark, 5230 Odense, Denmark
| | - Anastasios Koulaouzidis
- Surgical Research Unit, Odense University Hospital, 5000 Svendborg, Denmark
- Department of Clinical Research, University of Southern Denmark, 5230 Odense, Denmark
- Department of Medicine, OUH Svendborg Sygehus, 5700 Svendborg, Denmark
- Department of Social Medicine and Public Health, Pomeranian Medical University, 70204 Szczecin, Poland
| | - Mehrshad Lalinia
- Department of Mechanical and Electrical Engineering, Digital and High-Frequency Electronics Section, University of Southern Denmark, 5230 Odense, Denmark
| |
Collapse
|
4
|
Bordbar M, Helfroush MS, Danyali H, Ejtehadi F. Wireless capsule endoscopy multiclass classification using three-dimensional deep convolutional neural network model. Biomed Eng Online 2023; 22:124. [PMID: 38098015 PMCID: PMC10722702 DOI: 10.1186/s12938-023-01186-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 11/29/2023] [Indexed: 12/17/2023] Open
Abstract
BACKGROUND Wireless capsule endoscopy (WCE) is a patient-friendly and non-invasive technology that scans the whole of the gastrointestinal tract, including difficult-to-access regions like the small bowel. Major drawback of this technology is that the visual inspection of a large number of video frames produced during each examination makes the physician diagnosis process tedious and prone to error. Several computer-aided diagnosis (CAD) systems, such as deep network models, have been developed for the automatic recognition of abnormalities in WCE frames. Nevertheless, most of these studies have only focused on spatial information within individual WCE frames, missing the crucial temporal data within consecutive frames. METHODS In this article, an automatic multiclass classification system based on a three-dimensional deep convolutional neural network (3D-CNN) is proposed, which utilizes the spatiotemporal information to facilitate the WCE diagnosis process. The 3D-CNN model fed with a series of sequential WCE frames in contrast to the two-dimensional (2D) model, which exploits frames as independent ones. Moreover, the proposed 3D deep model is compared with some pre-trained networks. The proposed models are trained and evaluated with 29 subject WCE videos (14,691 frames before augmentation). The performance advantages of 3D-CNN over 2D-CNN and pre-trained networks are verified in terms of sensitivity, specificity, and accuracy. RESULTS 3D-CNN outperforms the 2D technique in all evaluation metrics (sensitivity: 98.92 vs. 98.05, specificity: 99.50 vs. 86.94, accuracy: 99.20 vs. 92.60). In conclusion, a novel 3D-CNN model for lesion detection in WCE frames is proposed in this study. CONCLUSION The results indicate the performance of 3D-CNN over 2D-CNN and some well-known pre-trained classifier networks. The proposed 3D-CNN model uses the rich temporal information in adjacent frames as well as spatial data to develop an accurate and efficient model.
Collapse
Affiliation(s)
- Mehrdokht Bordbar
- Department of Electrical Engineering, Shiraz University of Technology, Shiraz, Iran
| | | | - Habibollah Danyali
- Department of Electrical Engineering, Shiraz University of Technology, Shiraz, Iran
| | - Fardad Ejtehadi
- Department of Internal Medicine, Gastroenterohepatology Research Center, School of Medicine, Shiraz University of Medical Sciences, Shiraz, Iran
| |
Collapse
|
5
|
Zhu S, Gao J, Liu L, Yin M, Lin J, Xu C, Xu C, Zhu J. Public Imaging Datasets of Gastrointestinal Endoscopy for Artificial Intelligence: a Review. J Digit Imaging 2023; 36:2578-2601. [PMID: 37735308 PMCID: PMC10584770 DOI: 10.1007/s10278-023-00844-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 05/03/2023] [Accepted: 05/03/2023] [Indexed: 09/23/2023] Open
Abstract
With the advances in endoscopic technologies and artificial intelligence, a large number of endoscopic imaging datasets have been made public to researchers around the world. This study aims to review and introduce these datasets. An extensive literature search was conducted to identify appropriate datasets in PubMed, and other targeted searches were conducted in GitHub, Kaggle, and Simula to identify datasets directly. We provided a brief introduction to each dataset and evaluated the characteristics of the datasets included. Moreover, two national datasets in progress were discussed. A total of 40 datasets of endoscopic images were included, of which 34 were accessible for use. Basic and detailed information on each dataset was reported. Of all the datasets, 16 focus on polyps, and 6 focus on small bowel lesions. Most datasets (n = 16) were constructed by colonoscopy only, followed by normal gastrointestinal endoscopy and capsule endoscopy (n = 9). This review may facilitate the usage of public dataset resources in endoscopic research.
Collapse
Affiliation(s)
- Shiqi Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Jingwen Gao
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Lu Liu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Minyue Yin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Jiaxi Lin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Chang Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Chunfang Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China.
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China.
| | - Jinzhou Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China.
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China.
| |
Collapse
|
6
|
Dimas G, Cholopoulou E, Iakovidis DK. E pluribus unum interpretable convolutional neural networks. Sci Rep 2023; 13:11421. [PMID: 37452133 PMCID: PMC10349135 DOI: 10.1038/s41598-023-38459-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Accepted: 07/08/2023] [Indexed: 07/18/2023] Open
Abstract
The adoption of convolutional neural network (CNN) models in high-stake domains is hindered by their inability to meet society's demand for transparency in decision-making. So far, a growing number of methodologies have emerged for developing CNN models that are interpretable by design. However, such models are not capable of providing interpretations in accordance with human perception, while maintaining competent performance. In this paper, we tackle these challenges with a novel, general framework for instantiating inherently interpretable CNN models, named E pluribus unum interpretable CNN (EPU-CNN). An EPU-CNN model consists of CNN sub-networks, each of which receives a different representation of an input image expressing a perceptual feature, such as color or texture. The output of an EPU-CNN model consists of the classification prediction and its interpretation, in terms of relative contributions of perceptual features in different regions of the input image. EPU-CNN models have been extensively evaluated on various publicly available datasets, as well as a contributed benchmark dataset. Medical datasets are used to demonstrate the applicability of EPU-CNN for risk-sensitive decisions in medicine. The experimental results indicate that EPU-CNN models can achieve a comparable or better classification performance than other CNN architectures while providing humanly perceivable interpretations.
Collapse
Affiliation(s)
- George Dimas
- Department of Computer Science and Biomedical Informatics, School of Science, University of Thessaly, Lamia, Greece
| | - Eirini Cholopoulou
- Department of Computer Science and Biomedical Informatics, School of Science, University of Thessaly, Lamia, Greece
| | - Dimitris K Iakovidis
- Department of Computer Science and Biomedical Informatics, School of Science, University of Thessaly, Lamia, Greece.
| |
Collapse
|
7
|
Horovistiz A, Oliveira M, Araújo H. Computer vision-based solutions to overcome the limitations of wireless capsule endoscopy. J Med Eng Technol 2023; 47:242-261. [PMID: 38231042 DOI: 10.1080/03091902.2024.2302025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 12/28/2023] [Indexed: 01/18/2024]
Abstract
Endoscopic investigation plays a critical role in the diagnosis of gastrointestinal (GI) diseases. Since 2001, Wireless Capsule Endoscopy (WCE) has been available for small bowel exploration and is in continuous development. Over the last decade, WCE has achieved impressive improvements in areas such as miniaturisation, image quality and battery life. As a result, WCE is currently a very useful alternative to wired enteroscopy in the investigation of various small bowel abnormalities and has the potential to become the leading screening technique for the entire gastrointestinal tract. However, commercial solutions still have several limitations, namely incomplete examination and limited diagnostic capacity. These deficiencies are related to technical issues, such as image quality, motion estimation and power consumption management. Computational methods, based on image processing and analysis, can help to overcome these challenges and reduce both the time required by reviewers and human interpretation errors. Research groups have proposed a series of methods including algorithms for locating the capsule or lesion, assessing intestinal motility and improving image quality.In this work, we provide a critical review of computational vision-based methods for WCE image analysis aimed at overcoming the technological challenges of capsules. This article also reviews several representative public datasets used to evaluate the performance of WCE techniques and methods. Finally, some promising solutions of computational methods based on the analysis of multiple-camera endoscopic images are presented.
Collapse
Affiliation(s)
- Ana Horovistiz
- Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal
| | - Marina Oliveira
- Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal
- Department of Electrical and Computer Engineering (DEEC), Faculty of Sciences and Technology, University of Coimbra, Coimbra, Portugal
| | - Helder Araújo
- Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal
- Department of Electrical and Computer Engineering (DEEC), Faculty of Sciences and Technology, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
8
|
A multi-centre polyp detection and segmentation dataset for generalisability assessment. Sci Data 2023; 10:75. [PMID: 36746950 PMCID: PMC9902556 DOI: 10.1038/s41597-023-01981-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 01/23/2023] [Indexed: 02/08/2023] Open
Abstract
Polyps in the colon are widely known cancer precursors identified by colonoscopy. Whilst most polyps are benign, the polyp's number, size and surface structure are linked to the risk of colon cancer. Several methods have been developed to automate polyp detection and segmentation. However, the main issue is that they are not tested rigorously on a large multicentre purpose-built dataset, one reason being the lack of a comprehensive public dataset. As a result, the developed methods may not generalise to different population datasets. To this extent, we have curated a dataset from six unique centres incorporating more than 300 patients. The dataset includes both single frame and sequence data with 3762 annotated polyp labels with precise delineation of polyp boundaries verified by six senior gastroenterologists. To our knowledge, this is the most comprehensive detection and pixel-level segmentation dataset (referred to as PolypGen) curated by a team of computational scientists and expert gastroenterologists. The paper provides insight into data construction and annotation strategies, quality assurance, and technical validation.
Collapse
|
9
|
Vats A, Mohammed A, Pedersen M. From labels to priors in capsule endoscopy: a prior guided approach for improving generalization with few labels. Sci Rep 2022; 12:15708. [PMID: 36127404 PMCID: PMC9489743 DOI: 10.1038/s41598-022-19675-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 09/01/2022] [Indexed: 01/15/2023] Open
Abstract
The lack of generalizability of deep learning approaches for the automated diagnosis of pathologies in Wireless Capsule Endoscopy (WCE) has prevented any significant advantages from trickling down to real clinical practices. As a result, disease management using WCE continues to depend on exhaustive manual investigations by medical experts. This explains its limited use despite several advantages. Prior works have considered using higher quality and quantity of labels as a way of tackling the lack of generalization, however this is hardly scalable considering pathology diversity not to mention that labeling large datasets encumbers the medical staff additionally. We propose using freely available domain knowledge as priors to learn more robust and generalizable representations. We experimentally show that domain priors can benefit representations by acting in proxy of labels, thereby significantly reducing the labeling requirement while still enabling fully unsupervised yet pathology-aware learning. We use the contrastive objective along with prior-guided views during pretraining, where the view choices inspire sensitivity to pathological information. Extensive experiments on three datasets show that our method performs better than (or closes gap with) the state-of-the-art in the domain, establishing a new benchmark in pathology classification and cross-dataset generalization, as well as scaling to unseen pathology categories.
Collapse
Affiliation(s)
- Anuja Vats
- grid.5947.f0000 0001 1516 2393Department of Computer Science, NTNU, 2819 Gjøvik, Norway
| | - Ahmed Mohammed
- grid.5947.f0000 0001 1516 2393Department of Computer Science, NTNU, 2819 Gjøvik, Norway ,grid.4319.f0000 0004 0448 3150SINTEF Digital, Smart Sensor Systems, Oslo, Norway
| | - Marius Pedersen
- grid.5947.f0000 0001 1516 2393Department of Computer Science, NTNU, 2819 Gjøvik, Norway
| |
Collapse
|
10
|
Raut V, Gunjan R, Shete VV, Eknath UD. Gastrointestinal tract disease segmentation and classification in wireless capsule endoscopy using intelligent deep learning model. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2022. [DOI: 10.1080/21681163.2022.2099298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/16/2022]
Affiliation(s)
- Vrushali Raut
- Electronics & Communication Engineering, MIT School of Engineering, MIT Art, Design and Technology University, Pune, India
| | - Reena Gunjan
- Electronics & Communication Engineering, MIT School of Engineering, MIT Art, Design and Technology University, Pune, India
| | - Virendra V. Shete
- Electronics & Communication Engineering, MIT School of Engineering, MIT Art, Design and Technology University, Pune, India
| | - Upasani Dhananjay Eknath
- Electronics & Communication Engineering, MIT School of Engineering, MIT Art, Design and Technology University, Pune, India
| |
Collapse
|
11
|
Liu X, Yuan Y. A Source-Free Domain Adaptive Polyp Detection Framework With Style Diversification Flow. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1897-1908. [PMID: 35139013 DOI: 10.1109/tmi.2022.3150435] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The automatic detection of polyps across colonoscopy and Wireless Capsule Endoscopy (WCE) datasets is crucial for early diagnosis and curation of colorectal cancer. Existing deep learning approaches either require mass training data collected from multiple sites or use unsupervised domain adaptation (UDA) technique with labeled source data. However, these methods are not applicable when the data is not accessible due to privacy concerns or data storage limitations. Aiming to achieve source-free domain adaptive polyp detection, we propose a consistency based model that utilizes Source Model as Proxy Teacher (SMPT) with only a transferable pretrained model and unlabeled target data. SMPT first transfers the stored domain-invariant knowledge in the pretrained source model to the target model via Source Knowledge Distillation (SKD), then uses Proxy Teacher Rectification (PTR) to rectify the source model with temporal ensemble of the target model. Moreover, to alleviate the biased knowledge caused by domain gaps, we propose Uncertainty-Guided Online Bootstrapping (UGOB) to adaptively assign weights for each target image regarding their uncertainty. In addition, we design Source Style Diversification Flow (SSDF) that gradually generates diverse style images and relaxes style-sensitive channels based on source and target information to enhance the robustness of the model towards style variation. The capacities of SMPT and SSDF are further boosted with iterative optimization, constructing a stronger framework SMPT++ for cross-domain polyp detection. Extensive experiments are conducted on five distinct polyp datasets under two types of cross-domain settings. Our proposed method shows the state-of-the-art performance and even outperforms previous UDA approaches that require the source data by a large margin. The source code is available at github.com/CityU-AIM-Group/SFPolypDA.
Collapse
|
12
|
Xie X, Xiao YF, Zhao XY, Li JJ, Yang QQ, Peng X, Nie XB, Zhou JY, Zhao YB, Yang H, Liu X, Liu E, Chen YY, Zhou YY, Fan CQ, Bai JY, Lin H, Koulaouzidis A, Yang SM. Development and Validation of an Artificial Intelligence Model for Small Bowel Capsule Endoscopy Video Review. JAMA Netw Open 2022; 5:e2221992. [PMID: 35834249 PMCID: PMC9284338 DOI: 10.1001/jamanetworkopen.2022.21992] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
Abstract
IMPORTANCE Reading small bowel capsule endoscopy (SBCE) videos is a tedious task for clinicians, and a new method should be applied to solve the situation. OBJECTIVES To develop and evaluate the performance of a convolutional neural network algorithm for SBCE video review in real-life clinical care. DESIGN, SETTING, AND PARTICIPANTS In this multicenter, retrospective diagnostic study, a deep learning neural network (SmartScan) was trained and validated for the SBCE video review. A total of 2927 SBCE examinations from 29 medical centers were used to train SmartScan to detect 17 types of CE structured terminology (CEST) findings from January 1, 2019, to June 30, 2020. SmartScan was later validated with conventional reading (CR) and SmartScan-assisted reading (SSAR) in 2898 SBCE examinations collected from 22 medical centers. Data analysis was performed from January 25 to December 31, 2021. EXPOSURE An artificial intelligence-based tool for interpreting clinical images of SBCE. MAIN OUTCOMES AND MEASURES The detection rate and efficiency of CEST findings detected by SSAR and CR were compared. RESULTS A total of 5825 SBCE examinations were retrospectively collected; 2898 examinations (1765 male participants [60.9%]; mean [SD] age, 49.8 [15.5] years) were included in the validation phase. From a total of 6084 CEST-classified SB findings, SSAR detected 5834 findings (95.9%; 95% CI, 95.4%-96.4%), significantly higher than CR, which detected 4630 findings (76.1%; 95% CI, 75.0%-77.2%). SmartScan-assisted reading achieved a higher per-patient detection rate (79.3% [2298 of 2898]) for CEST findings compared with CR (70.7% [2048 of 2298]; 95% CI, 69.0%-72.3%). With SSAR, the mean (SD) number of images (per SBCE video) requiring review was reduced to 779.2 (337.2) compared with 27 910.8 (12 882.9) with CR, for a mean (SD) reduction rate of 96.1% (4.3%). The mean (SD) reading time with SSAR was shortened to 5.4 (1.5) minutes compared with CR (51.4 [11.6] minutes), for a mean (SD) reduction rate of 89.3% (3.1%). CONCLUSIONS AND RELEVANCE This study suggests that a convolutional neural network-based algorithm is associated with an increased detection rate of SBCE findings and reduced SBCE video reading time.
Collapse
Affiliation(s)
- Xia Xie
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Yu-Feng Xiao
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Xiao-Yan Zhao
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Jian-Jun Li
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Qiang-Qiang Yang
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Xue Peng
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Xu-Biao Nie
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Jian-Yun Zhou
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Yong-Bing Zhao
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Huan Yang
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Xi Liu
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - En Liu
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Yu-Yang Chen
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Yuan-Yuan Zhou
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Chao-Qiang Fan
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Jian-Ying Bai
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| | - Hui Lin
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
- Department of Epidemiology, the Third Military Medical University, Chongqing, China
| | | | - Shi-Ming Yang
- Department of Gastroenterology, The Second Affiliated Hospital, the Third Military Medical University, Chongqing, China
| |
Collapse
|
13
|
Amiri Z, Hassanpour H, Beghdadi A. Feature extraction for abnormality detection in capsule endoscopy images. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103219] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
14
|
Artificial Intelligence in Gastroenterology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-58080-3_163-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
15
|
Strümke I, Hicks SA, Thambawita V, Jha D, Parasa S, Riegler MA, Halvorsen P. Artificial Intelligence in Gastroenterology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
16
|
Aljabri M, AlAmir M, AlGhamdi M, Abdel-Mottaleb M, Collado-Mesa F. Towards a better understanding of annotation tools for medical imaging: a survey. MULTIMEDIA TOOLS AND APPLICATIONS 2022; 81:25877-25911. [PMID: 35350630 PMCID: PMC8948453 DOI: 10.1007/s11042-022-12100-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 08/04/2021] [Accepted: 01/03/2022] [Indexed: 05/07/2023]
Abstract
Medical imaging refers to several different technologies that are used to view the human body to diagnose, monitor, or treat medical conditions. It requires significant expertise to efficiently and correctly interpret the images generated by each of these technologies, which among others include radiography, ultrasound, and magnetic resonance imaging. Deep learning and machine learning techniques provide different solutions for medical image interpretation including those associated with detection and diagnosis. Despite the huge success of deep learning algorithms in image analysis, training algorithms to reach human-level performance in these tasks depends on the availability of large amounts of high-quality training data, including high-quality annotations to serve as ground-truth. Different annotation tools have been developed to assist with the annotation process. In this survey, we present the currently available annotation tools for medical imaging, including descriptions of graphical user interfaces (GUI) and supporting instruments. The main contribution of this study is to provide an intensive review of the popular annotation tools and show their successful usage in annotating medical imaging dataset to guide researchers in this area.
Collapse
Affiliation(s)
- Manar Aljabri
- Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia
| | - Manal AlAmir
- Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia
| | - Manal AlGhamdi
- Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia
| | | | - Fernando Collado-Mesa
- Department of Radiology, University of Miami Miller School of Medicine, Florida, FL USA
| |
Collapse
|
17
|
Zhu M, Chen Z, Yuan Y. DSI-Net: Deep Synergistic Interaction Network for Joint Classification and Segmentation With Endoscope Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:3315-3325. [PMID: 34033538 DOI: 10.1109/tmi.2021.3083586] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Automatic classification and segmentation of wireless capsule endoscope (WCE) images are two clinically significant and relevant tasks in a computer-aided diagnosis system for gastrointestinal diseases. Most of existing approaches, however, considered these two tasks individually and ignored their complementary information, leading to limited performance. To overcome this bottleneck, we propose a deep synergistic interaction network (DSI-Net) for joint classification and segmentation with WCE images, which mainly consists of the classification branch (C-Branch), the coarse segmentation (CS-Branch) and the fine segmentation branches (FS-Branch). In order to facilitate the classification task with the segmentation knowledge, a lesion location mining (LLM) module is devised in C-Branch to accurately highlight lesion regions through mining neglected lesion areas and erasing misclassified background areas. To assist the segmentation task with the classification prior, we propose a category-guided feature generation (CFG) module in FS-Branch to improve pixel representation by leveraging the category prototypes of C-Branch to obtain the category-aware features. In such way, these modules enable the deep synergistic interaction between these two tasks. In addition, we introduce a task interaction loss to enhance the mutual supervision between the classification and segmentation tasks and guarantee the consistency of their predictions. Relying on the proposed deep synergistic interaction mechanism, DSI-Net achieves superior classification and segmentation performance on public dataset in comparison with state-of-the-art methods. The source code is available at https://github.com/CityU-AIM-Group/DSI-Net.
Collapse
|
18
|
Dray X, Toth E, de Lange T, Koulaouzidis A. Artificial intelligence, capsule endoscopy, databases, and the Sword of Damocles. Endosc Int Open 2021; 9:E1754-E1755. [PMID: 34790540 PMCID: PMC8589560 DOI: 10.1055/a-1521-4882] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/28/2022] Open
Affiliation(s)
- Xavier Dray
- Sorbonne University, Centre for Digestive Endoscopy, Hôpital Saint Antoine, APHP, Paris, France
| | - Ervin Toth
- Skane University Hospitals, Endoscopy Unit, Department of Gastroenterology, Malmo, Sweden
| | - Thomas de Lange
- Sahlgrenska University Hospital-Molndal, Medical Department, Gothenburg, Sweden
| | - Anastasio Koulaouzidis
- Pomeranian Medical University in Szczecin, Department of Social Medicine & Public Health, Faculty of Health Sciences, Zchodniopomorskie, Poland
| |
Collapse
|
19
|
Anteby R, Klang E, Horesh N, Nachmany I, Shimon O, Barash Y, Kopylov U, Soffer S. Deep learning for noninvasive liver fibrosis classification: A systematic review. Liver Int 2021; 41:2269-2278. [PMID: 34008300 DOI: 10.1111/liv.14966] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Revised: 04/23/2021] [Accepted: 05/13/2021] [Indexed: 12/17/2022]
Abstract
BACKGROUND AND AIMS While biopsy is the gold standard for liver fibrosis staging, it poses significant risks. Noninvasive assessment of liver fibrosis is a growing field. Recently, deep learning (DL) technology has revolutionized medical image analysis. This technology has the potential to enhance noninvasive fibrosis assessment. We systematically examined the application of DL in noninvasive liver fibrosis imaging. METHODS Embase, MEDLINE, Web of Science, and IEEE Xplore databases were used to identify studies that reported on the accuracy of DL for classification of liver fibrosis on noninvasive imaging. The search keywords were "liver or hepatic," "fibrosis or cirrhosis," and "neural or DL networks." Risk of bias and applicability were evaluated using the QUADAS-2 tool. RESULTS Sixteen studies were retrieved. Imaging modalities included ultrasound (n = 10), computed tomography (n = 3), and magnetic resonance imaging (n = 3). The studies analyzed a total of 40 405 radiological images from 15 853 patients. All but two of the studies were retrospective. In most studies the "ground truth" reference was the METAVIR score for pathological staging (n = 9.56%). The majority of the studies reported an accuracy >85% when compared to histopathology. Fourteen studies (87.5%) had a high risk of bias and concerns regarding applicability. CONCLUSIONS Deep learning has the potential to play an emerging role in liver fibrosis classification. Yet, it is still limited by a relatively small number of retrospective studies. Clinicians should facilitate the use of this technology by sharing databases and standardized reports. This may optimize the noninvasive evaluation of liver fibrosis on a large scale.
Collapse
Affiliation(s)
- Roi Anteby
- School of Public Health, Harvard University, Boston, MA, USA
| | - Eyal Klang
- Department of Population Health Science and Policy, Institute for Healthcare Delivery Science, New York, NY, USA.,Sackler Medical School, Tel Aviv University, Tel Aviv, Israel.,Deep Vision Lab, The Chaim Sheba Medical Center, Ramat Gan, Israel.,Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel
| | - Nir Horesh
- Sackler Medical School, Tel Aviv University, Tel Aviv, Israel.,Department of Surgery and Transplantation B, Sheba Medical Center, Tel Hashomer, Israel
| | - Ido Nachmany
- Sackler Medical School, Tel Aviv University, Tel Aviv, Israel.,Department of Surgery and Transplantation B, Sheba Medical Center, Tel Hashomer, Israel
| | - Orit Shimon
- Sackler Medical School, Tel Aviv University, Tel Aviv, Israel.,Department of Anesthesia, Rabin Medical Center, Beilinson Hospital, Petach Tikvah, Israel
| | - Yiftach Barash
- Sackler Medical School, Tel Aviv University, Tel Aviv, Israel.,Deep Vision Lab, The Chaim Sheba Medical Center, Ramat Gan, Israel.,Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel
| | - Uri Kopylov
- Sackler Medical School, Tel Aviv University, Tel Aviv, Israel.,Department of Gastroenterology, Sheba Medical Center, Tel Hashomer, Israel
| | - Shelly Soffer
- Deep Vision Lab, The Chaim Sheba Medical Center, Ramat Gan, Israel.,Internal Medicine B, Assuta Medical Center, Ashdod, Israel.,Ben-Gurion University of the Negev, Be'er Sheva, Israel
| |
Collapse
|
20
|
Artificial Intelligence in Capsule Endoscopy: A Practical Guide to Its Past and Future Challenges. Diagnostics (Basel) 2021; 11:diagnostics11091722. [PMID: 34574063 PMCID: PMC8469774 DOI: 10.3390/diagnostics11091722] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Revised: 09/15/2021] [Accepted: 09/17/2021] [Indexed: 12/20/2022] Open
Abstract
Artificial intelligence (AI) has revolutionized the medical diagnostic process of various diseases. Since the manual reading of capsule endoscopy videos is a time-intensive, error-prone process, computerized algorithms have been introduced to automate this process. Over the past decade, the evolution of convolutional neural network (CNN) enabled AI to detect multiple lesions simultaneously with increasing accuracy and sensitivity. Difficulty in validating CNN performance and unique characteristics of capsule endoscopy images make computer-aided reading systems in capsule endoscopy still on a preclinical level. Although AI technology can be used as an auxiliary second observer in capsule endoscopy, it is expected that in the near future, it will effectively reduce the reading time and ultimately become an independent, integrated reading system.
Collapse
|
21
|
Vieira PM, Freitas NR, Lima VB, Costa D, Rolanda C, Lima CS. Multi-pathology detection and lesion localization in WCE videos by using the instance segmentation approach. Artif Intell Med 2021; 119:102141. [PMID: 34531016 DOI: 10.1016/j.artmed.2021.102141] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Revised: 06/10/2021] [Accepted: 08/03/2021] [Indexed: 12/13/2022]
Abstract
The majority of current systems for automatic diagnosis considers the detection of a unique and previously known pathology. Considering specifically the diagnosis of lesions in the small bowel using endoscopic capsule images, very few consider the possible existence of more than one pathology and when they do, they are mainly detection based systems therefore unable to localize the suspected lesions. Such systems do not fully satisfy the medical community, that in fact needs a system that detects any pathology and eventually more than one, when they coexist. In addition, besides the diagnostic capability of these systems, localizing the lesions in the image has been of great interest to the medical community, mainly for training medical personnel purposes. So, nowadays, the inclusion of the lesion location in automatic diagnostic systems is practically mandatory. Multi-pathology detection can be seen as a multi-object detection task and as each frame can contain different instances of the same lesion, instance segmentation seems to be appropriate for the purpose. Consequently, we argue that a multi-pathology system benefits from using the instance segmentation approach, since classification and segmentation modules are both required complementing each other in lesion detection and localization. According to our best knowledge such a system does not yet exist for the detection of WCE pathologies. This paper proposes a multi-pathology system that can be applied to WCE images, which uses the Mask Improved RCNN (MI-RCNN), a new mask subnet scheme which has shown to significantly improve mask predictions of the high performing state-of-the-art Mask-RCNN and PANet systems. A novel training strategy based on the second momentum is also proposed for the first time for training Mask-RCNN and PANet based systems. These approaches were tested using the public database KID, and the included pathologies were bleeding, angioectasias, polyps and inflammatory lesions. Experimental results show significant improvements for the proposed versions, reaching increases of almost 7% over the PANet model when the new proposed training approach was employed.
Collapse
Affiliation(s)
- Pedro M Vieira
- CMEMS-UMinho Research Unit, Universidade do Minho, Guimarães, Portugal.
| | - Nuno R Freitas
- CMEMS-UMinho Research Unit, Universidade do Minho, Guimarães, Portugal
| | - Veríssimo B Lima
- CMEMS-UMinho Research Unit, Universidade do Minho, Guimarães, Portugal; School of Engineering (ISEP), Polytechnic Institute of Porto (P.PORTO), Porto, Portugal
| | - Dalila Costa
- Life and Health Sciences Research Institute, University of Minho, Campus Gualtar, 4710-057 Braga, Portugal; ICVS/3Bs - PT Government Associate Laboratory, Braga/Guimarães, Portugal; Department of Gastroenterology, Hospital de Braga, Braga, Portugal
| | - Carla Rolanda
- Life and Health Sciences Research Institute, University of Minho, Campus Gualtar, 4710-057 Braga, Portugal; ICVS/3Bs - PT Government Associate Laboratory, Braga/Guimarães, Portugal; Department of Gastroenterology, Hospital de Braga, Braga, Portugal
| | - Carlos S Lima
- CMEMS-UMinho Research Unit, Universidade do Minho, Guimarães, Portugal
| |
Collapse
|
22
|
Jain S, Seal A, Ojha A, Yazidi A, Bures J, Tacheci I, Krejcar O. A deep CNN model for anomaly detection and localization in wireless capsule endoscopy images. Comput Biol Med 2021; 137:104789. [PMID: 34455302 DOI: 10.1016/j.compbiomed.2021.104789] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2021] [Revised: 08/18/2021] [Accepted: 08/18/2021] [Indexed: 12/22/2022]
Abstract
Wireless capsule endoscopy (WCE) is one of the most efficient methods for the examination of gastrointestinal tracts. Computer-aided intelligent diagnostic tools alleviate the challenges faced during manual inspection of long WCE videos. Several approaches have been proposed in the literature for the automatic detection and localization of anomalies in WCE images. Some of them focus on specific anomalies such as bleeding, polyp, lesion, etc. However, relatively fewer generic methods have been proposed to detect all those common anomalies simultaneously. In this paper, a deep convolutional neural network (CNN) based model 'WCENet' is proposed for anomaly detection and localization in WCE images. The model works in two phases. In the first phase, a simple and efficient attention-based CNN classifies an image into one of the four categories: polyp, vascular, inflammatory, or normal. If the image is classified in one of the abnormal categories, it is processed in the second phase for the anomaly localization. Fusion of Grad-CAM++ and a custom SegNet is used for anomalous region segmentation in the abnormal image. WCENet classifier attains accuracy and area under receiver operating characteristic of 98% and 99%. The WCENet segmentation model obtains a frequency weighted intersection over union of 81%, and an average dice score of 56% on the KID dataset. WCENet outperforms nine different state-of-the-art conventional machine learning and deep learning models on the KID dataset. The proposed model demonstrates potential for clinical applications.
Collapse
Affiliation(s)
- Samir Jain
- PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, 482005, India
| | - Ayan Seal
- PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, 482005, India.
| | - Aparajita Ojha
- PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, 482005, India
| | - Anis Yazidi
- Department of Computer Science, OsloMet - Oslo Metropolitan University, Oslo, Norway; Department of Plastic and Reconstructive Surgery, Oslo University Hospital, Oslo, Norway; Department of Computer Science, Norwegian University of Science and Technology, Trondheim, Norway
| | - Jan Bures
- Second Department of Internal Medicine-Gastroenterology, Charles University, Faculty of Medicine in Hradec Kralove and University Hospital Hradec Kralove, Sokolska 581, Hradec Kralove, 50005, Czech Republic
| | - Ilja Tacheci
- Second Department of Internal Medicine-Gastroenterology, Charles University, Faculty of Medicine in Hradec Kralove and University Hospital Hradec Kralove, Sokolska 581, Hradec Kralove, 50005, Czech Republic
| | - Ondrej Krejcar
- Center for Basic and Applied Research, Faculty of Informatics and Management, University of Hradec Kralove, Hradecka 1249, Hradec Kralove, 50003, Czech Republic; Malaysia Japan International Institute of Technology, Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, 54100, Kuala Lumpur, Malaysia
| |
Collapse
|
23
|
Tziortziotis I, Laskaratos FM, Coda S. Role of Artificial Intelligence in Video Capsule Endoscopy. Diagnostics (Basel) 2021; 11:1192. [PMID: 34209029 PMCID: PMC8303156 DOI: 10.3390/diagnostics11071192] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2021] [Accepted: 06/28/2021] [Indexed: 02/06/2023] Open
Abstract
Capsule endoscopy (CE) has been increasingly utilised in recent years as a minimally invasive tool to investigate the whole gastrointestinal (GI) tract and a range of capsules are currently available for evaluation of upper GI, small bowel, and lower GI pathology. Although CE is undoubtedly an invaluable test for the investigation of small bowel pathology, it presents considerable challenges and limitations, such as long and laborious reading times, risk of missing lesions, lack of bowel cleansing score and lack of locomotion. Artificial intelligence (AI) seems to be a promising tool that may help improve the performance metrics of CE, and consequently translate to better patient care. In the last decade, significant progress has been made to apply AI in the field of endoscopy, including CE. Although it is certain that AI will find soon its place in day-to-day endoscopy clinical practice, there are still some open questions and barriers limiting its widespread application. In this review, we provide some general information about AI, and outline recent advances in AI and CE, issues around implementation of AI in medical practice and potential future applications of AI-aided CE.
Collapse
Affiliation(s)
- Ioannis Tziortziotis
- Endoscopy Unit, Digestive Diseases Centre, Queen's Hospital, Barking Havering and Redbridge University Hospitals NHS Trust, Rom Valley Way, Romford, London RM7 0AG, UK
| | - Faidon-Marios Laskaratos
- Endoscopy Unit, Digestive Diseases Centre, Queen's Hospital, Barking Havering and Redbridge University Hospitals NHS Trust, Rom Valley Way, Romford, London RM7 0AG, UK
| | - Sergio Coda
- Endoscopy Unit, Digestive Diseases Centre, Queen's Hospital, Barking Havering and Redbridge University Hospitals NHS Trust, Rom Valley Way, Romford, London RM7 0AG, UK
- Photonics Group-Department of Physics, Imperial College London, Exhibition Rd, South Kensington, London SW7 2BX, UK
| |
Collapse
|
24
|
Smedsrud PH, Thambawita V, Hicks SA, Gjestang H, Nedrejord OO, Næss E, Borgli H, Jha D, Berstad TJD, Eskeland SL, Lux M, Espeland H, Petlund A, Nguyen DTD, Garcia-Ceja E, Johansen D, Schmidt PT, Toth E, Hammer HL, de Lange T, Riegler MA, Halvorsen P. Kvasir-Capsule, a video capsule endoscopy dataset. Sci Data 2021; 8:142. [PMID: 34045470 PMCID: PMC8160146 DOI: 10.1038/s41597-021-00920-z] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Accepted: 04/15/2021] [Indexed: 12/12/2022] Open
Abstract
Artificial intelligence (AI) is predicted to have profound effects on the future of video capsule endoscopy (VCE) technology. The potential lies in improving anomaly detection while reducing manual labour. Existing work demonstrates the promising benefits of AI-based computer-assisted diagnosis systems for VCE. They also show great potential for improvements to achieve even better results. Also, medical data is often sparse and unavailable to the research community, and qualified medical personnel rarely have time for the tedious labelling work. We present Kvasir-Capsule, a large VCE dataset collected from examinations at a Norwegian Hospital. Kvasir-Capsule consists of 117 videos which can be used to extract a total of 4,741,504 image frames. We have labelled and medically verified 47,238 frames with a bounding box around findings from 14 different classes. In addition to these labelled images, there are 4,694,266 unlabelled frames included in the dataset. The Kvasir-Capsule dataset can play a valuable role in developing better algorithms in order to reach true potential of VCE technology.
Collapse
Affiliation(s)
- Pia H Smedsrud
- SimulaMet, Oslo, Norway.
- University of Oslo, Oslo, Norway.
- Augere Medical AS, Oslo, Norway.
| | | | - Steven A Hicks
- SimulaMet, Oslo, Norway
- Oslo Metropolitan University, Oslo, Norway
| | | | | | - Espen Næss
- SimulaMet, Oslo, Norway
- University of Oslo, Oslo, Norway
| | - Hanna Borgli
- SimulaMet, Oslo, Norway
- University of Oslo, Oslo, Norway
| | - Debesh Jha
- SimulaMet, Oslo, Norway
- UIT The Arctic University of Norway, Tromsø, Norway
| | | | | | | | | | | | | | | | - Dag Johansen
- UIT The Arctic University of Norway, Tromsø, Norway
| | - Peter T Schmidt
- Karolinska Institutet, Department of Medicine, Solna, Sweden
- Ersta Hospital, Department of Medicine, Stockholm, Sweden
| | - Ervin Toth
- Department of Gastroenterology, Skåne University Hospital, Malmö Lund University, Malmö, Sweden
| | - Hugo L Hammer
- SimulaMet, Oslo, Norway
- Oslo Metropolitan University, Oslo, Norway
| | - Thomas de Lange
- Department of Medical Research, Bærum Hospital, Gjettum, Norway
- Augere Medical AS, Oslo, Norway
- Medical Department, Sahlgrenska University Hospital-Mölndal Hospital, Göteborg, Sweden
- Department of Molecular and Clinical Medicine, Sahlgrenska Academy, University of Gothenburg, Göteborg, Sweden
| | | | - Pål Halvorsen
- SimulaMet, Oslo, Norway
- Oslo Metropolitan University, Oslo, Norway
| |
Collapse
|
25
|
Ozyoruk KB, Gokceler GI, Bobrow TL, Coskun G, Incetan K, Almalioglu Y, Mahmood F, Curto E, Perdigoto L, Oliveira M, Sahin H, Araujo H, Alexandrino H, Durr NJ, Gilbert HB, Turan M. EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Med Image Anal 2021; 71:102058. [PMID: 33930829 DOI: 10.1016/j.media.2021.102058] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Revised: 01/23/2021] [Accepted: 03/29/2021] [Indexed: 02/07/2023]
Abstract
Deep learning techniques hold promise to develop dense topography reconstruction and pose estimation methods for endoscopic videos. However, currently available datasets do not support effective quantitative benchmarking. In this paper, we introduce a comprehensive endoscopic SLAM dataset consisting of 3D point cloud data for six porcine organs, capsule and standard endoscopy recordings, synthetically generated data as well as clinically in use conventional endoscope recording of the phantom colon with computed tomography(CT) scan ground truth. A Panda robotic arm, two commercially available capsule endoscopes, three conventional endoscopes with different camera properties, two high precision 3D scanners, and a CT scanner were employed to collect data from eight ex-vivo porcine gastrointestinal (GI)-tract organs and a silicone colon phantom model. In total, 35 sub-datasets are provided with 6D pose ground truth for the ex-vivo part: 18 sub-datasets for colon, 12 sub-datasets for stomach, and 5 sub-datasets for small intestine, while four of these contain polyp-mimicking elevations carried out by an expert gastroenterologist. To verify the applicability of this data for use with real clinical systems, we recorded a video sequence with a state-of-the-art colonoscope from a full representation silicon colon phantom. Synthetic capsule endoscopy frames from stomach, colon, and small intestine with both depth and pose annotations are included to facilitate the study of simulation-to-real transfer learning algorithms. Additionally, we propound Endo-SfMLearner, an unsupervised monocular depth and pose estimation method that combines residual networks with a spatial attention module in order to dictate the network to focus on distinguishable and highly textured tissue regions. The proposed approach makes use of a brightness-aware photometric loss to improve the robustness under fast frame-to-frame illumination changes that are commonly seen in endoscopic videos. To exemplify the use-case of the EndoSLAM dataset, the performance of Endo-SfMLearner is extensively compared with the state-of-the-art: SC-SfMLearner, Monodepth2, and SfMLearner. The codes and the link for the dataset are publicly available at https://github.com/CapsuleEndoscope/EndoSLAM. A video demonstrating the experimental setup and procedure is accessible as Supplementary Video 1.
Collapse
Affiliation(s)
| | | | - Taylor L Bobrow
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Gulfize Coskun
- Institute of Biomedical Engineering, Bogazici University, Turkey
| | - Kagan Incetan
- Institute of Biomedical Engineering, Bogazici University, Turkey
| | | | - Faisal Mahmood
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Data Science, Dana Farber Cancer Institute, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
| | - Eva Curto
- Institute for Systems and Robotics, University of Coimbra, Portugal
| | - Luis Perdigoto
- Institute for Systems and Robotics, University of Coimbra, Portugal
| | - Marina Oliveira
- Institute for Systems and Robotics, University of Coimbra, Portugal
| | - Hasan Sahin
- Institute of Biomedical Engineering, Bogazici University, Turkey
| | - Helder Araujo
- Institute for Systems and Robotics, University of Coimbra, Portugal
| | - Henrique Alexandrino
- Faculty of Medicine, Clinical Academic Center of Coimbra, University of Coimbra, Coimbra, Portugal
| | - Nicholas J Durr
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Hunter B Gilbert
- Department of Mechanical and Industrial Engineering, Louisiana State University, Baton Rouge, LA, USA
| | - Mehmet Turan
- Institute of Biomedical Engineering, Bogazici University, Turkey.
| |
Collapse
|
26
|
Babu C, Chandy DA. A Review on Lossless Compression Techniques for Wireless Capsule Endoscopic Data. Curr Med Imaging 2021; 17:27-38. [PMID: 32324517 DOI: 10.2174/1573405616666200423084725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2019] [Revised: 02/07/2020] [Accepted: 02/27/2020] [Indexed: 11/22/2022]
Abstract
BACKGROUND The videos produced during wireless capsule endoscopy have larger data size causing difficulty in transmission with limited bandwidth. The constraint on wireless capsule endoscopy hinders the performance of the compression module. OBJECTIVES The objectives of this paper are as follows: (i) to conduct an extensive review of the lossless compression techniques and (ii) to find out the limitations of the existing system and the possibilities for improvement. METHODS The literature review was conducted with a focus on the compression schemes satisfying minimum computational complexity, less power dissipation and low memory requirements for hardware implementation. A thorough study of various lossless compression techniques was conducted under two perspectives, i.e., techniques applied to Bayer CFA and RGB images. The detail of the various stages of wireless capsule endoscopy compression was investigated to have a better understanding. The suitable performance metrics for evaluating the compression techniques were listed from various literature studies. RESULTS In addition to the Gastrolab database, WEO clinical endoscopy atlas and Gastrointestinal atlas were found to be better alternatives for experimentation. Pre-processing operations, especially new subsampling patterns need to be given more focus to exploit the redundancies in the images. Investigations showed that encoder module can be modified to bring more improvement towards compression. The real-time endoscopy still exists as a promising area for exploration. CONCLUSION This review presents a research update on the details of wireless capsule endoscopy compression together with the findings as an eye-opener and guidance for further research.
Collapse
Affiliation(s)
- Caren Babu
- Department of Electronics and Communication Engineering, Karunya Institute of Technology and Sciences, Coimbatore, India
| | - D Abraham Chandy
- Department of Electronics and Communication Engineering, Karunya Institute of Technology and Sciences, Coimbatore, India
| |
Collapse
|
27
|
Caroppo A, Leone A, Siciliano P. Deep transfer learning approaches for bleeding detection in endoscopy images. Comput Med Imaging Graph 2021; 88:101852. [PMID: 33493998 DOI: 10.1016/j.compmedimag.2020.101852] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 12/17/2020] [Accepted: 12/18/2020] [Indexed: 12/17/2022]
Abstract
Wireless capsule endoscopy is a non-invasive, wireless imaging tool that has developed rapidly over the last several years. One of the main limiting factors using this technology is that it produces a huge number of images, whose analysis, to be done by a doctor, is an extremely time-consuming process. In this research area, the management of this problem has been addressed with the development of Computer-aided Diagnosis systems thanks to which the automatic inspection and analysis of images acquired by the capsule has clearly improved. Recently, a big advance in classification of endoscopic images is achieved with the emergence of deep learning methods. The proposed expert system employs three pre-trained deep convolutional neural networks for feature extraction. In order to construct efficient feature sets, the features from VGG19, InceptionV3 and ResNet50 models are then selected and fused using the minimum Redundancy Maximum Relevance method and different fusion rules. Finally, supervised machine learning algorithms are employed to classify the images using the extracted features into two categories: bleeding and nonbleeding images. For performance evaluation a series of experiments are performed on two standard benchmark datasets. It has been observed that the proposed architecture outclass the single deep learning architectures, with an average accuracy in detection bleeding regions of 97.65 % and 95.70 % on well-known state-of-the-art datasets considering three different fusion rules, with the best combination in terms of accuracy and training time obtained using mean value pooling as fusion rule and Support Vector Machine as classifier.
Collapse
Affiliation(s)
- Andrea Caroppo
- Institute for Microelectronics and Microsystems, National Research Council of Italy, Lecce 73100, Italy.
| | - Alessandro Leone
- Institute for Microelectronics and Microsystems, National Research Council of Italy, Lecce 73100, Italy.
| | - Pietro Siciliano
- Institute for Microelectronics and Microsystems, National Research Council of Italy, Lecce 73100, Italy.
| |
Collapse
|
28
|
Trasolini R, Byrne MF. Artificial intelligence and deep learning for small bowel capsule endoscopy. Dig Endosc 2021; 33:290-297. [PMID: 33211357 DOI: 10.1111/den.13896] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 11/16/2020] [Indexed: 12/20/2022]
Abstract
Capsule endoscopy is ideally suited to artificial intelligence-based interpretation given its reliance on pattern recognition in still images. Time saving viewing modes and lesion detection features currently available rely on machine learning algorithms, a form of artificial intelligence. Current software necessitates close human supervision given poor sensitivity relative to an expert reader. However, with the advent of deep learning, artificial intelligence is becoming increasingly reliable and will be increasingly relied upon. We review the major advances in artificial intelligence for capsule endoscopy in recent publications and briefly review artificial intelligence development for historical understanding. Importantly, recent advancements in artificial intelligence have not yet been incorporated into practice and it is immature to judge the potential of this technology based on current platforms. Remaining regulatory and standardization hurdles are being overcome and artificial intelligence-based clinical applications are likely to proliferate rapidly.
Collapse
Affiliation(s)
- Roberto Trasolini
- Department of Medicine, The University of British Columbia, Vancouver, Canada
| | - Michael F Byrne
- Department of Medicine, The University of British Columbia, Vancouver, Canada
| |
Collapse
|
29
|
Artificial Intelligence in Medicine. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_163-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
30
|
Tontini GE, Neumann H. Artificial intelligence: Thinking outside the box. Best Pract Res Clin Gastroenterol 2020; 52-53:101720. [PMID: 34172247 DOI: 10.1016/j.bpg.2020.101720] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Revised: 12/01/2020] [Accepted: 12/03/2020] [Indexed: 01/31/2023]
Abstract
Artificial intelligence (AI) for luminal gastrointestinal endoscopy is rapidly evolving. To date, most applications have focused on colon polyp detection and characterization. However, the potential of AI to revolutionize our current practice in endoscopy is much more broadly positioned. In this review article, the Authors provide new ideas on how AI might help endoscopists in the future to rediscover endoscopy practice.
Collapse
Affiliation(s)
- Gian Eugenio Tontini
- Gastroenterology and Endoscopy Unit, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy; Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | - Helmut Neumann
- Department of Interdisciplinary Endoscopy, University Hospital Mainz, Mainz, Germany.
| |
Collapse
|
31
|
Xing X, Yuan Y, Meng MQH. Zoom in Lesions for Better Diagnosis: Attention Guided Deformation Network for WCE Image Classification. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:4047-4059. [PMID: 32746146 DOI: 10.1109/tmi.2020.3010102] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Wireless capsule endoscopy (WCE) is a novel imaging tool that allows noninvasive visualization of the entire gastrointestinal (GI) tract without causing discomfort to patients. Convolutional neural networks (CNNs), though perform favorably against traditional machine learning methods, show limited capacity in WCE image classification due to the small lesions and background interference. To overcome these limits, we propose a two-branch Attention Guided Deformation Network (AGDN) for WCE image classification. Specifically, the attention maps of branch1 are utilized to guide the amplification of lesion regions on the input images of branch2, thus leading to better representation and inspection of the small lesions. What's more, we devise and insert Third-order Long-range Feature Aggregation (TLFA) modules into the network. By capturing long-range dependencies and aggregating contextual features, TLFAs endow the network with a global contextual view and stronger feature representation and discrimination capability. Furthermore, we propose a novel Deformation based Attention Consistency (DAC) loss to refine the attention maps and achieve the mutual promotion of the two branches. Finally, the global feature embeddings from the two branches are fused to make image label predictions. Extensive experiments show that the proposed AGDN outperforms state-of-the-art methods with an overall classification accuracy of 91.29% on two public WCE datasets. The source code is available at https://github.com/hathawayxxh/WCE-AGDN.
Collapse
|
32
|
Jain S, Seal A, Ojha A, Krejcar O, Bureš J, Tachecí I, Yazidi A. Detection of abnormality in wireless capsule endoscopy images using fractal features. Comput Biol Med 2020; 127:104094. [PMID: 33152668 DOI: 10.1016/j.compbiomed.2020.104094] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2020] [Revised: 10/23/2020] [Accepted: 10/23/2020] [Indexed: 12/14/2022]
Abstract
One of the most recent non-invasive technologies to examine the gastrointestinal tract is wireless capsule endoscopy (WCE). As there are thousands of endoscopic images in an 8-15 h long video, an evaluator has to pay constant attention for a relatively long time (60-120 min). Therefore the possibility of the presence of pathological findings in a few images (displayed for evaluation for a few seconds only) brings a significant risk of missing the pathology with all negative consequences for the patient. Hence, manually reviewing a video to identify abnormal images is not only a tedious and time consuming task that overwhelms human attention but also is error prone. In this paper, a method is proposed for the automatic detection of abnormal WCE images. The differential box counting method is used for the extraction of fractal dimension (FD) of WCE images and the random forest based ensemble classifier is used for the identification of abnormal frames. The FD is a well-known technique for extraction of features related to texture, smoothness, and roughness. In this paper, FDs are extracted from pixel-blocks of WCE images and are fed to the classifier for identification of images with abnormalities. To determine a suitable pixel block size for FD feature extraction, various sizes of blocks are considered and are fed into six frequently used classifiers separately, and the block size of 7×7 giving the best performance is empirically determined. Further, the selection of the random forest ensemble classifier is also done using the same empirical study. Performance of the proposed method is evaluated on two datasets containing WCE frames. Results demonstrate that the proposed method outperforms some of the state-of-the-art methods with AUC of 85% and 99% on Dataset-I and Dataset-II respectively.
Collapse
Affiliation(s)
- Samir Jain
- PDPM Indian Institute of Information Technology Design and Manufacturing, Jabalpur, 482005, India
| | - Ayan Seal
- PDPM Indian Institute of Information Technology Design and Manufacturing, Jabalpur, 482005, India; Center for Basic and Applied Research, Faculty of Informatics and Management, University of Hradec Kralove, Hradecka, 1249, Hradec Kralove, 50003, Czech Republic.
| | - Aparajita Ojha
- PDPM Indian Institute of Information Technology Design and Manufacturing, Jabalpur, 482005, India
| | - Ondrej Krejcar
- Center for Basic and Applied Research, Faculty of Informatics and Management, University of Hradec Kralove, Hradecka, 1249, Hradec Kralove, 50003, Czech Republic; Malaysia Japan International Institute of Technology, Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, 54100, Kuala Lumpur, Malaysia
| | - Jan Bureš
- Second Department of Internal Medicine-Gastroenterology, Charles University, Faculty of Medicine in Hradec Kralove, University Hospital Hradec Kralove, Sokolska 581, Hradec Kralove, 50005, Czech Republic
| | - Ilja Tachecí
- Second Department of Internal Medicine-Gastroenterology, Charles University, Faculty of Medicine in Hradec Kralove, University Hospital Hradec Kralove, Sokolska 581, Hradec Kralove, 50005, Czech Republic
| | - Anis Yazidi
- Artificial Intelligence Lab, Oslo Metropolitan University, 460167, Norway
| |
Collapse
|
33
|
Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc 2020; 92:831-839.e8. [PMID: 32334015 DOI: 10.1016/j.gie.2020.04.039] [Citation(s) in RCA: 95] [Impact Index Per Article: 23.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2019] [Accepted: 04/13/2020] [Indexed: 12/11/2022]
Abstract
BACKGROUND AND AIMS Deep learning is an innovative algorithm based on neural networks. Wireless capsule endoscopy (WCE) is considered the criterion standard for detecting small-bowel diseases. Manual examination of WCE is time-consuming and can benefit from automatic detection using artificial intelligence (AI). We aimed to perform a systematic review of the current literature pertaining to deep learning implementation in WCE. METHODS We conducted a search in PubMed for all original publications on the subject of deep learning applications in WCE published between January 1, 2016 and December 15, 2019. Evaluation of the risk of bias was performed using tailored Quality Assessment of Diagnostic Accuracy Studies-2. Pooled sensitivity and specificity were calculated. Summary receiver operating characteristic curves were plotted. RESULTS Of the 45 studies retrieved, 19 studies were included. All studies were retrospective. Deep learning applications for WCE included detection of ulcers, polyps, celiac disease, bleeding, and hookworm. Detection accuracy was above 90% for most studies and diseases. Pooled sensitivity and specificity for ulcer detection were .95 (95% confidence interval [CI], .89-.98) and .94 (95% CI, .90-.96), respectively. Pooled sensitivity and specificity for bleeding or bleeding source were .98 (95% CI, .96-.99) and .99 (95% CI, .97-.99), respectively. CONCLUSIONS Deep learning has achieved excellent performance for the detection of a range of diseases in WCE. Notwithstanding, current research is based on retrospective studies with a high risk of bias. Thus, future prospective, multicenter studies are necessary for this technology to be implemented in the clinical use of WCE.
Collapse
|
34
|
Borgli H, Thambawita V, Smedsrud PH, Hicks S, Jha D, Eskeland SL, Randel KR, Pogorelov K, Lux M, Nguyen DTD, Johansen D, Griwodz C, Stensland HK, Garcia-Ceja E, Schmidt PT, Hammer HL, Riegler MA, Halvorsen P, de Lange T. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 2020; 7:283. [PMID: 32859981 PMCID: PMC7455694 DOI: 10.1038/s41597-020-00622-y] [Citation(s) in RCA: 92] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Accepted: 07/21/2020] [Indexed: 02/08/2023] Open
Abstract
Artificial intelligence is currently a hot topic in medicine. However, medical data is often sparse and hard to obtain due to legal restrictions and lack of medical personnel for the cumbersome and tedious process to manually label training data. These constraints make it difficult to develop systems for automatic analysis, like detecting disease or other lesions. In this respect, this article presents HyperKvasir, the largest image and video dataset of the gastrointestinal tract available today. The data is collected during real gastro- and colonoscopy examinations at Bærum Hospital in Norway and partly labeled by experienced gastrointestinal endoscopists. The dataset contains 110,079 images and 374 videos, and represents anatomical landmarks as well as pathological and normal findings. The total number of images and video frames together is around 1 million. Initial experiments demonstrate the potential benefits of artificial intelligence-based computer-assisted diagnosis systems. The HyperKvasir dataset can play a valuable role in developing better algorithms and computer-assisted examination systems not only for gastro- and colonoscopy, but also for other fields in medicine.
Collapse
Affiliation(s)
- Hanna Borgli
- SimulaMet, Oslo, Norway
- University of Oslo, Oslo, Norway
| | | | - Pia H Smedsrud
- SimulaMet, Oslo, Norway
- University of Oslo, Oslo, Norway
- Augere Medical AS, Oslo, Norway
| | - Steven Hicks
- SimulaMet, Oslo, Norway
- Oslo Metropolitan University, Oslo, Norway
| | - Debesh Jha
- SimulaMet, Oslo, Norway
- UIT The Arctic University of Norway, Tromsø, Norway
| | | | | | | | | | | | - Dag Johansen
- UIT The Arctic University of Norway, Tromsø, Norway
| | | | - Håkon K Stensland
- University of Oslo, Oslo, Norway
- Simula Research Laboratory, Oslo, Norway
| | | | - Peter T Schmidt
- Department of Medicine (Solna), Karolinska Institutet, Stockholm, Sweden
- Department of Medicine, Ersta hospital, Stockholm, Sweden
| | - Hugo L Hammer
- SimulaMet, Oslo, Norway
- Oslo Metropolitan University, Oslo, Norway
| | | | - Pål Halvorsen
- SimulaMet, Oslo, Norway.
- Oslo Metropolitan University, Oslo, Norway.
| | - Thomas de Lange
- Department of Medical Research, Bærum Hospital, Bærum, Norway
- Augere Medical AS, Oslo, Norway
- Medical Department, Sahlgrenska University Hospital-Mölndal, Mölndal, Sweden
| |
Collapse
|
35
|
Namikawa K, Hirasawa T, Yoshio T, Fujisaki J, Ozawa T, Ishihara S, Aoki T, Yamada A, Koike K, Suzuki H, Tada T. Utilizing artificial intelligence in endoscopy: a clinician's guide. Expert Rev Gastroenterol Hepatol 2020; 14:689-706. [PMID: 32500760 DOI: 10.1080/17474124.2020.1779058] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) that surpasses human ability in image recognition is expected to be applied in the field of gastrointestinal endoscopes. Accordingly, its research and development (R &D) is being actively conducted. With the development of endoscopic diagnosis, there is a shortage of specialists who can perform high-precision endoscopy. We will examine whether AI with excellent image recognition ability can overcome this problem. AREAS COVERED Since 2016, papers on artificial intelligence using convolutional neural network (CNN in other word Deep Learning) have been published. CNN is generally capable of more accurate detection and classification than conventional machine learning. This is a review of papers using CNN in the gastrointestinal endoscopy area, along with the reasons why AI is required in clinical practice. We divided this review into four parts: stomach, esophagus, large intestine, and capsule endoscope (small intestine). EXPERT OPINION Potential applications for the AI include colorectal polyp detection and differentiation, gastric and esophageal cancer detection, and lesion detection in capsule endoscopy. The accuracy of endoscopic diagnosis will increase if the AI and endoscopist perform the endoscopy together.
Collapse
Affiliation(s)
- Ken Namikawa
- Department of Gastroenterology, Cancer Institute Hospital, Japanese Foundation for Cancer Research , Tokyo, Japan
| | - Toshiaki Hirasawa
- Department of Gastroenterology, Cancer Institute Hospital, Japanese Foundation for Cancer Research , Tokyo, Japan
| | - Toshiyuki Yoshio
- Department of Gastroenterology, Cancer Institute Hospital, Japanese Foundation for Cancer Research , Tokyo, Japan
| | - Junko Fujisaki
- Department of Gastroenterology, Cancer Institute Hospital, Japanese Foundation for Cancer Research , Tokyo, Japan
| | - Tsuyoshi Ozawa
- Department of Surgery, Teikyo University School of Medicine , Tokyo, Japan
| | - Soichiro Ishihara
- Department of Surgical Oncology, Graduate School of Medicine, the University of Tokyo , Tokyo, Japan
| | - Tomonori Aoki
- Department of Gastroenterology, Graduate School of Medicine, the University of Tokyo , Tokyo, Japan
| | - Atsuo Yamada
- Department of Gastroenterology, Graduate School of Medicine, the University of Tokyo , Tokyo, Japan
| | - Kazuhiko Koike
- Department of Gastroenterology, Graduate School of Medicine, the University of Tokyo , Tokyo, Japan
| | - Hideo Suzuki
- Department of Gastroenterology, Graduate School of Institute Clinical Medicine, University of Tsukuba , Ibaraki, Japan
| | - Tomohiro Tada
- Department of Surgical Oncology, Graduate School of Medicine, the University of Tokyo , Tokyo, Japan.,AI Medical Service Inc ., Tokyo, Japan.,Tada Tomohiro the Institute of Gastroenterology and Proctology , Saitama, Japan
| |
Collapse
|
36
|
Affiliation(s)
- Anastasios Koulaouzidis
- Endoscopy Unit, Centre for Liver & Digestive Disorders, The Royal Infirmary of Edinburgh, Scotland
| | - Ervin Toth
- Department of Gastroenterology, Skåne University Hospital, Malmö, Lund University, Sweden
| |
Collapse
|
37
|
Leenhardt R, Li C, Le Mouel JP, Rahmi G, Saurin JC, Cholet F, Boureille A, Amiot X, Delvaux M, Duburque C, Leandri C, Gérard R, Lecleire S, Mesli F, Nion-Larmurier I, Romain O, Sacher-Huvelin S, Simon-Shane C, Vanbiervliet G, Marteau P, Histace A, Dray X. CAD-CAP: a 25,000-image database serving the development of artificial intelligence for capsule endoscopy. Endosc Int Open 2020; 8:E415-E420. [PMID: 32118115 PMCID: PMC7035135 DOI: 10.1055/a-1035-9088] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Accepted: 09/16/2019] [Indexed: 02/06/2023] Open
Abstract
Background and study aims Capsule endoscopy (CE) is the preferred method for small bowel (SB) exploration. With a mean number of 50,000 SB frames per video, SBCE reading is time-consuming and tedious (30 to 60 minutes per video). We describe a large, multicenter database named CAD-CAP (Computer-Assisted Diagnosis for CAPsule Endoscopy, CAD-CAP). This database aims to serve the development of CAD tools for CE reading. Materials and methods Twelve French endoscopy centers were involved. All available third-generation SB-CE videos (Pillcam, Medtronic) were retrospectively selected from these centers and deidentified. Any pathological frame was extracted and included in the database. Manual segmentation of findings within these frames was performed by two pre-med students trained and supervised by an expert reader. All frames were then classified by type and clinical relevance by a panel of three expert readers. An automated extraction process was also developed to create a dataset of normal, proofread, control images from normal, complete, SB-CE videos. Results Four-thousand-one-hundred-and-seventy-four SB-CE were included. Of them, 1,480 videos (35 %) containing at least one pathological finding were selected. Findings from 5,184 frames (with their short video sequences) were extracted and delimited: 718 frames with fresh blood, 3,097 frames with vascular lesions, and 1,369 frames with inflammatory and ulcerative lesions. Twenty-thousand normal frames were extracted from 206 SB-CE normal videos. CAD-CAP has already been used for development of automated tools for angiectasia detection and also for two international challenges on medical computerized analysis.
Collapse
Affiliation(s)
| | - Cynthia Li
- Drexel University, College of Arts & Sciences, Philadelphia, Pennsylvania, United States
| | - Jean-Philippe Le Mouel
- Gastroenterology, Amiens University Hospital, Université de Picardie Jules Verne, Amiens, France
| | - Gabriel Rahmi
- Georges Pompidou European Hospital, APHP, Department of Gastroenterology and Endoscopy, Paris, France
| | - Jean Christophe Saurin
- Department of Endoscopy and Gastroenterology, Pavillon L, Hôpital Edouard Herriot, Lyon, France
| | - Franck Cholet
- Digestive Endoscopy Unit, University Hospital, Brest, France
| | - Arnaud Boureille
- Department of Hepato-Gastroenterology, Institut des Maladies de l'Appareil Digestif, Nantes, France
| | - Xavier Amiot
- Tenon Hospital, Gastroenterology Department, Paris, France
| | - Michel Delvaux
- CHU Strasbourg, Gastroenterology Department, Strasbourg, France
| | | | - Chloé Leandri
- Cochin Hospital Gastroenterology Department, Paris, France
| | - Romain Gérard
- CHRU Lille, Gastroenterology Department, Lille, France
| | | | - Farida Mesli
- CHU Henri Mondor, Gastroenterology Department, Creteil, France
| | | | - Olivier Romain
- ETIS, Université de Cergy-Pontoise, ENSEA, CNRS, Cergy-Pontoise Cedex, France
| | - Sylvie Sacher-Huvelin
- Department of Hepato-Gastroenterology, Institut des Maladies de l'Appareil Digestif, Nantes, France
| | - Camille Simon-Shane
- ETIS, Université de Cergy-Pontoise, ENSEA, CNRS, Cergy-Pontoise Cedex, France
| | | | | | - Aymeric Histace
- ETIS, Université de Cergy-Pontoise, ENSEA, CNRS, Cergy-Pontoise Cedex, France
| | - Xavier Dray
- Sorbonne University, Endoscopy Unit,ETIS, Université de Cergy-Pontoise, ENSEA, CNRS, Cergy-Pontoise Cedex, France,Corresponding author Pr Xavier Dray Hopital Saint-Antoine – Endoscopy Unit184 Rue du Faubourg Saint-AntoineParis 75012France
| |
Collapse
|
38
|
Tontini GE, Manfredi G, Orlando S, Neumann H, Vecchi M, Buscarini E, Elli L. Endoscopic ultrasonography and small-bowel endoscopy: Present and future. Dig Endosc 2019; 31:627-643. [PMID: 31090965 DOI: 10.1111/den.13429] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/08/2019] [Accepted: 05/13/2019] [Indexed: 12/12/2022]
Abstract
Over the last decade, impressive technological advances have occurred in ultrasonography and small-bowel endoscopy. Nowadays, endoscopic ultrasonography is an essential diagnostic tool and a therapeutic weapon for pancreatobiliary disorders. Capsule endoscopy and device-assisted enteroscopy have quickly become the reference standard for the diagnosis of small-bowel luminal diseases, thereby leading to radical changes in diagnostic and therapeutic pathways. We herein provide an up-to-date overview of the latest advances in endoscopic ultrasonography and small-bowel endoscopy, focusing on the emerging paradigms and technological innovations that might improve clinical practice in the near future.
Collapse
Affiliation(s)
- Gian Eugenio Tontini
- Gastroenterology and Endoscopy Unit, Foundation IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy.,Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | | | - Stefania Orlando
- Gastroenterology and Endoscopy Unit, Foundation IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy.,Digestive Endoscopy and Gastroenterology Unit, ASST of Cremona, Cremona, Italy
| | - Helmut Neumann
- Department of Medicine I, University Medical Center Mainz, Mainz, Germany
| | - Maurizio Vecchi
- Gastroenterology and Endoscopy Unit, Foundation IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy.,Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | | | - Luca Elli
- Gastroenterology and Endoscopy Unit, Foundation IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| |
Collapse
|
39
|
Diamantis DE, Iakovidis DK, Koulaouzidis A. Look-behind fully convolutional neural network for computer-aided endoscopy. Biomed Signal Process Control 2019. [DOI: 10.1016/j.bspc.2018.12.005] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
|
40
|
Vasilakakis M, Koulaouzidis A, Yung DE, Plevris JN, Toth E, Iakovidis DK. Follow-up on: optimizing lesion detection in small bowel capsule endoscopy and beyond: from present problems to future solutions. Expert Rev Gastroenterol Hepatol 2019; 13:129-141. [PMID: 30791780 DOI: 10.1080/17474124.2019.1553616] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
This review presents noteworthy advances in clinical and experimental Capsule Endoscopy (CE), focusing on the progress that has been reported over the last 5 years since our previous review on the subject. Areas covered: This study presents the commercially available CE platforms, as well as the advances made in optimizing the diagnostic capabilities of CE. The latter includes recent concept and prototype capsule endoscopes, medical approaches to improve diagnostic yield, and progress in software for enhancing visualization, abnormality detection, and lesion localization. Expert commentary: Currently, moving through the second decade of CE evolution, there are still several open issues and remarkable challenges to overcome.
Collapse
Affiliation(s)
- Michael Vasilakakis
- a Department of Computer Science and Biomedical Informatics , University of Thessaly , Lamia , Greece
| | - Anastasios Koulaouzidis
- b Endoscopy Unit , The Royal Infirmary of Edinburgh , Edinburgh , Scotland.,c Department of Clinical Sciences , Lund University , Malmö , Sweden
| | - Diana E Yung
- b Endoscopy Unit , The Royal Infirmary of Edinburgh , Edinburgh , Scotland
| | - John N Plevris
- b Endoscopy Unit , The Royal Infirmary of Edinburgh , Edinburgh , Scotland
| | - Ervin Toth
- c Department of Clinical Sciences , Lund University , Malmö , Sweden.,d Section of Gastroenterology, Department of Clinical Sciences , Skåne University Hospital Malmö , Malmö , Sweden
| | - Dimitris K Iakovidis
- a Department of Computer Science and Biomedical Informatics , University of Thessaly , Lamia , Greece
| |
Collapse
|
41
|
Guo X, Yuan Y. Triple ANet: Adaptive Abnormal-aware Attention Network for WCE Image Classification. LECTURE NOTES IN COMPUTER SCIENCE 2019. [DOI: 10.1007/978-3-030-32239-7_33] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
|
42
|
de Lange T, Halvorsen P, Riegler M. Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy. World J Gastroenterol 2018; 24:5057-5062. [PMID: 30568383 PMCID: PMC6288655 DOI: 10.3748/wjg.v24.i45.5057] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/07/2018] [Revised: 10/25/2018] [Accepted: 11/02/2018] [Indexed: 02/06/2023] Open
Abstract
Assisted diagnosis using artificial intelligence has been a holy grail in medical research for many years, and recent developments in computer hardware have enabled the narrower area of machine learning to equip clinicians with potentially useful tools for computer assisted diagnosis (CAD) systems. However, training and assessing a computer's ability to diagnose like a human are complex tasks, and successful outcomes depend on various factors. We have focused our work on gastrointestinal (GI) endoscopy because it is a cornerstone for diagnosis and treatment of diseases of the GI tract. About 2.8 million luminal GI (esophageal, stomach, colorectal) cancers are detected globally every year, and although substantial technical improvements in endoscopes have been made over the last 10-15 years, a major limitation of endoscopic examinations remains operator variation. This translates into a substantial inter-observer variation in the detection and assessment of mucosal lesions, causing among other things an average polyp miss-rate of 20% in the colon and thus the subsequent development of a number of post-colonoscopy colorectal cancers. CAD systems might eliminate this variation and lead to more accurate diagnoses. In this editorial, we point out some of the current challenges in the development of efficient computer-based digital assistants. We give examples of proposed tools using various techniques, identify current challenges, and give suggestions for the development and assessment of future CAD systems.
Collapse
Affiliation(s)
- Thomas de Lange
- Department of Transplantation, Oslo University Hospital, Oslo 0424, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo 0316, Norway
| | - Pål Halvorsen
- Center for Digital Engineering Simula Metropolitan, Fornebu 1364, Norway
- Department for Informatics, University of Oslo, Oslo 0316, Norway
| | - Michael Riegler
- Center for Digital Engineering Simula Metropolitan, Fornebu 1364, Norway
- Department for Informatics, University of Oslo, Oslo 0316, Norway
| |
Collapse
|
43
|
Iakovidis DK, Georgakopoulos SV, Vasilakakis M, Koulaouzidis A, Plagianakos VP. Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2196-2210. [PMID: 29994763 DOI: 10.1109/tmi.2018.2837002] [Citation(s) in RCA: 68] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.
Collapse
|
44
|
DINOSARC: Color Features Based on Selective Aggregation of Chromatic Image Components for Wireless Capsule Endoscopy. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2018; 2018:2026962. [PMID: 30250496 PMCID: PMC6140007 DOI: 10.1155/2018/2026962] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/31/2018] [Revised: 07/22/2018] [Accepted: 07/31/2018] [Indexed: 12/12/2022]
Abstract
Wireless Capsule Endoscopy (WCE) is a noninvasive diagnostic technique enabling the inspection of the whole gastrointestinal (GI) tract by capturing and wirelessly transmitting thousands of color images. Proprietary software "stitches" the images into videos for examination by accredited readers. However, the videos produced are of large length and consequently the reading task becomes harder and more prone to human errors. Automating the WCE reading process could contribute in both the reduction of the examination time and the improvement of its diagnostic accuracy. In this paper, we present a novel feature extraction methodology for automated WCE image analysis. It aims at discriminating various kinds of abnormalities from the normal contents of WCE images, in a machine learning-based classification framework. The extraction of the proposed features involves an unsupervised color-based saliency detection scheme which, unlike current approaches, combines both point and region-level saliency information and the estimation of local and global image color descriptors. The salient point detection process involves estimation of DIstaNces On Selective Aggregation of chRomatic image Components (DINOSARC). The descriptors are extracted from superpixels by coevaluating both point and region-level information. The main conclusions of the experiments performed on a publicly available dataset of WCE images are (a) the proposed salient point detection scheme results in significantly less and more relevant salient points; (b) the proposed descriptors are more discriminative than relevant state-of-the-art descriptors, promising a wider adoption of the proposed approach for computer-aided diagnosis in WCE.
Collapse
|
45
|
Weakly supervised multilabel classification for semantic interpretation of endoscopy video frames. EVOLVING SYSTEMS 2018. [DOI: 10.1007/s12530-018-9236-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
|