1
|
Tahir AM, Guo L, Ward RK, Yu X, Rideout A, Hore M, Wang ZJ. Explainable machine learning for assessing upper respiratory tract of racehorses from endoscopy videos. Comput Biol Med 2024; 181:109030. [PMID: 39173488 DOI: 10.1016/j.compbiomed.2024.109030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2023] [Revised: 06/20/2024] [Accepted: 08/13/2024] [Indexed: 08/24/2024]
Abstract
Laryngeal hemiplegia (LH) is a major upper respiratory tract (URT) complication in racehorses. Endoscopy imaging of horse throat is a gold standard for URT assessment. However, current manual assessment faces several challenges, stemming from the poor quality of endoscopy videos and subjectivity of manual grading. To overcome such limitations, we propose an explainable machine learning (ML)-based solution for efficient URT assessment. Specifically, a cascaded YOLOv8 architecture is utilized to segment the key semantic regions and landmarks per frame. Several spatiotemporal features are then extracted from key landmarks points and fed to a decision tree (DT) model to classify LH as Grade 1,2,3 or 4 denoting absence of LH, mild, moderate, and severe LH, respectively. The proposed method, validated through 5-fold cross-validation on 107 videos, showed promising performance in classifying different LH grades with 100%, 91.18%, 94.74% and 100% sensitivity values for Grade 1 to 4, respectively. Further validation on an external dataset of 72 cases confirmed its generalization capability with 90%, 80.95%, 100%, and 100% sensitivity values for Grade 1 to 4, respectively. We introduced several explainability related assessment functions, including: (i) visualization of YOLOv8 output to detect landmark estimation errors which can affect the final classification, (ii) time-series visualization to assess video quality, and (iii) backtracking of the DT output to identify borderline cases. We incorporated domain knowledge (e.g., veterinarian diagnostic procedures) into the proposed ML framework. This provides an assistive tool with clinical-relevance and explainability that can ease and speed up the URT assessment by veterinarians.
Collapse
Affiliation(s)
- Anas Mohammed Tahir
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Li Guo
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Rabab K Ward
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Xinhui Yu
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Andrew Rideout
- Point To Point Research & Development, Vancouver, BC, Canada.
| | - Michael Hore
- Hagyard Equine Medical Institute, Lexington, KY, USA.
| | - Z Jane Wang
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| |
Collapse
|
2
|
Daneshpajooh V, Ahmad D, Toth J, Bascom R, Higgins WE. Automatic lesion detection for narrow-band imaging bronchoscopy. J Med Imaging (Bellingham) 2024; 11:036002. [PMID: 38827776 PMCID: PMC11138083 DOI: 10.1117/1.jmi.11.3.036002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 04/04/2024] [Accepted: 05/14/2024] [Indexed: 06/05/2024] Open
Abstract
Purpose Early detection of cancer is crucial for lung cancer patients, as it determines disease prognosis. Lung cancer typically starts as bronchial lesions along the airway walls. Recent research has indicated that narrow-band imaging (NBI) bronchoscopy enables more effective bronchial lesion detection than other bronchoscopic modalities. Unfortunately, NBI video can be hard to interpret because physicians currently are forced to perform a time-consuming subjective visual search to detect bronchial lesions in a long airway-exam video. As a result, NBI bronchoscopy is not regularly used in practice. To alleviate this problem, we propose an automatic two-stage real-time method for bronchial lesion detection in NBI video and perform a first-of-its-kind pilot study of the method using NBI airway exam video collected at our institution. Approach Given a patient's NBI video, the first method stage entails a deep-learning-based object detection network coupled with a multiframe abnormality measure to locate candidate lesions on each video frame. The second method stage then draws upon a Siamese network and a Kalman filter to track candidate lesions over multiple frames to arrive at final lesion decisions. Results Tests drawing on 23 patient NBI airway exam videos indicate that the method can process an incoming video stream at a real-time frame rate, thereby making the method viable for real-time inspection during a live bronchoscopic airway exam. Furthermore, our studies showed a 93% sensitivity and 86% specificity for lesion detection; this compares favorably to a sensitivity and specificity of 80% and 84% achieved over a series of recent pooled clinical studies using the current time-consuming subjective clinical approach. Conclusion The method shows potential for robust lesion detection in NBI video at a real-time frame rate. Therefore, it could help enable more common use of NBI bronchoscopy for bronchial lesion detection.
Collapse
Affiliation(s)
- Vahid Daneshpajooh
- The Pennsylvania State University, School of Electrical Engineering and Computer Science, University Park, Pennsylvania, United States
| | - Danish Ahmad
- The Pennsylvania State University, College of Medicine, Hershey, Pennsylvania, United States
| | - Jennifer Toth
- The Pennsylvania State University, College of Medicine, Hershey, Pennsylvania, United States
| | - Rebecca Bascom
- The Pennsylvania State University, College of Medicine, Hershey, Pennsylvania, United States
| | - William E. Higgins
- The Pennsylvania State University, School of Electrical Engineering and Computer Science, University Park, Pennsylvania, United States
| |
Collapse
|
3
|
Gan T, Jin Z, Yu L, Liang X, Zhang H, Ye X. Self-supervised representation learning using feature pyramid siamese networks for colorectal polyp detection. Sci Rep 2023; 13:21655. [PMID: 38066207 PMCID: PMC10709402 DOI: 10.1038/s41598-023-49057-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 12/04/2023] [Indexed: 12/18/2023] Open
Abstract
Colorectal cancer is a leading cause of cancer-related deaths globally. In recent years, the use of convolutional neural networks in computer-aided diagnosis (CAD) has facilitated simpler detection of early lesions like polyps during real-time colonoscopy. However, the majority of existing techniques require a large training dataset annotated by experienced experts. To alleviate the laborious task of image annotation and utilize the vast amounts of readily available unlabeled colonoscopy data to further improve the polyp detection ability, this study proposed a novel self-supervised representation learning method called feature pyramid siamese networks (FPSiam). First, a feature pyramid encoder module was proposed to effectively extract and fuse both local and global feature representations among colonoscopic images, which is important for dense prediction tasks like polyp detection. Next, a self-supervised visual feature representation containing the general feature of colonoscopic images is learned by the siamese networks. Finally, the feature representation will be transferred to the downstream colorectal polyp detection task. A total of 103 videos (861,400 frames), 100 videos (24,789 frames), and 60 videos (15,397 frames) in the LDPolypVideo dataset are used to pre-train, train, and test the performance of the proposed FPSiam and its counterparts, respectively. The experimental results have illustrated that our FPSiam approach obtains the optimal capability, which is better than that of other state-of-the-art self-supervised learning methods and is also higher than the method based on transfer learning by 2.3 mAP and 3.6 mAP for two typical detectors. In conclusion, FPSiam provides a cost-efficient solution for developing colorectal polyp detection systems, especially in conditions where only a small fraction of the dataset is labeled while the majority remains unlabeled. Besides, it also brings fresh perspectives into other endoscopic image analysis tasks.
Collapse
Affiliation(s)
- Tianyuan Gan
- Biosensor National Special Laboratory, College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, China
| | - Ziyi Jin
- Biosensor National Special Laboratory, College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, China
| | - Liangliang Yu
- Department of Gastroenterology, Endoscopy Center, Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, 310016, China
| | - Xiao Liang
- Department of General Surgery, Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, 310016, China
| | - Hong Zhang
- Biosensor National Special Laboratory, College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, China
| | - Xuesong Ye
- Biosensor National Special Laboratory, College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, China.
| |
Collapse
|
4
|
Nanni L, Fantozzi C, Loreggia A, Lumini A. Ensembles of Convolutional Neural Networks and Transformers for Polyp Segmentation. SENSORS (BASEL, SWITZERLAND) 2023; 23:4688. [PMID: 37430601 DOI: 10.3390/s23104688] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Revised: 04/29/2023] [Accepted: 05/09/2023] [Indexed: 07/12/2023]
Abstract
In the realm of computer vision, semantic segmentation is the task of recognizing objects in images at the pixel level. This is done by performing a classification of each pixel. The task is complex and requires sophisticated skills and knowledge about the context to identify objects' boundaries. The importance of semantic segmentation in many domains is undisputed. In medical diagnostics, it simplifies the early detection of pathologies, thus mitigating the possible consequences. In this work, we provide a review of the literature on deep ensemble learning models for polyp segmentation and develop new ensembles based on convolutional neural networks and transformers. The development of an effective ensemble entails ensuring diversity between its components. To this end, we combined different models (HarDNet-MSEG, Polyp-PVT, and HSNet) trained with different data augmentation techniques, optimization methods, and learning rates, which we experimentally demonstrate to be useful to form a better ensemble. Most importantly, we introduce a new method to obtain the segmentation mask by averaging intermediate masks after the sigmoid layer. In our extensive experimental evaluation, the average performance of the proposed ensembles over five prominent datasets beat any other solution that we know of. Furthermore, the ensembles also performed better than the state-of-the-art on two of the five datasets, when individually considered, without having been specifically trained for them.
Collapse
Affiliation(s)
- Loris Nanni
- Department of Information Engineering, University of Padova, 35122 Padova, Italy
| | - Carlo Fantozzi
- Department of Information Engineering, University of Padova, 35122 Padova, Italy
| | - Andrea Loreggia
- Department of Information Engineering, University of Brescia, 25121 Brescia, Italy
| | - Alessandra Lumini
- Department of Computer Science and Engineering, University of Bologna, 40126 Bologna, Italy
| |
Collapse
|
5
|
Houwen BBSL, Nass KJ, Vleugels JLA, Fockens P, Hazewinkel Y, Dekker E. Comprehensive review of publicly available colonoscopic imaging databases for artificial intelligence research: availability, accessibility, and usability. Gastrointest Endosc 2023; 97:184-199.e16. [PMID: 36084720 DOI: 10.1016/j.gie.2022.08.043] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 08/24/2022] [Accepted: 08/30/2022] [Indexed: 01/28/2023]
Abstract
BACKGROUND AND AIMS Publicly available databases containing colonoscopic imaging data are valuable resources for artificial intelligence (AI) research. Currently, little is known regarding the available number and content of these databases. This review aimed to describe the availability, accessibility, and usability of publicly available colonoscopic imaging databases, focusing on polyp detection, polyp characterization, and quality of colonoscopy. METHODS A systematic literature search was performed in MEDLINE and Embase to identify AI studies describing publicly available colonoscopic imaging databases published after 2010. Second, a targeted search using Google's Dataset Search, Google Search, GitHub, and Figshare was done to identify databases directly. Databases were included if they contained data about polyp detection, polyp characterization, or quality of colonoscopy. To assess accessibility of databases, the following categories were defined: open access, open access with barriers, and regulated access. To assess the potential usability of the included databases, essential details of each database were extracted using a checklist derived from the Checklist for Artificial Intelligence in Medical Imaging. RESULTS We identified 22 databases with open access, 3 databases with open access with barriers, and 15 databases with regulated access. The 22 open access databases contained 19,463 images and 952 videos. Nineteen of these databases focused on polyp detection, localization, and/or segmentation; 6 on polyp characterization, and 3 on quality of colonoscopy. Only half of these databases have been used by other researcher to develop, train, or benchmark their AI system. Although technical details were in general well reported, important details such as polyp and patient demographics and the annotation process were under-reported in almost all databases. CONCLUSIONS This review provides greater insight on public availability of colonoscopic imaging databases for AI research. Incomplete reporting of important details limits the ability of researchers to assess the usability of current databases.
Collapse
Affiliation(s)
- Britt B S L Houwen
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Karlijn J Nass
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Jasper L A Vleugels
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Paul Fockens
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Yark Hazewinkel
- Department of Gastroenterology and Hepatology, Radboud University Nijmegen Medical Center, Radboud University of Nijmegen, Nijmegen, the Netherlands
| | - Evelien Dekker
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
6
|
Yue G, Han W, Li S, Zhou T, Lv J, Wang T. Automated polyp segmentation in colonoscopy images via deep network with lesion-aware feature selection and refinement. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103846] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
7
|
Su B, Wang Z, Gong Y, Li M, Teng Y, Yu S, Zong Y, Yao W, Wang J. Anal center detection and classification of perianal healthy condition. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
8
|
Adjei PE, Lonseko ZM, Du W, Zhang H, Rao N. Examining the effect of synthetic data augmentation in polyp detection and segmentation. Int J Comput Assist Radiol Surg 2022; 17:1289-1302. [PMID: 35678960 DOI: 10.1007/s11548-022-02651-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Accepted: 04/21/2022] [Indexed: 12/17/2022]
Abstract
PURPOSE As with several medical image analysis tasks based on deep learning, gastrointestinal image analysis is plagued with data scarcity, privacy concerns and an insufficient number of pathology samples. This study examines the generation and utility of synthetic samples of colonoscopy images with polyps for data augmentation. METHODS We modify and train a pix2pix model to generate synthetic colonoscopy samples with polyps to augment the original dataset. Subsequently, we create a variety of datasets by varying the quantity of synthetic samples and traditional augmentation samples, to train a U-Net network and Faster R-CNN model for segmentation and detection of polyps, respectively. We compare the performance of the models when trained with the resulting datasets in terms of F1 score, intersection over union, precision and recall. Further, we compare the performances of the models with unseen polyp datasets to assess their generalization ability. RESULTS The average F1 coefficient and intersection over union are improved with increasing number of synthetic samples in U-Net over all test datasets. The performance of the Faster R-CNN model is also improved in terms of polyp detection, while decreasing the false-negative rate. Further, the experimental results for polyp detection outperform similar studies in the literature on the ETIS-PolypLaribDB dataset. CONCLUSION By varying the quantity of synthetic and traditional augmentation, there is the potential to control the sensitivity of deep learning models in polyp segmentation and detection. Further, GAN-based augmentation is a viable option for improving the performance of models for polyp segmentation and detection.
Collapse
Affiliation(s)
- Prince Ebenezer Adjei
- Key Laboratory for Neuroinformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu, 610054, China.,School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.,Department of Computer Engineering, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana
| | - Zenebe Markos Lonseko
- Key Laboratory for Neuroinformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu, 610054, China.,School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Wenju Du
- Key Laboratory for Neuroinformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu, 610054, China.,School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Han Zhang
- Key Laboratory for Neuroinformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu, 610054, China.,School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Nini Rao
- Key Laboratory for Neuroinformation of Ministry of Education, University of Electronic Science and Technology of China, Chengdu, 610054, China. .,School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| |
Collapse
|
9
|
Luca M, Ciobanu A. Polyp detection in video colonoscopy using deep learning. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-219276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Video colonoscopy automatic processing is a challenge and further development of computer assisted diagnosis is very helpful in correctness assessment of the exam, in e-learning and training, for statistics on polyps’ malignity or in polyps’ survey. New devices and programming languages are emerging and deep learning begun already to furnish astonishing results, in the quest for high speed and optimal polyp detection software. This paper presents a successful attempt in detecting the intestinal polyps in real time video colonoscopy with deep learning, using Mobile Net.
Collapse
Affiliation(s)
- Mihaela Luca
- Institute of Computer Science, Romanian Academy Iaşi Branch, Iaşi, Romania
| | - Adrian Ciobanu
- Institute of Computer Science, Romanian Academy Iaşi Branch, Iaşi, Romania
| |
Collapse
|
10
|
Pan H, Cai M, Liao Q, Jiang Y, Liu Y, Zhuang X, Yu Y. Artificial Intelligence-Aid Colonoscopy Vs. Conventional Colonoscopy for Polyp and Adenoma Detection: A Systematic Review of 7 Discordant Meta-Analyses. Front Med (Lausanne) 2022; 8:775604. [PMID: 35096870 PMCID: PMC8792899 DOI: 10.3389/fmed.2021.775604] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Accepted: 12/20/2021] [Indexed: 12/16/2022] Open
Abstract
Objectives: Multiple meta-analyses which investigated the comparative efficacy and safety of artificial intelligence (AI)-aid colonoscopy (AIC) vs. conventional colonoscopy (CC) in the detection of polyp and adenoma have been published. However, a definitive conclusion has not yet been generated. This systematic review selected from discordant meta-analyses to draw a definitive conclusion about whether AIC is better than CC for the detection of polyp and adenoma. Methods: We comprehensively searched potentially eligible literature in PubMed, Embase, Cochrane library, and China National Knowledgement Infrastructure (CNKI) databases from their inceptions until to April 2021. Assessment of Multiple Systematic Reviews (AMSTAR) instrument was used to assess the methodological quality. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist was used to assess the reporting quality. Two investigators independently used the Jadad decision algorithm to select high-quality meta-analyses which summarized the best available evidence. Results: Seven meta-analyses met our selection criteria finally. AMSTAR score ranged from 8 to 10, and PRISMA score ranged from 23 to 26. According to the Jadad decision algorithm, two high-quality meta-analyses were selected. These two meta-analyses suggested that AIC was superior to CC for colonoscopy outcomes, especially for polyp detection rate (PDR) and adenoma detection rate (ADR). Conclusion: Based on the best available evidence, we conclude that AIC should be preferentially selected for the route screening of colorectal lesions because it has potential value of increasing the polyp and adenoma detection. However, the continued improvement of AIC in differentiating the shape and pathology of colorectal lesions is needed.
Collapse
Affiliation(s)
- Hui Pan
- Department of Endoscopy, Shanghai Jiangong Hospital, Shanghai, China
| | - Mingyan Cai
- Endoscopy Center, Zhongshan Hospital, Fudan University, Shanghai, China
| | - Qi Liao
- Department of Gastroenterology, Shanghai Jiangong Hospital, Shanghai, China
| | - Yong Jiang
- Department of Surgery, Shanghai Jiangong Hospital, Shanghai, China
| | - Yige Liu
- Department of Endoscopy, Shanghai Jiangong Hospital, Shanghai, China
| | - Xiaolong Zhuang
- Department of Endoscopy, Shanghai Jiangong Hospital, Shanghai, China
| | - Ying Yu
- Department of Endoscopy, Shanghai Jiangong Hospital, Shanghai, China
| |
Collapse
|
11
|
Zhang Q, Ren X, Wei B. Segmentation of infected region in CT images of COVID-19 patients based on QC-HC U-net. Sci Rep 2021; 11:22854. [PMID: 34819524 PMCID: PMC8613253 DOI: 10.1038/s41598-021-01502-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Accepted: 10/25/2021] [Indexed: 12/24/2022] Open
Abstract
Since the outbreak of COVID-19 in 2019, the rapid spread of the epidemic has brought huge challenges to medical institutions. If the pathological region in the COVID-19 CT image can be automatically segmented, it will help doctors quickly determine the patient's infection, thereby speeding up the diagnosis process. To be able to automatically segment the infected area, we proposed a new network structure and named QC-HC U-Net. First, we combine residual connection and dense connection to form a new connection method and apply it to the encoder and the decoder. Second, we choose to add Hypercolumns in the decoder section. Compared with the benchmark 3D U-Net, the improved network can effectively avoid vanishing gradient while extracting more features. To improve the situation of insufficient data, resampling and data enhancement methods are selected in this paper to expand the datasets. We used 63 cases of MSD lung tumor data for training and testing, continuously verified to ensure the training effect of this model, and then selected 20 cases of public COVID-19 data for training and testing. Experimental results showed that in the segmentation of COVID-19, the specificity and sensitivity were 85.3% and 83.6%, respectively, and in the segmentation of MSD lung tumors, the specificity and sensitivity were 81.45% and 80.93%, respectively, without any fitting.
Collapse
Affiliation(s)
- Qin Zhang
- School of Computer Science and Technology, Qilu University of Technology, Jinan, 250301, China
| | - Xiaoqiang Ren
- School of Computer Science and Technology, Qilu University of Technology, Jinan, 250301, China.
| | - Benzheng Wei
- Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine, Jinan, China.
| |
Collapse
|
12
|
Deliwala SS, Hamid K, Barbarawi M, Lakshman H, Zayed Y, Kandel P, Malladi S, Singh A, Bachuwa G, Gurvits GE, Chawla S. Artificial intelligence (AI) real-time detection vs. routine colonoscopy for colorectal neoplasia: a meta-analysis and trial sequential analysis. Int J Colorectal Dis 2021; 36:2291-2303. [PMID: 33934173 DOI: 10.1007/s00384-021-03929-3] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/07/2021] [Indexed: 02/04/2023]
Abstract
GOALS AND BACKGROUND Studies analyzing artificial intelligence (AI) in colonoscopies have reported improvements in detecting colorectal cancer (CRC) lesions, however its utility in the realworld remains limited. In this systematic review and meta-analysis, we evaluate the efficacy of AI-assisted colonoscopies against routine colonoscopy (RC). STUDY We performed an extensive search of major databases (through January 2021) for randomized controlled trials (RCTs) reporting adenoma and polyp detection rates. Odds ratio (OR) and standardized mean differences (SMD) with 95% confidence intervals (CIs) were reported. Additionally, trial sequential analysis (TSA) was performed to guard against errors. RESULTS Six RCTs were included (4996 participants). The mean age (SD) was 51.99 (4.43) years, and 49% were females. Detection rates favored AI over RC for adenomas (OR 1.77; 95% CI: 1.570-2.08) and polyps (OR 1.91; 95% CI: 1.68-2.16). Secondary outcomes including mean number of adenomas (SMD 0.23; 95% CI: 0.18-0.29) and polyps (SMD 0.23; 95% CI: 0.17-0.29) detected per procedure favored AI. However, RC outperformed AI in detecting pedunculated polyps. Withdrawal times (WTs) favored AI when biopsies were included, while WTs without biopsies, cecal intubation times, and bowel preparation adequacy were similar. CONCLUSIONS Colonoscopies equipped with AI detection algorithms could significantly detect previously missed adenomas and polyps while retaining the ability to self-assess and improve periodically. More effective clearance of diminutive adenomas may allow lengthening in surveillance intervals, reducing the burden of surveillance colonoscopies, and increasing its accessibility to those at higher risk. TSA ruled out the risk for false-positive results and confirmed a sufficient sample size to detect the observed effect. Currently, these findings suggest that AI-assisted colonoscopy can serve as a useful proxy to address critical gaps in CRC identification.
Collapse
Affiliation(s)
- Smit S Deliwala
- Department of Internal Medicine, Michigan State University at Hurley Medical Center, Two Hurley Plaza, Ste 212, Flint, MI, 48503, USA.
| | - Kewan Hamid
- Department of Internal Medicine/Pediatrics, Michigan State University at Hurley Medical Center, Flint, MI, USA
| | - Mahmoud Barbarawi
- Department of Internal Medicine, Michigan State University at Hurley Medical Center, Two Hurley Plaza, Ste 212, Flint, MI, 48503, USA
| | - Harini Lakshman
- Department of Internal Medicine, Michigan State University at Hurley Medical Center, Two Hurley Plaza, Ste 212, Flint, MI, 48503, USA
| | - Yazan Zayed
- Department of Internal Medicine, Michigan State University at Hurley Medical Center, Two Hurley Plaza, Ste 212, Flint, MI, 48503, USA
| | - Pujan Kandel
- Department of Internal Medicine, Michigan State University at Hurley Medical Center, Two Hurley Plaza, Ste 212, Flint, MI, 48503, USA
| | - Srikanth Malladi
- Department of Internal Medicine/Pediatrics, Michigan State University at Hurley Medical Center, Flint, MI, USA
| | - Adiraj Singh
- Department of Internal Medicine/Pediatrics, Michigan State University at Hurley Medical Center, Flint, MI, USA
| | - Ghassan Bachuwa
- Department of Internal Medicine, Michigan State University at Hurley Medical Center, Two Hurley Plaza, Ste 212, Flint, MI, 48503, USA
| | - Grigoriy E Gurvits
- Department of Internal Medicine - Division of Gastroenterology, New York University/Langone Medical Center, New York, NY, USA
| | - Saurabh Chawla
- Department of Internal Medicine - Division of Gastroenterology, Emory University, Atlanta, GA, USA
| |
Collapse
|
13
|
Zhou J, Hu N, Huang ZY, Song B, Wu CC, Zeng FX, Wu M. Application of artificial intelligence in gastrointestinal disease: a narrative review. ANNALS OF TRANSLATIONAL MEDICINE 2021; 9:1188. [PMID: 34430629 PMCID: PMC8350704 DOI: 10.21037/atm-21-3001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Accepted: 06/29/2021] [Indexed: 02/05/2023]
Abstract
Objective We collected evidence on the application of artificial intelligence (AI) in gastroenterology field. The review was carried out from two aspects of endoscopic types and gastrointestinal diseases, and briefly summarized the challenges and future directions in this field. Background Due to the advancement of computational power and a surge of available data, a solid foundation has been laid for the growth of AI. Specifically, varied machine learning (ML) techniques have been emerging in endoscopic image analysis. To improve the accuracy and efficiency of clinicians, AI has been widely applied to gastrointestinal endoscopy. Methods PubMed electronic database was searched using the keywords containing “AI”, “ML”, “deep learning (DL)”, “convolution neural network”, “endoscopy (such as white light endoscopy (WLE), narrow band imaging (NBI) endoscopy, magnifying endoscopy with narrow band imaging (ME-NBI), chromoendoscopy, endocytoscopy (EC), and capsule endoscopy (CE))”. Search results were assessed for relevance and then used for detailed discussion. Conclusions This review described the basic knowledge of AI, ML, and DL, and summarizes the application of AI in various endoscopes and gastrointestinal diseases. Finally, the challenges and directions of AI in clinical application were discussed. At present, the application of AI has solved some clinical problems, but more still needs to be done.
Collapse
Affiliation(s)
- Jun Zhou
- Huaxi MR Research Center (HMRRC), Department of Radiology, West China Hospital of Sichuan University, Chengdu, China.,Department of Clinical Research Center, Dazhou Central Hospital, Dazhou, China
| | - Na Hu
- Department of Radiology, West China Hospital of Sichuan University, Chengdu, China
| | - Zhi-Yin Huang
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu, China
| | - Bin Song
- Department of Radiology, West China Hospital of Sichuan University, Chengdu, China
| | - Chun-Cheng Wu
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu, China
| | - Fan-Xin Zeng
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou, China
| | - Min Wu
- Huaxi MR Research Center (HMRRC), Department of Radiology, West China Hospital of Sichuan University, Chengdu, China.,Department of Clinical Research Center, Dazhou Central Hospital, Dazhou, China
| |
Collapse
|
14
|
Automatic Polyp Segmentation in Colonoscopy Images Using a Modified Deep Convolutional Encoder-Decoder Architecture. SENSORS 2021; 21:s21165630. [PMID: 34451072 PMCID: PMC8402594 DOI: 10.3390/s21165630] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 08/07/2021] [Accepted: 08/19/2021] [Indexed: 11/25/2022]
Abstract
Colorectal cancer has become the third most commonly diagnosed form of cancer, and has the second highest fatality rate of cancers worldwide. Currently, optical colonoscopy is the preferred tool of choice for the diagnosis of polyps and to avert colorectal cancer. Colon screening is time-consuming and highly operator dependent. In view of this, a computer-aided diagnosis (CAD) method needs to be developed for the automatic segmentation of polyps in colonoscopy images. This paper proposes a modified SegNet Visual Geometry Group-19 (VGG-19), a form of convolutional neural network, as a CAD method for polyp segmentation. The modifications include skip connections, 5 × 5 convolutional filters, and the concatenation of four dilated convolutions applied in parallel form. The CVC-ClinicDB, CVC-ColonDB, and ETIS-LaribPolypDB databases were used to evaluate the model, and it was found that our proposed polyp segmentation model achieved an accuracy, sensitivity, specificity, precision, mean intersection over union, and dice coefficient of 96.06%, 94.55%, 97.56%, 97.48%, 92.3%, and 95.99%, respectively. These results indicate that our model performs as well as or better than previous schemes in the literature. We believe that this study will offer benefits in terms of the future development of CAD tools for polyp segmentation for colorectal cancer diagnosis and management. In the future, we intend to embed our proposed network into a medical capsule robot for practical usage and try it in a hospital setting with clinicians.
Collapse
|
15
|
Jiang J, Xie Q, Cheng Z, Cai J, Xia T, Yang H, Yang B, Peng H, Bai X, Yan M, Li X, Zhou J, Huang X, Wang L, Long H, Wang P, Chu Y, Zeng FW, Zhang X, Wang G, Zeng F. AI based colorectal disease detection using real-time screening colonoscopy. PRECISION CLINICAL MEDICINE 2021; 4:109-118. [PMID: 35694157 PMCID: PMC8982552 DOI: 10.1093/pcmedi/pbab013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 04/23/2021] [Accepted: 05/17/2021] [Indexed: 12/24/2022] Open
Abstract
Abstract
Colonoscopy is an effective tool for early screening of colorectal diseases. However, the application of colonoscopy in distinguishing different intestinal diseases still faces great challenges of efficiency and accuracy. Here we constructed and evaluated a deep convolution neural network (CNN) model based on 117 055 images from 16 004 individuals, which achieved a high accuracy of 0.933 in the validation dataset in identifying patients with polyp, colitis, colorectal cancer (CRC) from normal. The proposed approach was further validated on multi-center real-time colonoscopy videos and images, which achieved accurate diagnostic performance on detecting colorectal diseases with high accuracy and precision to generalize across external validation datasets. The diagnostic performance of the model was further compared to the skilled endoscopists and the novices. In addition, our model has potential in diagnosis of adenomatous polyp and hyperplastic polyp with an area under the receiver operating characteristic curve of 0.975. Our proposed CNN models have potential in assisting clinicians in making clinical decisions with efficiency during application.
Collapse
Affiliation(s)
- Jiawei Jiang
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
- Department of Computer Science, Eidgenossische Technische Hochschule Zurich, Zurich 999034, Switzerland
| | - Qianrong Xie
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Zhuo Cheng
- Digestive endoscopy center, Dazhou Central Hospital, Dazhou 635000, China
| | - Jianqiang Cai
- Department of Hepatobiliary Surgery, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China
| | - Tian Xia
- National Center of Biomedical Analysis, Beijing 100850, China
| | - Hang Yang
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Bo Yang
- Digestive endoscopy center, Dazhou Central Hospital, Dazhou 635000, China
| | - Hui Peng
- College of Informatics, Huazhong Agricultural University, Wuhan 430070, China
| | - Xuesong Bai
- Digestive endoscopy center, Dazhou Central Hospital, Dazhou 635000, China
| | - Mingque Yan
- Digestive endoscopy center, Dazhou Central Hospital, Dazhou 635000, China
| | - Xue Li
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Jun Zhou
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Xuan Huang
- Department of Ophthalmology, Medical Research Center, Beijing Chao-Yang Hospital, Capital Medical University, Beijing 100020, China
| | - Liang Wang
- Information Department, Dazhou Central Hospital, Dazhou 635000, China
| | - Haiyan Long
- Digestive endoscopy center, Quxian People's Hospital, Dazhou 635000, China
| | - Pingxi Wang
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Yanpeng Chu
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Fan-Wei Zeng
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
| | - Xiuqin Zhang
- Institute of Molecular Medicine, Peking University, Beijing 100871, China
| | - Guangyu Wang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
| | - Fanxin Zeng
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou 635000, China
- Department of Medicine, Sichuan University of Arts and Science, Dazhou 635000, China
| |
Collapse
|
16
|
Ali S, Dmitrieva M, Ghatwary N, Bano S, Polat G, Temizel A, Krenzer A, Hekalo A, Guo YB, Matuszewski B, Gridach M, Voiculescu I, Yoganand V, Chavan A, Raj A, Nguyen NT, Tran DQ, Huynh LD, Boutry N, Rezvy S, Chen H, Choi YH, Subramanian A, Balasubramanian V, Gao XW, Hu H, Liao Y, Stoyanov D, Daul C, Realdon S, Cannizzaro R, Lamarque D, Tran-Nguyen T, Bailey A, Braden B, East JE, Rittscher J. Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy. Med Image Anal 2021; 70:102002. [PMID: 33657508 DOI: 10.1016/j.media.2021.102002] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 02/04/2021] [Accepted: 02/11/2021] [Indexed: 12/12/2022]
Abstract
The Endoscopy Computer Vision Challenge (EndoCV) is a crowd-sourcing initiative to address eminent problems in developing reliable computer aided detection and diagnosis endoscopy systems and suggest a pathway for clinical translation of technologies. Whilst endoscopy is a widely used diagnostic and treatment tool for hollow-organs, there are several core challenges often faced by endoscopists, mainly: 1) presence of multi-class artefacts that hinder their visual interpretation, and 2) difficulty in identifying subtle precancerous precursors and cancer abnormalities. Artefacts often affect the robustness of deep learning methods applied to the gastrointestinal tract organs as they can be confused with tissue of interest. EndoCV2020 challenges are designed to address research questions in these remits. In this paper, we present a summary of methods developed by the top 17 teams and provide an objective comparison of state-of-the-art methods and methods designed by the participants for two sub-challenges: i) artefact detection and segmentation (EAD2020), and ii) disease detection and segmentation (EDD2020). Multi-center, multi-organ, multi-class, and multi-modal clinical endoscopy datasets were compiled for both EAD2020 and EDD2020 sub-challenges. The out-of-sample generalization ability of detection algorithms was also evaluated. Whilst most teams focused on accuracy improvements, only a few methods hold credibility for clinical usability. The best performing teams provided solutions to tackle class imbalance, and variabilities in size, origin, modality and occurrences by exploring data augmentation, data fusion, and optimal class thresholding techniques.
Collapse
Affiliation(s)
- Sharib Ali
- Institute of Biomedical Engineering and Big Data Institute, Old Road Campus, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK.
| | - Mariia Dmitrieva
- Institute of Biomedical Engineering and Big Data Institute, Old Road Campus, University of Oxford, Oxford, UK
| | - Noha Ghatwary
- Computer Engineering Department, Arab Academy for Science and Technology, Alexandria, Egypt
| | - Sophia Bano
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences(WEISS) and Department of Computer Science, University College London, London, UK
| | - Gorkem Polat
- Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
| | - Alptekin Temizel
- Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
| | - Adrian Krenzer
- Department of Artificial Intelligence and Knowledge Systems, University of Würzburg, Germany
| | - Amar Hekalo
- Department of Artificial Intelligence and Knowledge Systems, University of Würzburg, Germany
| | - Yun Bo Guo
- School of Engineering, University of Central Lancashire, UK
| | | | - Mourad Gridach
- Ibn Zohr University, Computer Science HIT, Agadir, Morocco
| | | | - Vishnusai Yoganand
- Mimyk Medical Simulations Pvt Ltd, Indian Institute of Science, Bengaluru, India
| | - Arnav Chavan
- Indian Institute of Technology (ISM), Dhanbad, India
| | - Aryan Raj
- Indian Institute of Technology (ISM), Dhanbad, India
| | - Nhan T Nguyen
- Medical Imaging Department, Vingroup Big Data Institute (VinBDI), Hanoi, Vietnam
| | - Dat Q Tran
- Medical Imaging Department, Vingroup Big Data Institute (VinBDI), Hanoi, Vietnam
| | - Le Duy Huynh
- EPITA Research and Development Laboratory (LRDE), F-94270 Le Kremlin-Bicêtre, France
| | - Nicolas Boutry
- EPITA Research and Development Laboratory (LRDE), F-94270 Le Kremlin-Bicêtre, France
| | - Shahadate Rezvy
- School of Science and Technology, Middlesex University London, UK
| | - Haijian Chen
- Department of Computer Science, School of Informatics, Xiamen University, China
| | - Yoon Ho Choi
- Dept. of Health Sciences & Tech., Samsung Advanced Institute for Health Sciences & Tech. (SAIHST), Sungkyunkwan University, Seoul, Republic of Korea
| | | | | | - Xiaohong W Gao
- School of Science and Technology, Middlesex University London, UK
| | - Hongyu Hu
- Shanghai Jiaotong University, Shanghai, China
| | | | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences(WEISS) and Department of Computer Science, University College London, London, UK
| | - Christian Daul
- CRAN UMR 7039, University of Lorraine, CNRS, Nancy, France
| | | | | | - Dominique Lamarque
- Université de Versailles St-Quentin en Yvelines, Hôpital Ambroise Paré, France
| | - Terry Tran-Nguyen
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Adam Bailey
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK
| | - Barbara Braden
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK
| | - James E East
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK
| | - Jens Rittscher
- Institute of Biomedical Engineering and Big Data Institute, Old Road Campus, University of Oxford, Oxford, UK
| |
Collapse
|
17
|
Cao C, Wang R, Yu Y, zhang H, Yu Y, Sun C. Gastric polyp detection in gastroscopic images using deep neural network. PLoS One 2021; 16:e0250632. [PMID: 33909671 PMCID: PMC8081222 DOI: 10.1371/journal.pone.0250632] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Accepted: 04/08/2021] [Indexed: 12/26/2022] Open
Abstract
This paper presents the research results of detecting gastric polyps with deep learning object detection method in gastroscopic images. Gastric polyps have various sizes. The difficulty of polyp detection is that small polyps are difficult to detect from the background. We propose a feature extraction and fusion module and combine it with the YOLOv3 network to form our network. This method performs better than other methods in the detection of small polyps because it can fuse the semantic information of high-level feature maps with low-level feature maps to help small polyps detection. In this work, we use a dataset of gastric polyps created by ourselves, containing 1433 training images and 508 validation images. We train and validate our network on our dataset. In comparison with other methods of polyps detection, our method has a significant improvement in precision, recall rate, F1, and F2 score. The precision, recall rate, F1 score, and F2 score of our method can achieve 91.6%, 86.2%, 88.8%, and 87.2%.
Collapse
Affiliation(s)
- Chanting Cao
- Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, China
| | - Ruilin Wang
- Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, China
| | - Yao Yu
- Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, China
- * E-mail:
| | - Hui zhang
- Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Ying Yu
- Beijing An Zhen Hospital, Beijing, China
| | - Changyin Sun
- School of Automation, Southeast University, Nanjing, China
| |
Collapse
|
18
|
Sánchez-Peralta LF, Picón A, Sánchez-Margallo FM, Pagador JB. Unravelling the effect of data augmentation transformations in polyp segmentation. Int J Comput Assist Radiol Surg 2020; 15:1975-1988. [PMID: 32989680 PMCID: PMC7671995 DOI: 10.1007/s11548-020-02262-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 09/14/2020] [Indexed: 12/27/2022]
Abstract
PURPOSE Data augmentation is a common technique to overcome the lack of large annotated databases, a usual situation when applying deep learning to medical imaging problems. Nevertheless, there is no consensus on which transformations to apply for a particular field. This work aims at identifying the effect of different transformations on polyp segmentation using deep learning. METHODS A set of transformations and ranges have been selected, considering image-based (width and height shift, rotation, shear, zooming, horizontal and vertical flip and elastic deformation), pixel-based (changes in brightness and contrast) and application-based (specular lights and blurry frames) transformations. A model has been trained under the same conditions without data augmentation transformations (baseline) and for each of the transformation and ranges, using CVC-EndoSceneStill and Kvasir-SEG, independently. Statistical analysis is performed to compare the baseline performance against results of each range of each transformation on the same test set for each dataset. RESULTS This basic method identifies the most adequate transformations for each dataset. For CVC-EndoSceneStill, changes in brightness and contrast significantly improve the model performance. On the contrary, Kvasir-SEG benefits to a greater extent from the image-based transformations, especially rotation and shear. Augmentation with synthetic specular lights also improves the performance. CONCLUSION Despite being infrequently used, pixel-based transformations show a great potential to improve polyp segmentation in CVC-EndoSceneStill. On the other hand, image-based transformations are more suitable for Kvasir-SEG. Problem-based transformations behave similarly in both datasets. Polyp area, brightness and contrast of the dataset have an influence on these differences.
Collapse
Affiliation(s)
| | - Artzai Picón
- Tecnalia Research and Innovation, Zamudio, Spain
| | | | - J Blas Pagador
- Jesús Usón Minimally Invasive Surgery Centre, Road N-521, km 41.8, 10071, Cáceres, Spain
| |
Collapse
|
19
|
Mohan BP, Khan SR, Kassab LL, Ponnada S, Dulai PS, Kochhar GS. Accuracy of convolutional neural network-based artificial intelligence in diagnosis of gastrointestinal lesions based on endoscopic images: A systematic review and meta-analysis. Endosc Int Open 2020; 8:E1584-E1594. [PMID: 33140014 PMCID: PMC7581460 DOI: 10.1055/a-1236-3007] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Accepted: 07/08/2020] [Indexed: 12/29/2022] Open
Abstract
Background and study aims Recently, a growing body of evidence has been amassed on evaluation of artificial intelligence (AI) known as deep learning in computer-aided diagnosis of gastrointestinal lesions by means of convolutional neural networks (CNN). We conducted this meta-analysis to study pooled rates of performance for CNN-based AI in diagnosis of gastrointestinal neoplasia from endoscopic images. Methods Multiple databases were searched (from inception to November 2019) and studies that reported on the performance of AI by means of CNN in the diagnosis of gastrointestinal tumors were selected. A random effects model was used and pooled accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated. Pooled rates were categorized based on the gastrointestinal location of lesion (esophagus, stomach and colorectum). Results Nineteen studies were included in our final analysis. The pooled accuracy of CNN in esophageal neoplasia was 87.2 % (76-93.6) and NPV was 92.1 % (85.9-95.7); the accuracy in lesions of stomach was 85.8 % (79.8-90.3) and NPV was 92.1 % (85.9-95.7); and in colorectal neoplasia the accuracy was 89.9 % (82-94.7) and NPV was 94.3 % (86.4-97.7). Conclusions Based on our meta-analysis, CNN-based AI achieved high accuracy in diagnosis of lesions in esophagus, stomach, and colorectum.
Collapse
Affiliation(s)
- Babu P Mohan
- Gastroenterology & Hepatology, University of Utah Health, Salt Lake City, Utah, United States
| | - Shahab R Khan
- Gastroenterology, Rush University Medical Center, Chicago, Illinois, United States
| | - Lena L Kassab
- Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
| | - Suresh Ponnada
- Internal Medicine, Roanoke Medical Center, Roanoke, Virginia, United States
| | - Parambir S Dulai
- Gastroenterology and Hepatology, University of California, San Diego, California, United States
| | - Gursimran S Kochhar
- Division of Gastroenterology and Hepatology, Allegheny Health Network, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
20
|
Pacal I, Karaboga D, Basturk A, Akay B, Nalbantoglu U. A comprehensive review of deep learning in colon cancer. Comput Biol Med 2020; 126:104003. [DOI: 10.1016/j.compbiomed.2020.104003] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2020] [Revised: 08/28/2020] [Accepted: 08/28/2020] [Indexed: 12/17/2022]
|
21
|
Zhang L, Zhang J, Li Z, Song Y. A multiple-channel and atrous convolution network for ultrasound image segmentation. Med Phys 2020; 47:6270-6285. [PMID: 33007105 DOI: 10.1002/mp.14512] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 09/12/2020] [Accepted: 09/22/2020] [Indexed: 11/08/2022] Open
Abstract
PURPOSE Ultrasound image segmentation is a challenging task due to a low signal-to-noise ratio and poor image quality. Although several approaches based on the convolutional neural network (CNN) have been applied to ultrasound image segmentation, they have weak generalization ability. We propose an end-to-end, multiple-channel and atrous CNN designed to extract a greater amount of semantic information for segmentation of ultrasound images. METHOD A multiple-channel and atrous convolution network is developed, referred to as MA-Net. Similar to U-Net, MA-Net is based on an encoder-decoder architecture and includes five modules: the encoder, atrous convolution, pyramid pooling, decoder, and residual skip pathway modules. In the encoder module, we aim to capture more information with multiple-channel convolution and use large kernel convolution instead of small filters in each convolution operation. In the last layer, atrous convolution and pyramid pooling are used to extract multi-scale features. The architecture of the decoder is similar to that of the encoder module, except that up-sampling is used instead of down-sampling. Furthermore, the residual skip pathway module connects the subnetworks of the encoder and decoder to optimize learning from the deeper layer and improve the accuracy of segmentation. During the learning process, we adopt multi-task learning to enhance segmentation performance. Five types of datasets are used in our experiments. Because the original training data are limited, we apply data augmentation (e.g., horizontal and vertical flipping, random rotations, and random scaling) to our training data. We use the Dice score, precision, recall, Hausdorff distance (HD), average symmetric surface distance (ASD), and root mean square symmetric surface distance (RMSD) as the metrics for segmentation evaluation. Meanwhile, Friedman test was performed as the nonparametric statistical analysis to evaluate the algorithms. RESULTS For the datasets of brachia plexus (BP), fetal head, and lymph node segmentations, MA-Net achieved average Dice scores of 0.776, 0.973, and 0.858, respectively; with average precisions of 0.787, 0.968, and 0.854, respectively; average recalls of 0.788, 0.978, and 0.885, respectively; average HDs (mm) of 13.591, 10.924, and 19.245, respectively; average ASDs (mm) of 4.822, 4.152, and 4.312, respectively; and average RMSDs (mm) of 4.979, 4.161, and 4.930, respectively. Compared with U-Net, U-Net++, M-Net, and Dilated U-Net, the average performance of the MA-Net increased by approximately 5.68%, 2.85%, 6.59%, 36.03%, 23.64%, and 31.71% for Dice, precision, recall, HD, ASD, and RMSD, respectively. Moreover, we verified the generalization of MA-Net segmentation to lower grade brain glioma MRI and lung CT images. In addition, the MA-Net achieved the highest mean rank in the Friedman test. CONCLUSION The proposed MA-Net accurately segments ultrasound images with high generalization, and therefore, it offers a useful tool for diagnostic application in ultrasound images.
Collapse
Affiliation(s)
- Lun Zhang
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China.,Yunnan Vocational Institute of Energy Technology, Qujing, Yunnan, 655001, China
| | - Junhua Zhang
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China
| | - Zonggui Li
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China
| | - Yingchao Song
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, 650091, China
| |
Collapse
|