1
|
Wang Y, Ni H, Zhou J, Liu L, Lin J, Yin M, Gao J, Zhu S, Yin Q, Zhu J, Li R. A Semi-Supervised Learning Framework for Classifying Colorectal Neoplasia Based on the NICE Classification. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01123-9. [PMID: 38653910 DOI: 10.1007/s10278-024-01123-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Revised: 04/02/2024] [Accepted: 04/12/2024] [Indexed: 04/25/2024]
Abstract
Labelling medical images is an arduous and costly task that necessitates clinical expertise and large numbers of qualified images. Insufficient samples can lead to underfitting during training and poor performance of supervised learning models. In this study, we aim to develop a SimCLR-based semi-supervised learning framework to classify colorectal neoplasia based on the NICE classification. First, the proposed framework was trained under self-supervised learning using a large unlabelled dataset; subsequently, it was fine-tuned on a limited labelled dataset based on the NICE classification. The model was evaluated on an independent dataset and compared with models based on supervised transfer learning and endoscopists using accuracy, Matthew's correlation coefficient (MCC), and Cohen's kappa. Finally, Grad-CAM and t-SNE were applied to visualize the models' interpretations. A ResNet-backboned SimCLR model (accuracy of 0.908, MCC of 0.862, and Cohen's kappa of 0.896) outperformed supervised transfer learning-based models (means: 0.803, 0.698, and 0.742) and junior endoscopists (0.816, 0.724, and 0.863), while performing only slightly worse than senior endoscopists (0.916, 0.875, and 0.944). Moreover, t-SNE showed a better clustering of ternary samples through self-supervised learning in SimCLR than through supervised transfer learning. Compared with traditional supervised learning, semi-supervised learning enables deep learning models to achieve improved performance with limited labelled endoscopic images.
Collapse
Affiliation(s)
- Yu Wang
- Department of Hepatobiliary Surgery, Jintan Affiliated Hospital of Jiangsu University, Changzhou, Jiangsu, 213200, China
| | - Haoxiang Ni
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China
| | - Jielu Zhou
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China
- Department of Geriatrics, Kowloon Affiliated Hospital of Shanghai Jiao Tong University, Suzhou, Jiangsu, 215006, China
| | - Lihe Liu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China
| | - Jiaxi Lin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China
| | - Minyue Yin
- Department of Gastroenterology, Beijing Friendship Hospital, Capital Medical University, Beijing, 100050, China
- National Clinical Research Center for Digestive Disease, Beijing Digestive Disease Center, State Key Laboratory of Digestive Health, Beijing, 100050, China
| | - Jingwen Gao
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China
| | - Shiqi Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China
| | - Qi Yin
- Department of Anesthesiology, Jintan Affiliated Hospital of Jiangsu University, Changzhou, Jiangsu, 213200, China
| | - Jinzhou Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China.
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China.
- Key Laboratory of Hepatosplenic Surgery, Ministry of Education, The First Affiliated Hospital of Harbin Medical University, Harbin, 150001, China.
| | - Rui Li
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, # 899 Pinghai St., Suzhou, Jiangsu, 215006, China.
- Suzhou Clinical Center of Digestive Disease, Suzhou, Jiangsu, 215006, China.
| |
Collapse
|
2
|
Gangrade S, Sharma PC, Sharma AK, Singh YP. Modified DeeplabV3+ with multi-level context attention mechanism for colonoscopy polyp segmentation. Comput Biol Med 2024; 170:108096. [PMID: 38320340 DOI: 10.1016/j.compbiomed.2024.108096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/08/2024]
Abstract
The development of automated methods for analyzing medical images of colon cancer is one of the main research fields. A colonoscopy is a medical treatment that enables a doctor to look for any abnormalities like polyps, cancer, or inflammatory tissue inside the colon and rectum. It falls under the category of gastrointestinal illnesses, and it claims the lives of almost two million people worldwide. Video endoscopy is an advanced medical imaging approach to diagnose gastrointestinal disorders such as inflammatory bowel, ulcerative colitis, esophagitis, and polyps. Medical video endoscopy generates several images, which must be reviewed by specialists. The difficulty of manual diagnosis has sparked research towards computer-aided techniques that can quickly and reliably diagnose all generated images. The proposed methodology establishes a framework for diagnosing coloscopy diseases. Endoscopists can lower the risk of polyps turning into cancer during colonoscopies by using more accurate computer-assisted polyp detection and segmentation. With the aim of creating a model that can automatically distinguish polyps from images, we presented a modified DeeplabV3+ model in this study to carry out segmentation tasks successfully and efficiently. The framework's encoder uses a pre-trained dilated convolutional residual network for optimal feature map resolution. The robustness of the modified model is tested against state-of-the-art segmentation approaches. In this work, we employed two publicly available datasets, CVC-Clinic DB and Kvasir-SEG, and obtained Dice similarity coefficients of 0.97 and 0.95, respectively. The results show that the improved DeeplabV3+ model improves segmentation efficiency and effectiveness in both software and hardware with only minor changes.
Collapse
Affiliation(s)
- Shweta Gangrade
- School of Information Technology, Manipal University Jaipur, Jaipur, Rajasthan, India; School of Computer Science and Engineering, Manipal University Jaipur, Jaipur, Rajasthan, India
| | - Prakash Chandra Sharma
- School of Information Technology, Manipal University Jaipur, Jaipur, Rajasthan, India; School of Computer Science and Engineering, Manipal University Jaipur, Jaipur, Rajasthan, India
| | - Akhilesh Kumar Sharma
- School of Information Technology, Manipal University Jaipur, Jaipur, Rajasthan, India; School of Computer Science and Engineering, Manipal University Jaipur, Jaipur, Rajasthan, India
| | - Yadvendra Pratap Singh
- School of Information Technology, Manipal University Jaipur, Jaipur, Rajasthan, India; School of Computer Science and Engineering, Manipal University Jaipur, Jaipur, Rajasthan, India.
| |
Collapse
|
3
|
Troya J, Sudarevic B, Krenzer A, Banck M, Brand M, Walter BM, Puppe F, Zoller WG, Meining A, Hann A. Direct comparison of multiple computer-aided polyp detection systems. Endoscopy 2024; 56:63-69. [PMID: 37532115 PMCID: PMC10736101 DOI: 10.1055/a-2147-0571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 08/01/2023] [Indexed: 08/04/2023]
Abstract
BACKGROUND AND STUDY AIMS Artificial intelligence (AI)-based systems for computer-aided detection (CADe) of polyps receive regular updates and occasionally offer customizable detection thresholds, both of which impact their performance, but little is known about these effects. This study aimed to compare the performance of different CADe systems on the same benchmark dataset. METHODS 101 colonoscopy videos were used as benchmark. Each video frame with a visible polyp was manually annotated with bounding boxes, resulting in 129 705 polyp images. The videos were then analyzed by three different CADe systems, representing five conditions: two versions of GI Genius, Endo-AID with detection Types A and B, and EndoMind, a freely available system. Evaluation included an analysis of sensitivity and false-positive rate, among other metrics. RESULTS Endo-AID detection Type A, the earlier version of GI Genius, and EndoMind detected all 93 polyps. Both the later version of GI Genius and Endo-AID Type B missed 1 polyp. The mean per-frame sensitivities were 50.63 % and 67.85 %, respectively, for the earlier and later versions of GI Genius, 65.60 % and 52.95 %, respectively, for Endo-AID Types A and B, and 60.22 % for EndoMind. CONCLUSIONS This study compares the performance of different CADe systems, different updates, and different configuration modes. This might help clinicians to select the most appropriate system for their specific needs.
Collapse
Affiliation(s)
- Joel Troya
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
- Bavarian Cancer Research Center, Würzburg, Germany
| | - Boban Sudarevic
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
- Department of Internal Medicine and Gastroenterology, Katharinenhospital, Stuttgart, Germany
| | - Adrian Krenzer
- Artificial Intelligence and Knowledge Systems, Institute for Computer Science, Julius-Maximilians-Universität, Würzburg, Germany
| | - Michael Banck
- Artificial Intelligence and Knowledge Systems, Institute for Computer Science, Julius-Maximilians-Universität, Würzburg, Germany
| | - Markus Brand
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
| | - Benjamin M. Walter
- Department of Internal Medicine I, University Hospital Ulm, Ulm, Germany
| | - Frank Puppe
- Artificial Intelligence and Knowledge Systems, Institute for Computer Science, Julius-Maximilians-Universität, Würzburg, Germany
| | - Wolfram G. Zoller
- Department of Internal Medicine and Gastroenterology, Katharinenhospital, Stuttgart, Germany
| | - Alexander Meining
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
- Bavarian Cancer Research Center, Würzburg, Germany
| | - Alexander Hann
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
| |
Collapse
|
4
|
van Bokhorst QNE, Houwen BBSL, Hazewinkel Y, Fockens P, Dekker E. Advances in artificial intelligence and computer science for computer-aided diagnosis of colorectal polyps: current status. Endosc Int Open 2023; 11:E752-E767. [PMID: 37593158 PMCID: PMC10431975 DOI: 10.1055/a-2098-1999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Accepted: 05/08/2023] [Indexed: 08/19/2023] Open
Affiliation(s)
- Querijn N E van Bokhorst
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Britt B S L Houwen
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Yark Hazewinkel
- Department of Gastroenterology and Hepatology, Tergooi Medical Center, Hilversum, the Netherlands
| | - Paul Fockens
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Evelien Dekker
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| |
Collapse
|
5
|
Liao WC, Mukundan A, Sadiaza C, Tsao YM, Huang CW, Wang HC. Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2023; 14:4383-4405. [PMID: 37799695 PMCID: PMC10549751 DOI: 10.1364/boe.492635] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 10/07/2023]
Abstract
One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.
Collapse
Affiliation(s)
- Wei-Chih Liao
- Department of Internal Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan
- Graduate Institute of Epidemiology and Preventive Medicine, National Taiwan University, Taipei, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Cleorita Sadiaza
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila, 1015, Philippines
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Chien-Wei Huang
- Department of Gastroenterology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st.Rd., Lingya District, Kaohsiung City 80284, Taiwan
- Department of Nursing, Tajen University, 20, Weixin Rd., Yanpu Township, Pingtung County 90741, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Department of Medical Research, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 2, Minsheng Road, Dalin, Chiayi, 62247, Taiwan
- Director of Technology Development, Hitspectra Intelligent Technology Co., Ltd., 4F., No. 2, Fuxing 4th Rd., Qianzhen Dist., Kaohsiung City 80661, Taiwan
| |
Collapse
|
6
|
Galati JS, Lin K, Gross SA. Recent advances in devices and technologies that might prove revolutionary for colonoscopy procedures. Expert Rev Med Devices 2023; 20:1087-1103. [PMID: 37934873 DOI: 10.1080/17434440.2023.2280773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 11/03/2023] [Indexed: 11/09/2023]
Abstract
INTRODUCTION Colorectal cancer (CRC) is the third most common malignancy and second leading cause of cancer-related mortality in the world. Adenoma detection rate (ADR), a quality indicator for colonoscopy, has gained prominence as it is inversely related to CRC incidence and mortality. As such, recent efforts have focused on developing novel colonoscopy devices and technologies to improve ADR. AREAS COVERED The main objective of this paper is to provide an overview of advancements in the fields of colonoscopy mechanical attachments, artificial intelligence-assisted colonoscopy, and colonoscopy optical enhancements with respect to ADR. We accomplished this by performing a comprehensive search of multiple electronic databases from inception to September 2023. This review is intended to be an introduction to colonoscopy devices and technologies. EXPERT OPINION Numerous mechanical attachments and optical enhancements have been developed that have the potential to improve ADR and AI has gone from being an inaccessible concept to a feasible means for improving ADR. While these advances are exciting and portend a change in what will be considered standard colonoscopy, they continue to require refinement. Future studies should focus on combining modalities to further improve ADR and exploring the use of these technologies in other facets of colonoscopy.
Collapse
Affiliation(s)
- Jonathan S Galati
- Department of Internal Medicine, NYU Langone Health, New York, NY, USA
| | - Kevin Lin
- Department of Internal Medicine, NYU Langone Health, New York, NY, USA
| | - Seth A Gross
- Division of Gastroenterology, NYU Langone Health, New York, NY, USA
| |
Collapse
|
7
|
Galati JS, Duve RJ, O'Mara M, Gross SA. Artificial intelligence in gastroenterology: A narrative review. Artif Intell Gastroenterol 2022; 3:117-141. [DOI: 10.35712/aig.v3.i5.117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Revised: 11/21/2022] [Accepted: 12/21/2022] [Indexed: 12/28/2022] Open
Abstract
Artificial intelligence (AI) is a complex concept, broadly defined in medicine as the development of computer systems to perform tasks that require human intelligence. It has the capacity to revolutionize medicine by increasing efficiency, expediting data and image analysis and identifying patterns, trends and associations in large datasets. Within gastroenterology, recent research efforts have focused on using AI in esophagogastroduodenoscopy, wireless capsule endoscopy (WCE) and colonoscopy to assist in diagnosis, disease monitoring, lesion detection and therapeutic intervention. The main objective of this narrative review is to provide a comprehensive overview of the research being performed within gastroenterology on AI in esophagogastroduodenoscopy, WCE and colonoscopy.
Collapse
Affiliation(s)
- Jonathan S Galati
- Department of Medicine, NYU Langone Health, New York, NY 10016, United States
| | - Robert J Duve
- Department of Internal Medicine, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, NY 14203, United States
| | - Matthew O'Mara
- Division of Gastroenterology, NYU Langone Health, New York, NY 10016, United States
| | - Seth A Gross
- Division of Gastroenterology, NYU Langone Health, New York, NY 10016, United States
| |
Collapse
|
8
|
Meining A, Hann A, Fuchs KH. Innovations in GI-endoscopy. Arab J Gastroenterol 2022; 23:139-143. [PMID: 35738990 DOI: 10.1016/j.ajg.2022.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Gastrointestinal endoscopy covers both diagnosis and therapy. Due to its diagnostic accuracy and minimal invasiveness, several innovations have been made within the last years including artificial intelligence and endoscopic tumor resection. The present review highlights some of these innovation. In addition, a special focus is set on the experience made by our own research group trying to combine the expertise of endoscopists/ physicians as well as engineers and computer scientists.
Collapse
Affiliation(s)
- Alexander Meining
- Interventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital Würzburg, Würzburg, Germany.
| | - Alexander Hann
- Interventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
| | - Karl Hermann Fuchs
- Interventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital Würzburg, Würzburg, Germany
| |
Collapse
|
9
|
Siau K, Berzin TM. Fine-tuning the artificial intelligence experience in endoscopy. United European Gastroenterol J 2022; 10:449-450. [PMID: 35596244 PMCID: PMC9189460 DOI: 10.1002/ueg2.12253] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Affiliation(s)
- Keith Siau
- Department of Gastroenterology, Royal Cornwall Hospitals NHS Trust, Truro, UK.,Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Tyler M Berzin
- Center for Advanced Endoscopy, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|