51
|
Taghiakbari M, Mori Y, von Renteln D. Artificial intelligence-assisted colonoscopy: A review of current state of practice and research. World J Gastroenterol 2021; 27:8103-8122. [PMID: 35068857 PMCID: PMC8704267 DOI: 10.3748/wjg.v27.i47.8103] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Revised: 08/22/2021] [Accepted: 12/08/2021] [Indexed: 02/06/2023] Open
Abstract
Colonoscopy is an effective screening procedure in colorectal cancer prevention programs; however, colonoscopy practice can vary in terms of lesion detection, classification, and removal. Artificial intelligence (AI)-assisted decision support systems for endoscopy is an area of rapid research and development. The systems promise improved detection, classification, screening, and surveillance for colorectal polyps and cancer. Several recently developed applications for AI-assisted colonoscopy have shown promising results for the detection and classification of colorectal polyps and adenomas. However, their value for real-time application in clinical practice has yet to be determined owing to limitations in the design, validation, and testing of AI models under real-life clinical conditions. Despite these current limitations, ambitious attempts to expand the technology further by developing more complex systems capable of assisting and supporting the endoscopist throughout the entire colonoscopy examination, including polypectomy procedures, are at the concept stage. However, further work is required to address the barriers and challenges of AI integration into broader colonoscopy practice, to navigate the approval process from regulatory organizations and societies, and to support physicians and patients on their journey to accepting the technology by providing strong evidence of its accuracy and safety. This article takes a closer look at the current state of AI integration into the field of colonoscopy and offers suggestions for future research.
Collapse
Affiliation(s)
- Mahsa Taghiakbari
- Department of Gastroenterology, CRCHUM, Montreal H2X 0A9, Quebec, Canada
| | - Yuichi Mori
- Clinical Effectiveness Research Group, University of Oslo, Oslo 0450, Norway
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama 224-8503, Japan
| | - Daniel von Renteln
- Department of Gastroenterology, CRCHUM, Montreal H2X 0A9, Quebec, Canada
| |
Collapse
|
52
|
Wang W, Mohseni P, Kilgore KL, Najafizadeh L. Cuff-less Blood Pressure Estimation from Photoplethysmography via Visibility Graph and Transfer Learning. IEEE J Biomed Health Inform 2021; 26:2075-2085. [PMID: 34784289 DOI: 10.1109/jbhi.2021.3128383] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This paper presents a new solution that enables the use of transfer learning for cuff-less blood pressure (BP) monitoring via short duration of photoplethysmogram (PPG). The proposed method estimates BP with low computational budget by 1) creating images from segments of PPG via visibility graph (VG) that preserves the temporal information of the PPG waveform, 2) using pre-trained deep convolutional neural network (CNN) to extract feature vectors from VG images, and 3) solving for the weights and bias between the feature vectors and the reference BPs with ridge regression. Using the University of California Irvine (UCI) database consisting of 348 records, the proposed method achieves a best error performance of 0.008.46 mmHg for systolic blood pressure (SBP), and -0.045.36 mmHg for diastolic blood pressure (DBP), respectively, in terms of the mean error (ME) and the standard deviation (SD) of error, ranking grade B for SBP and grade A for DBP under the British Hypertension Society (BHS) protocol. Our novel data-driven method offers a computationally-efficient end-to-end solution for rapid and user-friendly cuff-less PPG-based BP estimation.
Collapse
|
53
|
Goyal H, Sherazi SAA, Mann R, Gandhi Z, Perisetti A, Aziz M, Chandan S, Kopel J, Tharian B, Sharma N, Thosani N. Scope of Artificial Intelligence in Gastrointestinal Oncology. Cancers (Basel) 2021; 13:5494. [PMID: 34771658 PMCID: PMC8582733 DOI: 10.3390/cancers13215494] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Accepted: 10/27/2021] [Indexed: 12/12/2022] Open
Abstract
Gastrointestinal cancers are among the leading causes of death worldwide, with over 2.8 million deaths annually. Over the last few decades, advancements in artificial intelligence technologies have led to their application in medicine. The use of artificial intelligence in endoscopic procedures is a significant breakthrough in modern medicine. Currently, the diagnosis of various gastrointestinal cancer relies on the manual interpretation of radiographic images by radiologists and various endoscopic images by endoscopists. This can lead to diagnostic variabilities as it requires concentration and clinical experience in the field. Artificial intelligence using machine or deep learning algorithms can provide automatic and accurate image analysis and thus assist in diagnosis. In the field of gastroenterology, the application of artificial intelligence can be vast from diagnosis, predicting tumor histology, polyp characterization, metastatic potential, prognosis, and treatment response. It can also provide accurate prediction models to determine the need for intervention with computer-aided diagnosis. The number of research studies on artificial intelligence in gastrointestinal cancer has been increasing rapidly over the last decade due to immense interest in the field. This review aims to review the impact, limitations, and future potentials of artificial intelligence in screening, diagnosis, tumor staging, treatment modalities, and prediction models for the prognosis of various gastrointestinal cancers.
Collapse
Affiliation(s)
- Hemant Goyal
- Department of Internal Medicine, The Wright Center for Graduate Medical Education, 501 S. Washington Avenue, Scranton, PA 18505, USA
| | - Syed A. A. Sherazi
- Department of Medicine, John H Stroger Jr Hospital of Cook County, 1950 W Polk St, Chicago, IL 60612, USA;
| | - Rupinder Mann
- Department of Medicine, Saint Agnes Medical Center, 1303 E. Herndon Ave, Fresno, CA 93720, USA;
| | - Zainab Gandhi
- Department of Medicine, Geisinger Wyoming Valley Medical Center, 1000 E Mountain Dr, Wilkes-Barre, PA 18711, USA;
| | - Abhilash Perisetti
- Division of Interventional Oncology & Surgical Endoscopy (IOSE), Parkview Cancer Institute, 11050 Parkview Circle, Fort Wayne, IN 46845, USA; (A.P.); (N.S.)
| | - Muhammad Aziz
- Department of Gastroenterology and Hepatology, University of Toledo Medical Center, 3000 Arlington Avenue, Toledo, OH 43614, USA;
| | - Saurabh Chandan
- Division of Gastroenterology and Hepatology, CHI Health Creighton University Medical Center, 7500 Mercy Rd, Omaha, NE 68124, USA;
| | - Jonathan Kopel
- Department of Medicine, Texas Tech University Health Sciences Center, 3601 4th St, Lubbock, TX 79430, USA;
| | - Benjamin Tharian
- Department of Gastroenterology and Hepatology, The University of Arkansas for Medical Sciences, 4301 W Markham St, Little Rock, AR 72205, USA;
| | - Neil Sharma
- Division of Interventional Oncology & Surgical Endoscopy (IOSE), Parkview Cancer Institute, 11050 Parkview Circle, Fort Wayne, IN 46845, USA; (A.P.); (N.S.)
| | - Nirav Thosani
- Division of Gastroenterology, Hepatology & Nutrition, McGovern Medical School, UTHealth, 6410 Fannin, St #1014, Houston, TX 77030, USA;
| |
Collapse
|
54
|
Wang J, Jin Y, Cai S, Xu H, Heng PA, Qin J, Wang L. Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network. Med Image Anal 2021; 75:102291. [PMID: 34753019 DOI: 10.1016/j.media.2021.102291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2021] [Revised: 10/22/2021] [Accepted: 10/25/2021] [Indexed: 10/19/2022]
Abstract
We propose a novel shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection (ESD) surgery. This task is of great clinical significance but extremely challenging due to bleeding, lighting reflection, and motion blur in the complicated surgical environment. Compared with existing solutions, which either neglect geometric relationships among targeting objects or capture the relationships by using complicated aggregation schemes, the proposed network is capable of achieving satisfactory accuracy while maintaining real-time performance by taking full advantage of the spatial relations among landmarks. We first devise an algorithm to automatically generate relation keypoint heatmaps, which are able to intuitively represent the prior knowledge of spatial relations among landmarks without using any extra manual annotation efforts. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process. While one scheme introduces pixel-level regularization by multi-task learning, the other integrates global-level regularization by harnessing a newly designed grouped consistency evaluator, which adds relation constraints to the proposed network in an adversarial manner. Both schemes are beneficial to the model in training, and can be readily unloaded in inference to achieve real-time detection. We establish a large in-house dataset of ESD surgery for esophageal cancer to validate the effectiveness of our proposed method. Extensive experimental results demonstrate that our approach outperforms state-of-the-art methods in terms of accuracy and efficiency, achieving better detection results faster. Promising results on two downstream applications further corroborate the great potential of our method in ESD clinical practice.
Collapse
Affiliation(s)
- Jiacheng Wang
- Department of Computer Science at School of Informatics, Xiamen University, Xiamen 361005, China
| | - Yueming Jin
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, China
| | - Shuntian Cai
- Department of Gastroenterology, Zhongshan Hospital affiliated to Xiamen University, Xiamen, China
| | - Hongzhi Xu
- Department of Gastroenterology, Zhongshan Hospital affiliated to Xiamen University, Xiamen, China
| | - Pheng-Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, China
| | - Jing Qin
- Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong
| | - Liansheng Wang
- Department of Computer Science at School of Informatics, Xiamen University, Xiamen 361005, China.
| |
Collapse
|
55
|
Zhu XW, Yan J, He YL, Liu G, Li X. Application of deep learning based artificial intelligence technology in identification of colorectal polyps. Shijie Huaren Xiaohua Zazhi 2021; 29:1201-1206. [DOI: 10.11569/wcjd.v29.i20.1201] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023] Open
Abstract
Colorectal cancer is a cancer type that is most suitable for screening since subjects at risk of this malignancy can clearly benefit from colonoscopy screening. In 2017, there were about 431951 new cases of colorectal cancer in China, with an increase of 203.5% in 28 years. Early detection and early removal of adenomatous polyps and other precancerous lesions during colonoscopy can prevent the occurrence of colorectal cancer. However, various factors lead to missed diagnosis of polyps during colonoscopy, which increases the risk of colorectal cancer. In recent years, with the rapid development of artificial intelligence technology in the medical field, colonoscopy assisted by artificial intelligence can increase the detection rate of polyps and improve the quality of colonoscopy. This paper mainly reviews the quality control, bowel preparation, diagnosis and classification of colorectal polyps, and the future opportunities and challenges faced by convolutional neural network based artificial intelligence technology in the field of colonoscopy, hoping to provide some reference for clinical work.
Collapse
Affiliation(s)
- Xing-Wang Zhu
- The First Clinical Medical College of Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Jun Yan
- The First Clinical Medical College of Lanzhou University, Lanzhou 730000, Gansu Province, China,Gansu Province Key Laboratory of Biological Therapy and Regenerative Medicine, Lanzhou 730000, Gansu Province, China,Cancer Prevention and Treatment Center of Lanzhou University School of Medicine, Lanzhou 730000, Gansu Province, China,Gansu Provincial Institute of Hepatobiliary and Pancreatic Surgery, Lanzhou 730000, Gansu Province, China,Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Ying-Li He
- Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Gang Liu
- Lanzhou University School of Information Science & Engineering, Lanzhou 730000, Gansu Province, China
| | - Xun Li
- The First Clinical Medical College of Lanzhou University, Lanzhou 730000, Gansu Province, China,Gansu Province Key Laboratory of Biological Therapy and Regenerative Medicine, Lanzhou 730000, Gansu Province, China,Cancer Prevention and Treatment Center of Lanzhou University School of Medicine, Lanzhou 730000, Gansu Province, China,Gansu Provincial Institute of Hepatobiliary and Pancreatic Surgery, Lanzhou 730000, Gansu Province, China,Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, Gansu Province, China
| |
Collapse
|
56
|
Viscaino M, Torres Bustos J, Muñoz P, Auat Cheein C, Cheein FA. Artificial intelligence for the early detection of colorectal cancer: A comprehensive review of its advantages and misconceptions. World J Gastroenterol 2021; 27:6399-6414. [PMID: 34720530 PMCID: PMC8517786 DOI: 10.3748/wjg.v27.i38.6399] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 04/26/2021] [Accepted: 09/14/2021] [Indexed: 02/06/2023] Open
Abstract
Colorectal cancer (CRC) was the second-ranked worldwide type of cancer during 2020 due to the crude mortality rate of 12.0 per 100000 inhabitants. It can be prevented if glandular tissue (adenomatous polyps) is detected early. Colonoscopy has been strongly recommended as a screening test for both early cancer and adenomatous polyps. However, it has some limitations that include the high polyp miss rate for smaller (< 10 mm) or flat polyps, which are easily missed during visual inspection. Due to the rapid advancement of technology, artificial intelligence (AI) has been a thriving area in different fields, including medicine. Particularly, in gastroenterology AI software has been included in computer-aided systems for diagnosis and to improve the assertiveness of automatic polyp detection and its classification as a preventive method for CRC. This article provides an overview of recent research focusing on AI tools and their applications in the early detection of CRC and adenomatous polyps, as well as an insightful analysis of the main advantages and misconceptions in the field.
Collapse
Affiliation(s)
- Michelle Viscaino
- Department of Electronic Engineering, Universidad Tecnica Federico Santa Maria, Valpaiso 2340000, Chile
| | - Javier Torres Bustos
- Department of Electronic Engineering, Universidad Tecnica Federico Santa Maria, Valpaiso 2340000, Chile
| | - Pablo Muñoz
- Hospital Clinico, University of Chile, Santiago 8380456, Chile
| | - Cecilia Auat Cheein
- Facultad de Medicina, Universidad Nacional de Santiago del Estero, Santiago del Estero 4200, Argentina
| | - Fernando Auat Cheein
- Department of Electronic Engineering, Universidad Técnica Federico Santa María, Valparaiso 2340000, Chile
| |
Collapse
|
57
|
Haak HE, Gao X, Maas M, Waktola S, Benson S, Beets-Tan RGH, Beets GL, van Leerdam M, Melenhorst J. The use of deep learning on endoscopic images to assess the response of rectal cancer after chemoradiation. Surg Endosc 2021; 36:3592-3600. [PMID: 34642794 PMCID: PMC9001548 DOI: 10.1007/s00464-021-08685-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2020] [Accepted: 08/07/2021] [Indexed: 12/30/2022]
Abstract
Background Accurate response evaluation is necessary to select complete responders (CRs) for a watch-and-wait approach. Deep learning may aid in this process, but so far has never been evaluated for this purpose. The aim was to evaluate the accuracy to assess response with deep learning methods based on endoscopic images in rectal cancer patients after neoadjuvant therapy. Methods Rectal cancer patients diagnosed between January 2012 and December 2015 and treated with neoadjuvant (chemo)radiotherapy were retrospectively selected from a single institute. All patients underwent flexible endoscopy for response evaluation. Diagnostic performance (accuracy, area under the receiver operator characteristics curve (AUC), positive- and negative predictive values, sensitivities and specificities) of different open accessible deep learning networks was calculated. Reference standard was histology after surgery, or long-term outcome (>2 years of follow-up) in a watch-and-wait policy. Results 226 patients were included for the study (117(52%) were non-CRs; 109(48%) were CRs). The accuracy, AUC, positive- and negative predictive values, sensitivity and specificity of the different models varied from 0.67–0.75%, 0.76–0.83%, 67–74%, 70–78%, 68–79% to 66–75%, respectively. Overall, EfficientNet-B2 was the most successful model with the highest diagnostic performance. Conclusions This pilot study shows that deep learning has a modest accuracy (AUCs 0.76-0.83). This is not accurate enough for clinical decision making, and lower than what is generally reported by experienced endoscopists. Deep learning models can however be further improved and may become useful to assist endoscopists in evaluating the response. More well-designed prospective studies are required. Supplementary Information The online version contains supplementary material available at 10.1007/s00464-021-08685-7.
Collapse
Affiliation(s)
- Hester E Haak
- Department of Surgery, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands.,GROW School for Oncology and Developmental Biology-Maastricht University, Maastricht, The Netherlands
| | - Xinpei Gao
- Department of Radiology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - Monique Maas
- Department of Radiology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - Selam Waktola
- Department of Radiology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - Sean Benson
- Department of Radiology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - Regina G H Beets-Tan
- GROW School for Oncology and Developmental Biology-Maastricht University, Maastricht, The Netherlands.,Department of Radiology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - Geerard L Beets
- Department of Surgery, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands.,GROW School for Oncology and Developmental Biology-Maastricht University, Maastricht, The Netherlands
| | - Monique van Leerdam
- Department of Gastroenterology, Netherlands Cancer Institute-Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - Jarno Melenhorst
- GROW School for Oncology and Developmental Biology-Maastricht University, Maastricht, The Netherlands. .,Department of Surgery, Maastricht University Medical Centre, Postbox 5800, 6202 AZ, Maastricht, The Netherlands.
| |
Collapse
|
58
|
Si X, Zhang X, Zhou Y, Chao Y, Lim SN, Sun Y, Yin S, Jin W, Zhao X, Li Q, Ming D. White matter structural connectivity as a biomarker for detecting juvenile myoclonic epilepsy by transferred deep convolutional neural networks with varying transfer rates. J Neural Eng 2021; 18. [PMID: 34507303 DOI: 10.1088/1741-2552/ac25d8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2021] [Accepted: 09/10/2021] [Indexed: 11/12/2022]
Abstract
Objective. By detecting abnormal white matter changes, diffusion magnetic resonance imaging (MRI) contributes to the detection of juvenile myoclonic epilepsy (JME). In addition, deep learning has greatly improved the detection performance of various brain disorders. However, there is almost no previous study effectively detecting JME by a deep learning approach with diffusion MRI.Approach. In this study, the white matter structural connectivity was generated by tracking the white matter fibers in detail based on Q-ball imaging and neurite orientation dispersion and density imaging. Four advanced deep convolutional neural networks (CNNs) were deployed by using the transfer learning approach, in which the transfer rate searching strategy was proposed to achieve the best detection performance.Main results. Our results showed: (a) Compared to normal control, the white matter' neurite density of JME was significantly decreased. The most significantly abnormal fiber tracts between the two groups were found to be cortico-cortical connection tracts. (b) The proposed transfer rate searching approach contributed to find each CNN's best performance, in which the best JME detection accuracy of 92.2% was achieved by using the Inception_resnet_v2 network with a 16% transfer rate.Significance. The results revealed: (a) Through detection of the abnormal white matter changes, the white matter structural connectivity can be used as a useful biomarker for detecting JME, which helps to characterize the pathophysiology of epilepsy. (b) The proposed transfer rate, as a new hyperparameter, promotes the CNNs transfer learning performance in detecting JME.
Collapse
Affiliation(s)
- Xiaopeng Si
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China.,Institute of Applied Psychology, Tianjin University, Tianjin 300350, People's Republic of China
| | - Xingjian Zhang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Yu Zhou
- School of Microelectronics, Tianjin University, Tianjin 300072, People's Republic of China
| | - Yiping Chao
- Graduate Institute of Biomedical Engineering, Chang Gung University, Taoyuan 33302, Taiwan
| | - Siew-Na Lim
- Department of Neurology, Chang Gung Memorial Hospital, Taoyuan 333, Taiwan
| | - Yulin Sun
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Shaoya Yin
- Department of Neurosurgery, Huanhu Hospital, Tianjin University, Tianjin 300072, People's Republic of China
| | - Weipeng Jin
- Department of Neurosurgery, Huanhu Hospital, Tianjin University, Tianjin 300072, People's Republic of China
| | - Xin Zhao
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Qiang Li
- School of Microelectronics, Tianjin University, Tianjin 300072, People's Republic of China
| | - Dong Ming
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| |
Collapse
|
59
|
Kumar Y, Gupta S, Singla R, Hu YC. A Systematic Review of Artificial Intelligence Techniques in Cancer Prediction and Diagnosis. ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING : STATE OF THE ART REVIEWS 2021; 29:2043-2070. [PMID: 34602811 PMCID: PMC8475374 DOI: 10.1007/s11831-021-09648-w] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2021] [Accepted: 09/11/2021] [Indexed: 05/05/2023]
Abstract
Artificial intelligence has aided in the advancement of healthcare research. The availability of open-source healthcare statistics has prompted researchers to create applications that aid cancer detection and prognosis. Deep learning and machine learning models provide a reliable, rapid, and effective solution to deal with such challenging diseases in these circumstances. PRISMA guidelines had been used to select the articles published on the web of science, EBSCO, and EMBASE between 2009 and 2021. In this study, we performed an efficient search and included the research articles that employed AI-based learning approaches for cancer prediction. A total of 185 papers are considered impactful for cancer prediction using conventional machine and deep learning-based classifications. In addition, the survey also deliberated the work done by the different researchers and highlighted the limitations of the existing literature, and performed the comparison using various parameters such as prediction rate, accuracy, sensitivity, specificity, dice score, detection rate, area undercover, precision, recall, and F1-score. Five investigations have been designed, and solutions to those were explored. Although multiple techniques recommended in the literature have achieved great prediction results, still cancer mortality has not been reduced. Thus, more extensive research to deal with the challenges in the area of cancer prediction is required.
Collapse
Affiliation(s)
- Yogesh Kumar
- Department of Computer Engineering, Indus Institute of Technology & Engineering, Indus University, Rancharda, Via: Shilaj, Ahmedabad, Gujarat 382115 India
| | - Surbhi Gupta
- School of Computer Science and Engineering, Model Institute of Engineering and Technology, Kot bhalwal, Jammu, J&K 181122 India
| | - Ruchi Singla
- Department of Research, Innovations, Sponsored Projects and Entrepreneurship, Chandigarh Group of Colleges, Landran, Mohali India
| | - Yu-Chen Hu
- Department of Computer Science and Information Management, Providence University, Taichung City, Taiwan, ROC
| |
Collapse
|
60
|
Ursuleanu TF, Luca AR, Gheorghe L, Grigorovici R, Iancu S, Hlusneac M, Preda C, Grigorovici A. Deep Learning Application for Analyzing of Constituents and Their Correlations in the Interpretations of Medical Images. Diagnostics (Basel) 2021; 11:1373. [PMID: 34441307 PMCID: PMC8393354 DOI: 10.3390/diagnostics11081373] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Revised: 07/25/2021] [Accepted: 07/27/2021] [Indexed: 12/13/2022] Open
Abstract
The need for time and attention, given by the doctor to the patient, due to the increased volume of medical data to be interpreted and filtered for diagnostic and therapeutic purposes has encouraged the development of the option to support, constructively and effectively, deep learning models. Deep learning (DL) has experienced an exponential development in recent years, with a major impact on interpretations of the medical image. This has influenced the development, diversification and increase of the quality of scientific data, the development of knowledge construction methods and the improvement of DL models used in medical applications. All research papers focus on description, highlighting, classification of one of the constituent elements of deep learning models (DL), used in the interpretation of medical images and do not provide a unified picture of the importance and impact of each constituent in the performance of DL models. The novelty in our paper consists primarily in the unitary approach, of the constituent elements of DL models, namely, data, tools used by DL architectures or specifically constructed DL architecture combinations and highlighting their "key" features, for completion of tasks in current applications in the interpretation of medical images. The use of "key" characteristics specific to each constituent of DL models and the correct determination of their correlations, may be the subject of future research, with the aim of increasing the performance of DL models in the interpretation of medical images.
Collapse
Affiliation(s)
- Tudor Florin Ursuleanu
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Surgery VI, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
- Department of Surgery I, Regional Institute of Oncology, 700483 Iasi, Romania
| | - Andreea Roxana Luca
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department Obstetrics and Gynecology, Integrated Ambulatory of Hospital “Sf. Spiridon”, 700106 Iasi, Romania
| | - Liliana Gheorghe
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Radiology, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
| | - Roxana Grigorovici
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
| | - Stefan Iancu
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
| | - Maria Hlusneac
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
| | - Cristina Preda
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Endocrinology, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
| | - Alexandru Grigorovici
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Surgery VI, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
| |
Collapse
|
61
|
Yan L, Li S, Guo Y, Ren P, Song H, Yang J, Shen X. Multi-state colposcopy image fusion for cervical precancerous lesion diagnosis using BF-CNN. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102700] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
62
|
Yu H, Yang LT, Zhang Q, Armstrong D, Deen MJ. Convolutional neural networks for medical image analysis: State-of-the-art, comparisons, improvement and perspectives. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2020.04.157] [Citation(s) in RCA: 46] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
63
|
Liew WS, Tang TB, Lin CH, Lu CK. Automatic colonic polyp detection using integration of modified deep residual convolutional neural network and ensemble learning approaches. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 206:106114. [PMID: 33984661 DOI: 10.1016/j.cmpb.2021.106114] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Accepted: 04/07/2021] [Indexed: 05/10/2023]
Abstract
BACKGROUND AND OBJECTIVE The increased incidence of colorectal cancer (CRC) and its mortality rate have attracted interest in the use of artificial intelligence (AI) based computer-aided diagnosis (CAD) tools to detect polyps at an early stage. Although these CAD tools have thus far achieved a good accuracy level to detect polyps, they still have room to improve further (e.g. sensitivity). Therefore, a new CAD tool is developed in this study to detect colonic polyps accurately. METHODS In this paper, we propose a novel approach to distinguish colonic polyps by integrating several techniques, including a modified deep residual network, principal component analysis and AdaBoost ensemble learning. A powerful deep residual network architecture, ResNet-50, was investigated to reduce the computational time by altering its architecture. To keep the interference to a minimum, median filter, image thresholding, contrast enhancement, and normalisation techniques were exploited on the endoscopic images to train the classification model. Three publicly available datasets, i.e., Kvasir, ETIS-LaribPolypDB, and CVC-ClinicDB, were merged to train the model, which included images with and without polyps. RESULTS The proposed approach trained with a combination of three datasets achieved Matthews Correlation Coefficient (MCC) of 0.9819 with accuracy, sensitivity, precision, and specificity of 99.10%, 98.82%, 99.37%, and 99.38%, respectively. CONCLUSIONS These results show that our method could repeatedly classify endoscopic images automatically and could be used to effectively develop computer-aided diagnostic tools for early CRC detection.
Collapse
Affiliation(s)
- Win Sheng Liew
- Department of Electrical and Electronic Engineering, Universiti Teknologi PETRONAS, 32610 Seri Iskandar, Perak, Malaysia
| | - Tong Boon Tang
- Department of Electrical and Electronic Engineering, Universiti Teknologi PETRONAS, 32610 Seri Iskandar, Perak, Malaysia
| | - Cheng-Hung Lin
- Department of Electrical Engineering and Biomedical Engineering Research Center, Yuan Ze University, Jungli 32003, Taiwan
| | - Cheng-Kai Lu
- Department of Electrical and Electronic Engineering, Universiti Teknologi PETRONAS, 32610 Seri Iskandar, Perak, Malaysia.
| |
Collapse
|
64
|
Wang Y, Feng Y, Zhang L, Wang Z, Lv Q, Yi Z. Deep adversarial domain adaptation for breast cancer screening from mammograms. Med Image Anal 2021; 73:102147. [PMID: 34246849 DOI: 10.1016/j.media.2021.102147] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Revised: 11/10/2020] [Accepted: 06/23/2021] [Indexed: 02/05/2023]
Abstract
The early detection of breast cancer greatly increases the chances that the right decision for a successful treatment plan will be made. Deep learning approaches are used in breast cancer screening and have achieved promising results when a large-scale labeled dataset is available for training. However, they may suffer from a dramatic decrease in performance when annotated data are limited. In this paper, we propose a method called deep adversarial domain adaptation (DADA) to improve the performance of breast cancer screening using mammography. Specifically, our aim is to extract the knowledge from a public dataset (source domain) and transfer the learned knowledge to improve the detection performance on the target dataset (target domain). Because of the different distributions of the source and target domains, the proposed method adopts an adversarial learning technique to perform domain adaptation using the two domains. Specifically, the adversarial procedure is trained by taking advantage of the disagreement of two classifiers. To evaluate the proposed method, the public well-labeled image-level dataset Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM) is employed as the source domain. Mammography samples from the West China Hospital were collected to construct our target domain dataset, and the samples are annotated at case-level based on the corresponding pathological reports. The experimental results demonstrate the effectiveness of the proposed method compared with several other state-of-the-art automatic breast cancer screening approaches.
Collapse
Affiliation(s)
- Yan Wang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China; Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Yangqin Feng
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Lei Zhang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China.
| | - Zizhou Wang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China
| | - Qing Lv
- Department of Galactophore Surgery, West China Hospital, Sichuan University, Chengdu 610041, PR China
| | - Zhang Yi
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China
| |
Collapse
|
65
|
Parsa N, Byrne MF. Artificial intelligence for identification and characterization of colonic polyps. Ther Adv Gastrointest Endosc 2021; 14:26317745211014698. [PMID: 34263163 PMCID: PMC8252334 DOI: 10.1177/26317745211014698] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Accepted: 04/07/2021] [Indexed: 12/27/2022] Open
Abstract
Colonoscopy remains the gold standard exam for colorectal cancer screening due to its ability to detect and resect pre-cancerous lesions in the colon. However, its performance is greatly operator dependent. Studies have shown that up to one-quarter of colorectal polyps can be missed on a single colonoscopy, leading to high rates of interval colorectal cancer. In addition, the American Society for Gastrointestinal Endoscopy has proposed the “resect-and-discard” and “diagnose-and-leave” strategies for diminutive colorectal polyps to reduce the costs of unnecessary polyp resection and pathology evaluation. However, the performance of optical biopsy has been suboptimal in community practice. With recent improvements in machine-learning techniques, artificial intelligence–assisted computer-aided detection and diagnosis have been increasingly utilized by endoscopists. The application of computer-aided design on real-time colonoscopy has been shown to increase the adenoma detection rate while decreasing the withdrawal time and improve endoscopists’ optical biopsy accuracy, while reducing the time to make the diagnosis. These are promising steps toward standardization and improvement of colonoscopy quality, and implementation of “resect-and-discard” and “diagnose-and-leave” strategies. Yet, issues such as real-world applications and regulatory approval need to be addressed before artificial intelligence models can be successfully implemented in clinical practice. In this review, we summarize the recent literature on the application of artificial intelligence for detection and characterization of colorectal polyps and review the limitation of existing artificial intelligence technologies and future directions for this field.
Collapse
Affiliation(s)
- Nasim Parsa
- Division of Gastroenterology and Hepatology, Department of Medicine, University of Missouri, Columbia, MO 65211, USA
| | - Michael F Byrne
- Division of Gastroenterology, Department of Medicine, The University of British Columbia, Vancouver, BC, Canada; Satisfai Health, Vancouver, BC, Canada
| |
Collapse
|
66
|
Shah N, Jyala A, Patel H, Makker J. Utility of artificial intelligence in colonoscopy. Artif Intell Gastrointest Endosc 2021; 2:79-88. [DOI: 10.37126/aige.v2.i3.79] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 06/20/2021] [Accepted: 06/28/2021] [Indexed: 02/06/2023] Open
Abstract
Colorectal cancer is one of the major causes of death worldwide. Colonoscopy is the most important tool that can identify neoplastic lesion in early stages and resect it in a timely manner which helps in reducing mortality related to colorectal cancer. However, the quality of colonoscopy findings depends on the expertise of the endoscopist and thus the rate of missed adenoma or polyp cannot be controlled. It is desirable to standardize the quality of colonoscopy by reducing the number of missed adenoma/polyps. Introduction of artificial intelligence (AI) in the field of medicine has become popular among physicians nowadays. The application of AI in colonoscopy can help in reducing miss rate and increasing colorectal cancer detection rate as per recent studies. Moreover, AI assistance during colonoscopy has also been utilized in patients with inflammatory bowel disease to improve diagnostic accuracy, assessing disease severity and predicting clinical outcomes. We conducted a literature review on the available evidence on use of AI in colonoscopy. In this review article, we discuss about the principles, application, limitations, and future aspects of AI in colonoscopy.
Collapse
Affiliation(s)
- Niel Shah
- Department of Internal Medicine, BronxCare Hospital Center, Bronx, NY 10457, United States
| | - Abhilasha Jyala
- Department of Internal Medicine, BronxCare Hospital Center, Bronx, NY 10457, United States
| | - Harish Patel
- Department of Internal Medicine, Gastroenterology, BronxCare Hospital Center, Bronx, NY 10457, United States
| | - Jasbir Makker
- Department of Internal Medicine, Gastroenterology, BronxCare Hospital Center, Bronx, NY 10457, United States
| |
Collapse
|
67
|
Wang RG. Progress and prospects of artificial intelligence in colonoscopy. Artif Intell Gastrointest Endosc 2021; 2:63-70. [DOI: 10.37126/aige.v2.i3.63] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 05/29/2021] [Accepted: 06/23/2021] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence (AI) is a branch of computer science. As a new technological science, it mainly develops and expands human intelligence through the research of intelligence theory, methods and technology. In the medical field, AI has bright application prospects (for example: imaging, diagnosis and treatment). The exploration of robotic gastroscopy and colonoscopy systems is not only a bold attempt, but also an inevitable trend of AI in the development of digestive endoscopy in the future. Based on the current research findings, this article summarizes the research progress of colonoscopy, and looking forward for the application of AI in colonoscopy.
Collapse
Affiliation(s)
- Rui-Gang Wang
- Department of Gastroenterology, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing 102218, China
| |
Collapse
|
68
|
Shah N, Jyala A, Patel H, Makker J. Utility of artificial intelligence in colonoscopy. Artif Intell Gastrointest Endosc 2021. [DOI: 10.37126/aige.v2.i3.78] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
|
69
|
Bardhi O, Sierra-Sosa D, Garcia-Zapirain B, Bujanda L. Deep Learning Models for Colorectal Polyps. INFORMATION 2021; 12:245. [DOI: 10.3390/info12060245] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/19/2023] Open
Abstract
Colorectal cancer is one of the main causes of cancer incident cases and cancer deaths worldwide. Undetected colon polyps, be them benign or malignant, lead to late diagnosis of colorectal cancer. Computer aided devices have helped to decrease the polyp miss rate. The application of deep learning algorithms and techniques has escalated during this last decade. Many scientific studies are published to detect, localize, and classify colon polyps. We present here a brief review of the latest published studies. We compare the accuracy of these studies with our results obtained from training and testing three independent datasets using a convolutional neural network and autoencoder model. A train, validate and test split was performed for each dataset, 75%, 15%, and 15%, respectively. An accuracy of 0.937 was achieved for CVC-ColonDB, 0.951 for CVC-ClinicDB, and 0.967 for ETIS-LaribPolypDB. Our results suggest slight improvements compared to the algorithms used to date.
Collapse
|
70
|
Kim KO, Kim EY. Application of Artificial Intelligence in the Detection and Characterization of Colorectal Neoplasm. Gut Liver 2021; 15:346-353. [PMID: 32773386 PMCID: PMC8129657 DOI: 10.5009/gnl20186] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/12/2020] [Accepted: 06/28/2020] [Indexed: 12/19/2022] Open
Abstract
Endoscpists always have tried to pursue a perfect colonoscopy, and application of artificial intelligence (AI) using deep-learning algorithms is one of the promising supportive options for detection and characterization of colorectal polyps during colonoscopy. Many retrospective studies conducted with real-time application of AI using convolutional neural networks have shown improved colorectal polyp detection. Moreover, a recent randomized clinical trial reported additional polyp detection with shorter analysis time. Studies conducted regarding polyp characterization provided additional promising results. Application of AI with narrow band imaging in real-time prediction of the pathology of diminutive polyps resulted in high diagnostic accuracy. In addition, application of AI with endocytoscopy or confocal laser endomicroscopy was investigated for real-time cellular diagnosis, and the diagnostic accuracy of some studies was comparable to that of pathologists. With AI technology, we can expect a higher polyp detection rate with reduced time and cost by avoiding unnecessary procedures, resulting in enhanced colonoscopy efficiency. However, for AI application in actual daily clinical practice, more prospective studies with minimized selection bias, consensus on standardized utilization, and regulatory approval are needed. (Gut Liver 2021;15:-353)
Collapse
Affiliation(s)
- Kyeong Ok Kim
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Yeungnam University College of Medicine, Daegu, Korea
| | - Eun Young Kim
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Daegu Catholic University School of Medicine, Daegu, Korea
| |
Collapse
|
71
|
Mitsala A, Tsalikidis C, Pitiakoudis M, Simopoulos C, Tsaroucha AK. Artificial Intelligence in Colorectal Cancer Screening, Diagnosis and Treatment. A New Era. ACTA ACUST UNITED AC 2021; 28:1581-1607. [PMID: 33922402 PMCID: PMC8161764 DOI: 10.3390/curroncol28030149] [Citation(s) in RCA: 77] [Impact Index Per Article: 25.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Revised: 04/09/2021] [Accepted: 04/20/2021] [Indexed: 12/24/2022]
Abstract
The development of artificial intelligence (AI) algorithms has permeated the medical field with great success. The widespread use of AI technology in diagnosing and treating several types of cancer, especially colorectal cancer (CRC), is now attracting substantial attention. CRC, which represents the third most commonly diagnosed malignancy in both men and women, is considered a leading cause of cancer-related deaths globally. Our review herein aims to provide in-depth knowledge and analysis of the AI applications in CRC screening, diagnosis, and treatment based on current literature. We also explore the role of recent advances in AI systems regarding medical diagnosis and therapy, with several promising results. CRC is a highly preventable disease, and AI-assisted techniques in routine screening represent a pivotal step in declining incidence rates of this malignancy. So far, computer-aided detection and characterization systems have been developed to increase the detection rate of adenomas. Furthermore, CRC treatment enters a new era with robotic surgery and novel computer-assisted drug delivery techniques. At the same time, healthcare is rapidly moving toward precision or personalized medicine. Machine learning models have the potential to contribute to individual-based cancer care and transform the future of medicine.
Collapse
Affiliation(s)
- Athanasia Mitsala
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
- Correspondence: ; Tel.: +30-6986423707
| | - Christos Tsalikidis
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
| | - Michail Pitiakoudis
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
| | - Constantinos Simopoulos
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
| | - Alexandra K. Tsaroucha
- Laboratory of Experimental Surgery & Surgical Research, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece;
| |
Collapse
|
72
|
Pang X, Zhao Z, Weng Y. The Role and Impact of Deep Learning Methods in Computer-Aided Diagnosis Using Gastrointestinal Endoscopy. Diagnostics (Basel) 2021; 11:694. [PMID: 33919669 PMCID: PMC8069844 DOI: 10.3390/diagnostics11040694] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2021] [Revised: 03/24/2021] [Accepted: 04/01/2021] [Indexed: 12/18/2022] Open
Abstract
At present, the application of artificial intelligence (AI) based on deep learning in the medical field has become more extensive and suitable for clinical practice compared with traditional machine learning. The application of traditional machine learning approaches to clinical practice is very challenging because medical data are usually uncharacteristic. However, deep learning methods with self-learning abilities can effectively make use of excellent computing abilities to learn intricate and abstract features. Thus, they are promising for the classification and detection of lesions through gastrointestinal endoscopy using a computer-aided diagnosis (CAD) system based on deep learning. This study aimed to address the research development of a CAD system based on deep learning in order to assist doctors in classifying and detecting lesions in the stomach, intestines, and esophagus. It also summarized the limitations of the current methods and finally presented a prospect for future research.
Collapse
Affiliation(s)
- Xuejiao Pang
- School of Control Science and Engineering, Shandong University, Jinan 250061, China;
| | - Zijian Zhao
- School of Control Science and Engineering, Shandong University, Jinan 250061, China;
| | - Ying Weng
- School of Computer Science, University of Nottingham, Nottingham NG7 2RD, UK;
| |
Collapse
|
73
|
Li Z, Xia Y. Deep Reinforcement Learning for Weakly-Supervised Lymph Node Segmentation in CT Images. IEEE J Biomed Health Inform 2021; 25:774-783. [PMID: 32749988 DOI: 10.1109/jbhi.2020.3008759] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Accurate and automated lymph node segmentation is pivotal for quantitatively accessing disease progression and potential therapeutics. The complex variation of lymph node morphology and the difficulty of acquiring voxel-wise manual annotations make lymph node segmentation a challenging task. Since the Response Evaluation Criteria in Solid Tumors (RECIST) annotation, which indicates the location, length, and width of a lymph node, is commonly available in hospital data archives, we advocate to use RECIST annotations as the supervision, and thus formulate this segmentation task into a weakly-supervised learning problem. In this paper, we propose a deep reinforcement learning-based lymph node segmentation (DRL-LNS) model. Based on RECIST annotations, we segment RECIST-slices in an unsupervised way to produce pseudo ground truths, which are then used to train U-Net as a segmentation network. Next, we train a DRL model, in which the segmentation network interacts with the policy network to optimize the lymph node bounding boxes and segmentation results simultaneously. The proposed DRL-LNS model was evaluated against three widely used image segmentation networks on a public thoracoabdominal Computed Tomography (CT) dataset that contains 984 3D lymph nodes, and achieves the mean Dice similarity coefficient (DSC) of 77.17% and the mean Intersection over Union (IoU) of 64.78% in the four-fold cross-validation. Our results suggest that the DRL-based bounding box prediction strategy outperforms the label propagation strategy and the proposed DRL-LNS model is able to achieve the state-of-the-art performance on this weakly-supervised lymph node segmentation task.
Collapse
|
74
|
Guleria S, Shah TU, Pulido JV, Fasullo M, Ehsan L, Lippman R, Sali R, Mutha P, Cheng L, Brown DE, Syed S. Deep learning systems detect dysplasia with human-like accuracy using histopathology and probe-based confocal laser endomicroscopy. Sci Rep 2021; 11:5086. [PMID: 33658592 DOI: 10.1038/s41598-021-84510-411:5086] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Accepted: 02/15/2021] [Indexed: 05/28/2023] Open
Abstract
Probe-based confocal laser endomicroscopy (pCLE) allows for real-time diagnosis of dysplasia and cancer in Barrett's esophagus (BE) but is limited by low sensitivity. Even the gold standard of histopathology is hindered by poor agreement between pathologists. We deployed deep-learning-based image and video analysis in order to improve diagnostic accuracy of pCLE videos and biopsy images. Blinded experts categorized biopsies and pCLE videos as squamous, non-dysplastic BE, or dysplasia/cancer, and deep learning models were trained to classify the data into these three categories. Biopsy classification was conducted using two distinct approaches-a patch-level model and a whole-slide-image-level model. Gradient-weighted class activation maps (Grad-CAMs) were extracted from pCLE and biopsy models in order to determine tissue structures deemed relevant by the models. 1970 pCLE videos, 897,931 biopsy patches, and 387 whole-slide images were used to train, test, and validate the models. In pCLE analysis, models achieved a high sensitivity for dysplasia (71%) and an overall accuracy of 90% for all classes. For biopsies at the patch level, the model achieved a sensitivity of 72% for dysplasia and an overall accuracy of 90%. The whole-slide-image-level model achieved a sensitivity of 90% for dysplasia and 94% overall accuracy. Grad-CAMs for all models showed activation in medically relevant tissue regions. Our deep learning models achieved high diagnostic accuracy for both pCLE-based and histopathologic diagnosis of esophageal dysplasia and its precursors, similar to human accuracy in prior studies. These machine learning approaches may improve accuracy and efficiency of current screening protocols.
Collapse
Affiliation(s)
- Shan Guleria
- Rush University Medical Center, Chicago, IL, USA
| | - Tilak U Shah
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| | - J Vincent Pulido
- Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA
- Department of Systems & Information Engineering, University of Virginia, Charlottesville, VA, USA
| | - Matthew Fasullo
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| | - Lubaina Ehsan
- Division of Gastroenterology, Hepatology, and Nutrition, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Robert Lippman
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
| | - Rasoul Sali
- Department of Systems & Information Engineering, University of Virginia, Charlottesville, VA, USA
| | - Pritesh Mutha
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| | - Lin Cheng
- Rush University Medical Center, Chicago, IL, USA
| | - Donald E Brown
- Department of Systems & Information Engineering, University of Virginia, Charlottesville, VA, USA
| | - Sana Syed
- Division of Gastroenterology, Hepatology, and Nutrition, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA.
| |
Collapse
|
75
|
Guleria S, Shah TU, Pulido JV, Fasullo M, Ehsan L, Lippman R, Sali R, Mutha P, Cheng L, Brown DE, Syed S. Deep learning systems detect dysplasia with human-like accuracy using histopathology and probe-based confocal laser endomicroscopy. Sci Rep 2021; 11:5086. [PMID: 33658592 PMCID: PMC7930108 DOI: 10.1038/s41598-021-84510-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Accepted: 02/15/2021] [Indexed: 12/20/2022] Open
Abstract
Probe-based confocal laser endomicroscopy (pCLE) allows for real-time diagnosis of dysplasia and cancer in Barrett's esophagus (BE) but is limited by low sensitivity. Even the gold standard of histopathology is hindered by poor agreement between pathologists. We deployed deep-learning-based image and video analysis in order to improve diagnostic accuracy of pCLE videos and biopsy images. Blinded experts categorized biopsies and pCLE videos as squamous, non-dysplastic BE, or dysplasia/cancer, and deep learning models were trained to classify the data into these three categories. Biopsy classification was conducted using two distinct approaches-a patch-level model and a whole-slide-image-level model. Gradient-weighted class activation maps (Grad-CAMs) were extracted from pCLE and biopsy models in order to determine tissue structures deemed relevant by the models. 1970 pCLE videos, 897,931 biopsy patches, and 387 whole-slide images were used to train, test, and validate the models. In pCLE analysis, models achieved a high sensitivity for dysplasia (71%) and an overall accuracy of 90% for all classes. For biopsies at the patch level, the model achieved a sensitivity of 72% for dysplasia and an overall accuracy of 90%. The whole-slide-image-level model achieved a sensitivity of 90% for dysplasia and 94% overall accuracy. Grad-CAMs for all models showed activation in medically relevant tissue regions. Our deep learning models achieved high diagnostic accuracy for both pCLE-based and histopathologic diagnosis of esophageal dysplasia and its precursors, similar to human accuracy in prior studies. These machine learning approaches may improve accuracy and efficiency of current screening protocols.
Collapse
Affiliation(s)
- Shan Guleria
- Rush University Medical Center, Chicago, IL, USA
| | - Tilak U Shah
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| | - J Vincent Pulido
- Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA
- Department of Systems & Information Engineering, University of Virginia, Charlottesville, VA, USA
| | - Matthew Fasullo
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| | - Lubaina Ehsan
- Division of Gastroenterology, Hepatology, and Nutrition, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Robert Lippman
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
| | - Rasoul Sali
- Department of Systems & Information Engineering, University of Virginia, Charlottesville, VA, USA
| | - Pritesh Mutha
- Hunter Holmes McGuire Veterans Affairs Medical Center, Richmond, VA, USA
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| | - Lin Cheng
- Rush University Medical Center, Chicago, IL, USA
| | - Donald E Brown
- Department of Systems & Information Engineering, University of Virginia, Charlottesville, VA, USA
| | - Sana Syed
- Division of Gastroenterology, Hepatology, and Nutrition, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA.
| |
Collapse
|
76
|
Nguyen TP, Chae DS, Park SJ, Kang KY, Yoon J. Deep learning system for Meyerding classification and segmental motion measurement in diagnosis of lumbar spondylolisthesis. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102371] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
77
|
An F, Li X, Ma X. Medical Image Classification Algorithm Based on Visual Attention Mechanism-MCNN. OXIDATIVE MEDICINE AND CELLULAR LONGEVITY 2021; 2021:6280690. [PMID: 33688390 PMCID: PMC7914083 DOI: 10.1155/2021/6280690] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Revised: 02/02/2021] [Accepted: 02/06/2021] [Indexed: 11/23/2022]
Abstract
Due to the complexity of medical images, traditional medical image classification methods have been unable to meet the actual application needs. In recent years, the rapid development of deep learning theory has provided a technical approach for solving medical image classification. However, deep learning has the following problems in the application of medical image classification. First, it is impossible to construct a deep learning model with excellent performance according to the characteristics of medical images. Second, the current deep learning network structure and training strategies are less adaptable to medical images. Therefore, this paper first introduces the visual attention mechanism into the deep learning model so that the information can be extracted more effectively according to the problem of medical images, and the reasoning is realized at a finer granularity. It can increase the interpretability of the model. Additionally, to solve the problem of matching the deep learning network structure and training strategy to medical images, this paper will construct a novel multiscale convolutional neural network model that can automatically extract high-level discriminative appearance features from the original image, and the loss function uses the Mahalanobis distance optimization model to obtain a better training strategy, which can improve the robust performance of the network model. The medical image classification task is completed by the above method. Based on the above ideas, this paper proposes a medical classification algorithm based on a visual attention mechanism-multiscale convolutional neural network. The lung nodules and breast cancer images were classified by the method in this paper. The experimental results show that the accuracy of medical image classification in this paper is not only higher than that of traditional machine learning methods but also improved compared with other deep learning methods, and the method has good stability and robustness.
Collapse
Affiliation(s)
- Fengping An
- School of Physics and Electronic Electrical Engineering, Huaiyin Normal University, Huaian 223300, China
| | - Xiaowei Li
- School of Physics and Electronic Electrical Engineering, Huaiyin Normal University, Huaian 223300, China
| | - Xingmin Ma
- System Second Department, North China Institute of Computing Technology, Beijing 100083, China
| |
Collapse
|
78
|
New polyp image classification technique using transfer learning of network-in-network structure in endoscopic images. Sci Rep 2021; 11:3605. [PMID: 33574394 PMCID: PMC7878472 DOI: 10.1038/s41598-021-83199-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2019] [Accepted: 01/18/2021] [Indexed: 12/13/2022] Open
Abstract
While colorectal cancer is known to occur in the gastrointestinal tract. It is the third most common form of cancer of 27 major types of cancer in South Korea and worldwide. Colorectal polyps are known to increase the potential of developing colorectal cancer. Detected polyps need to be resected to reduce the risk of developing cancer. This research improved the performance of polyp classification through the fine-tuning of Network-in-Network (NIN) after applying a pre-trained model of the ImageNet database. Random shuffling is performed 20 times on 1000 colonoscopy images. Each set of data are divided into 800 images of training data and 200 images of test data. An accuracy evaluation is performed on 200 images of test data in 20 experiments. Three compared methods were constructed from AlexNet by transferring the weights trained by three different state-of-the-art databases. A normal AlexNet based method without transfer learning was also compared. The accuracy of the proposed method was higher in statistical significance than the accuracy of four other state-of-the-art methods, and showed an 18.9% improvement over the normal AlexNet based method. The area under the curve was approximately 0.930 ± 0.020, and the recall rate was 0.929 ± 0.029. An automatic algorithm can assist endoscopists in identifying polyps that are adenomatous by considering a high recall rate and accuracy. This system can enable the timely resection of polyps at an early stage.
Collapse
|
79
|
Application of Artificial Intelligence in Gastrointestinal Endoscopy. J Clin Gastroenterol 2021; 55:110-120. [PMID: 32925304 DOI: 10.1097/mcg.0000000000001423] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/10/2020] [Accepted: 08/07/2020] [Indexed: 12/24/2022]
Abstract
Artificial intelligence (AI), also known as computer-aided diagnosis, is a technology that enables machines to process information and functions at or above human level and has great potential in gastrointestinal endoscopy applications. At present, the research on medical image recognition usually adopts the deep-learning algorithm based on the convolutional neural network. AI has been used in gastrointestinal endoscopy including esophagogastroduodenoscopy, capsule endoscopy, colonoscopy, etc. AI can help endoscopic physicians improve the diagnosis rate of various lesions, reduce the rate of missed diagnosis, improve the quality of endoscopy, assess the severity of the disease, and improve the efficiency of endoscopy. The diversity, susceptibility, and imaging specificity of gastrointestinal endoscopic images are all difficulties and challenges on the road to intelligence. We need more large-scale, high-quality, multicenter prospective studies to explore the clinical applicability of AI, and ethical issues need to be taken into account.
Collapse
|
80
|
Lee KS, Son SH, Park SH, Kim ES. Automated detection of colorectal tumors based on artificial intelligence. BMC Med Inform Decis Mak 2021; 21:33. [PMID: 33522919 PMCID: PMC7849081 DOI: 10.1186/s12911-020-01314-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2020] [Accepted: 10/30/2020] [Indexed: 01/22/2023] Open
Abstract
Background This study developed a diagnostic tool to automatically detect normal, unclear and tumor images from colonoscopy videos using artificial intelligence. Methods For the creation of training and validation sets, 47,555 images in the jpg format were extracted from colonoscopy videos for 24 patients in Korea University Anam Hospital. A gastroenterologist with the clinical experience of 15 years divided the 47,555 images into three classes of Normal (25,895), Unclear (2038) and Tumor (19,622). A single shot detector, a deep learning framework designed for object detection, was trained using the 47,255 images and validated with two sets of 300 images—each validation set included 150 images (50 normal, 50 unclear and 50 tumor cases). Half of the 47,255 images were used for building the model and the other half were used for testing the model. The learning rate of the model was 0.0001 during 250 epochs (training cycles). Results The average accuracy, precision, recall, and F1 score over the category were 0.9067, 0.9744, 0.9067 and 0.9393, respectively. These performance measures had no change with respect to the intersection-over-union threshold (0.45, 0.50, and 0.55). This finding suggests the stability of the model. Conclusion Automated detection of normal, unclear and tumor images from colonoscopy videos is possible by using a deep learning framework. This is expected to provide an invaluable decision supporting system for clinical experts.
Collapse
Affiliation(s)
- Kwang-Sig Lee
- AI Center, Korea University College of Medicine, Seoul, Korea.,Biomedical Research Institute, Korea University College of Medicine, Seoul, Korea
| | - Sang-Hyuk Son
- Storage Solution for Doctors Co., Ltd., Seoul, Korea
| | - Sang-Hyun Park
- Biomedical Research Institute, Korea University College of Medicine, Seoul, Korea
| | - Eun Sun Kim
- Department of Gastroenterology, Korea University College of Medicine, University Anam Hospital, 73 Goryeodae-ro, Seongbuk-gu, Seoul, 02841, Korea.
| |
Collapse
|
81
|
Xie X, Niu J, Liu X, Chen Z, Tang S, Yu S. A survey on incorporating domain knowledge into deep learning for medical image analysis. Med Image Anal 2021; 69:101985. [PMID: 33588117 DOI: 10.1016/j.media.2021.101985] [Citation(s) in RCA: 83] [Impact Index Per Article: 27.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2020] [Revised: 12/04/2020] [Accepted: 01/26/2021] [Indexed: 12/27/2022]
Abstract
Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas they pay particular attention to. In this survey, we summarize the current progress on integrating medical domain knowledge into deep learning models for various tasks, such as disease diagnosis, lesion, organ and abnormality detection, lesion and organ segmentation. For each task, we systematically categorize different kinds of medical domain knowledge that have been utilized and their corresponding integrating methods. We also provide current challenges and directions for future research.
Collapse
Affiliation(s)
- Xiaozheng Xie
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China
| | - Jianwei Niu
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China; Beijing Advanced Innovation Center for Big Data and Brain Computing (BDBC) and Hangzhou Innovation Institute of Beihang University, 18 Chuanghui Street, Binjiang District, Hangzhou 310000, China
| | - Xuefeng Liu
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China.
| | - Zhengsu Chen
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, 37 Xueyuan Road, Haidian District, Beijing 100191, China
| | - Shaojie Tang
- Jindal School of Management, The University of Texas at Dallas, 800 W Campbell Rd, Richardson, TX 75080-3021, USA
| | - Shui Yu
- School of Computer Science, University of Technology Sydney, 15 Broadway, Ultimo NSW 2007, Australia
| |
Collapse
|
82
|
Comparison of deep learning and conventional machine learning methods for classification of colon polyp types. EUROBIOTECH JOURNAL 2021. [DOI: 10.2478/ebtj-2021-0006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Abstract
Determination of polyp types requires tissue biopsy during colonoscopy and then histopathological examination of the microscopic images which tremendously time-consuming and costly. The first aim of this study was to design a computer-aided diagnosis system to classify polyp types using colonoscopy images (optical biopsy) without the need for tissue biopsy. For this purpose, two different approaches were designed based on conventional machine learning (ML) and deep learning. Firstly, classification was performed using random forest approach by means of the features obtained from the histogram of gradients descriptor. Secondly, simple convolutional neural networks (CNN) based architecture was built to train with the colonoscopy images containing colon polyps. The performances of these approaches on two (adenoma & serrated vs. hyperplastic) or three (adenoma vs. hyperplastic vs. serrated) category classifications were investigated. Furthermore, the effect of imaging modality on the classification was also examined using white-light and narrow band imaging systems. The performance of these approaches was compared with the results obtained by 3 novice and 4 expert doctors. Two-category classification results showed that conventional ML approach achieved significantly better than the simple CNN based approach did in both narrow band and white-light imaging modalities. The accuracy reached almost 95% for white-light imaging. This performance surpassed the correct classification rate of all 7 doctors. Additionally, the second task (three-category) results indicated that the simple CNN architecture outperformed both conventional ML based approaches and the doctors. This study shows the feasibility of using conventional machine learning or deep learning based approaches in automatic classification of colon types on colonoscopy images.
Collapse
|
83
|
Automated Classification and Segmentation in Colorectal Images Based on Self-Paced Transfer Network. BIOMED RESEARCH INTERNATIONAL 2021; 2021:6683931. [PMID: 33542924 PMCID: PMC7843175 DOI: 10.1155/2021/6683931] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 12/22/2020] [Accepted: 01/06/2021] [Indexed: 02/08/2023]
Abstract
Colorectal imaging improves on diagnosis of colorectal diseases by providing colorectal images. Manual diagnosis of colorectal disease is labor-intensive and time-consuming. In this paper, we present a method for automatic colorectal disease classification and segmentation. Because of label unbalanced and difficult colorectal data, the classification based on self-paced transfer VGG network (STVGG) is proposed. ImageNet pretraining network parameters are transferred to VGG network with training colorectal data to acquire good initial network performance. And self-paced learning is used to optimize the network so that the classification performance of label unbalanced and difficult samples is improved. In order to assist the colonoscopist to accurately determine whether the polyp needs surgical resection, feature of trained STVGG model is shared to Unet segmentation network as the encoder part and to avoid repeat learning of polyp segmentation model. The experimental results on 3061 colorectal images illustrated that the proposed method obtained higher classification accuracy (96%) and segmentation performance compared with a few other methods. The polyp can be segmented accurately from around tissues by the proposed method. The segmentation results underpin the potential of deep learning methods for assisting colonoscopist in identifying polyps and enabling timely resection of these polyps at an early stage.
Collapse
|
84
|
Lin TH, Jhang JY, Huang CR, Tsai YC, Cheng HC, Sheu BS. Deep Ensemble Feature Network for Gastric Section Classification. IEEE J Biomed Health Inform 2021; 25:77-87. [PMID: 32750926 DOI: 10.1109/jbhi.2020.2999731] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In this paper, we propose a novel deep ensemble feature (DEF) network to classify gastric sections from endoscopic images. Different from recent deep ensemble learning methods, which need to train deep features and classifiers individually to obtain fused classification results, the proposed method can simultaneously learn the deep ensemble feature from arbitrary number of convolutional neural networks (CNNs) and the decision classifier in an end-to-end trainable manner. It comprises two sub networks, the ensemble feature network and the decision network. The former sub network learns the deep ensemble feature from multiple CNNs to represent endoscopic images. The latter sub network learns to obtain the classification labels by using the deep ensemble feature. Both sub networks are optimized based on the proposed ensemble feature loss and the decision loss which guide the learning of deep features and decisions. As shown in the experimental results, the proposed method outperforms the state-of-the-art deep learning, ensemble learning, and deep ensemble learning methods.
Collapse
|
85
|
|
86
|
Golhar M, Bobrow TL, Khoshknab MP, Jit S, Ngamruengphong S, Durr NJ. Improving Colonoscopy Lesion Classification Using Semi-Supervised Deep Learning. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2021; 9:631-640. [PMID: 33747680 PMCID: PMC7978231 DOI: 10.1109/access.2020.3047544] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
While data-driven approaches excel at many image analysis tasks, the performance of these approaches is often limited by a shortage of annotated data available for training. Recent work in semi-supervised learning has shown that meaningful representations of images can be obtained from training with large quantities of unlabeled data, and that these representations can improve the performance of supervised tasks. Here, we demonstrate that an unsupervised jigsaw learning task, in combination with supervised training, results in up to a 9.8% improvement in correctly classifying lesions in colonoscopy images when compared to a fully-supervised baseline. We additionally benchmark improvements in domain adaptation and out-of-distribution detection, and demonstrate that semi-supervised learning outperforms supervised learning in both cases. In colonoscopy applications, these metrics are important given the skill required for endoscopic assessment of lesions, the wide variety of endoscopy systems in use, and the homogeneity that is typical of labeled datasets.
Collapse
Affiliation(s)
- Mayank Golhar
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Taylor L Bobrow
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | | | - Simran Jit
- Division of Gastroenterology and Hepatology, Johns Hopkins Hospital, Baltimore, MD 21287, USA
| | - Saowanee Ngamruengphong
- Division of Gastroenterology and Hepatology, Johns Hopkins Hospital, Baltimore, MD 21287, USA
| | - Nicholas J Durr
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
87
|
Le A, Salifu MO, McFarlane IM. Artificial Intelligence in Colorectal Polyp Detection and Characterization. INTERNATIONAL JOURNAL OF CLINICAL RESEARCH & TRIALS 2021; 6:157. [PMID: 33884326 PMCID: PMC8057724 DOI: 10.15344/2456-8007/2021/157] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
BACKGROUND Over the past 20 years, the advancement of artificial intelligence (AI) and deep learning (DL) has allowed for fast sorting and analysis of large sets of data. In the field of gastroenterology, colorectal screening procedures produces an abundance of data through video and imaging. With AI and DL, this information can be used to create systems where automatic polyp detection and characterization is possible. Convoluted Neural Networks (CNNs) have proven to be an effective way to increase polyp detection and ultimately adenoma detection rates. Different methods of polyp characterization of being hyperplastic vs. adenomatous or non-neoplastic vs. neoplastic has also been investigated showing promising results. FINDINGS The rate of missed polyps on colonoscopy can be as high as 25%. At the beginning of the 2000s, hand-crafted machine learning (ML) algorithms were created and trained retrospectively on colonoscopy images and videos, achieving high sensitivity, specificity, and accuracy of over 90% in many of the studies. Over time, the advancement of DL and CNNs has allowed algorithms to be trained on non-medical images and applied retrospectively to colonoscopy videos and images with similar results. Within the past few years, these algorithms have been applied in real-time colonoscopies and has shown mixed results, one showing no difference while others showing increased polyp detection.Various methods of polyp characterization have also been investigated. Through AI, DL, and CNNs polyps can be identified has hyperplastic/adenomatous or non-neoplastic/neoplastic with high sensitivity, specificity, and accuracy. One of the research areas in polyp characterization is how to capture the polyp image. This paper looks at different modalities of characterizing polyps such as magnifying narrow band imaging (NBI), endocytoscopy, laser-induced florescent spectroscopy, auto-florescent endoscopy, and white-light endoscopy. CONCLUSIONS Overall, much progress has been made in automatic detection and characterization of polyps in real time. Barring ethical or mass adoption setbacks, it is inevitable that AI will be involved in the field of GI, especially in colorectal polyp detection and identification.
Collapse
Affiliation(s)
| | | | - Isabel M. McFarlane
- Corresponding Author: Dr. Isabel M. McFarlane, Clinical Assistant Professor of Medicine, Director, Third Year Internal Medicine Clerkship, Department of Internal Medicine, Brooklyn, NY 11203, USA Tel: 718-270-2390, Fax: 718-270-1324;
| |
Collapse
|
88
|
Sinonquel P, Eelbode T, Bossuyt P, Maes F, Bisschops R. Artificial intelligence and its impact on quality improvement in upper and lower gastrointestinal endoscopy. Dig Endosc 2021; 33:242-253. [PMID: 33145847 DOI: 10.1111/den.13888] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 10/14/2020] [Accepted: 11/01/2020] [Indexed: 12/24/2022]
Abstract
Artificial intelligence (AI) and its application in medicine has grown large interest. Within gastrointestinal (GI) endoscopy, the field of colonoscopy and polyp detection is the most investigated, however, upper GI follows the lead. Since endoscopy is performed by humans, it is inherently an imperfect procedure. Computer-aided diagnosis may improve its quality by helping prevent missing lesions and supporting optical diagnosis for those detected. An entire evolution in AI systems has been established in the last decades, resulting in optimization of the diagnostic performance with lower variability and matching or even outperformance of expert endoscopists. This shows a great potential for future quality improvement of endoscopy, given the outstanding diagnostic features of AI. With this narrative review, we highlight the potential benefit of AI to improve overall quality in daily endoscopy and describe the most recent developments for characterization and diagnosis as well as the recent conditions for regulatory approval.
Collapse
Affiliation(s)
- Pieter Sinonquel
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium.,Departments of, Department of, Translational Research in Gastrointestinal Diseases (TARGID), KU Leuven, Leuven, Belgium
| | - Tom Eelbode
- Medical Imaging Research Center (MIRC), University Hospitals Leuven, Leuven, Belgium.,Department of Electrical Engineering (ESAT/PSI), KU Leuven, Leuven, Belgium
| | - Peter Bossuyt
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium.,Department of Gastroenterology and Hepatology, Imelda Hospital, Bonheiden, Belgium
| | - Frederik Maes
- Medical Imaging Research Center (MIRC), University Hospitals Leuven, Leuven, Belgium.,Department of Electrical Engineering (ESAT/PSI), KU Leuven, Leuven, Belgium
| | - Raf Bisschops
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, Leuven, Belgium.,Departments of, Department of, Translational Research in Gastrointestinal Diseases (TARGID), KU Leuven, Leuven, Belgium
| |
Collapse
|
89
|
Sadad T, Rehman A, Hussain A, Abbasi AA, Khan MQ. A Review on Multi-organ Cancer Detection Using Advanced Machine Learning Techniques. Curr Med Imaging 2020; 17:686-694. [PMID: 33334293 DOI: 10.2174/1573405616666201217112521] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2020] [Revised: 07/07/2020] [Accepted: 07/23/2020] [Indexed: 12/24/2022]
Abstract
Abnormal behaviors of tumors pose a risk to human survival. Thus, the detection of cancers at their initial stage is beneficial for patients and lowers the mortality rate. However, this can be difficult due to various factors related to imaging modalities, such as complex background, low contrast, brightness issues, poorly defined borders and the shape of the affected area. Recently, computer-aided diagnosis (CAD) models have been used to accurately diagnose tumors in different parts of the human body, especially breast, brain, lung, liver, skin and colon cancers. These cancers are diagnosed using various modalities, including computed tomography (CT), magnetic resonance imaging (MRI), colonoscopy, mammography, dermoscopy and histopathology. The aim of this review was to investigate existing approaches for the diagnosis of breast, brain, lung, liver, skin and colon tumors. The review focuses on decision-making systems, including handcrafted features and deep learning architectures for tumor detection.
Collapse
Affiliation(s)
- Tariq Sadad
- Department of Computer Science and Software Engineering, International Islamic University, Islamabad, Pakistan
| | - Amjad Rehman
- Artificial Intelligence & Data Analytics Lab CCIS Prince Sultan University, Riyadh 11586, Saudi Arabia
| | - Ayyaz Hussain
- Department of Computer Science, Quaid-i-Azam University, Islamabad, Pakistan
| | - Aaqif Afzaal Abbasi
- Department of Software Engineering, Foundation University, Islamabad, Pakistan
| | - Muhammad Qasim Khan
- Department of Computer Science, COMSATS University (Attock Campus) Islamabad, Pakistan
| |
Collapse
|
90
|
Owais M, Arsalan M, Mahmood T, Kang JK, Park KR. Automated Diagnosis of Various Gastrointestinal Lesions Using a Deep Learning-Based Classification and Retrieval Framework With a Large Endoscopic Database: Model Development and Validation. J Med Internet Res 2020; 22:e18563. [PMID: 33242010 PMCID: PMC7728528 DOI: 10.2196/18563] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2020] [Revised: 09/16/2020] [Accepted: 11/11/2020] [Indexed: 12/14/2022] Open
Abstract
Background The early diagnosis of various gastrointestinal diseases can lead to effective treatment and reduce the risk of many life-threatening conditions. Unfortunately, various small gastrointestinal lesions are undetectable during early-stage examination by medical experts. In previous studies, various deep learning–based computer-aided diagnosis tools have been used to make a significant contribution to the effective diagnosis and treatment of gastrointestinal diseases. However, most of these methods were designed to detect a limited number of gastrointestinal diseases, such as polyps, tumors, or cancers, in a specific part of the human gastrointestinal tract. Objective This study aimed to develop a comprehensive computer-aided diagnosis tool to assist medical experts in diagnosing various types of gastrointestinal diseases. Methods Our proposed framework comprises a deep learning–based classification network followed by a retrieval method. In the first step, the classification network predicts the disease type for the current medical condition. Then, the retrieval part of the framework shows the relevant cases (endoscopic images) from the previous database. These past cases help the medical expert validate the current computer prediction subjectively, which ultimately results in better diagnosis and treatment. Results All the experiments were performed using 2 endoscopic data sets with a total of 52,471 frames and 37 different classes. The optimal performances obtained by our proposed method in accuracy, F1 score, mean average precision, and mean average recall were 96.19%, 96.99%, 98.18%, and 95.86%, respectively. The overall performance of our proposed diagnostic framework substantially outperformed state-of-the-art methods. Conclusions This study provides a comprehensive computer-aided diagnosis framework for identifying various types of gastrointestinal diseases. The results show the superiority of our proposed method over various other recent methods and illustrate its potential for clinical diagnosis and treatment. Our proposed network can be applicable to other classification domains in medical imaging, such as computed tomography scans, magnetic resonance imaging, and ultrasound sequences.
Collapse
Affiliation(s)
- Muhammad Owais
- Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Republic of Korea
| | - Muhammad Arsalan
- Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Republic of Korea
| | - Tahir Mahmood
- Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Republic of Korea
| | - Jin Kyu Kang
- Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Republic of Korea
| | - Kang Ryoung Park
- Division of Electronics and Electrical Engineering, Dongguk University, Seoul, Republic of Korea
| |
Collapse
|
91
|
Wang Y, Nie H, He X, Liao Z, Zhou Y, Zhou J, Ou C. The emerging role of super enhancer-derived noncoding RNAs in human cancer. Theranostics 2020; 10:11049-11062. [PMID: 33042269 PMCID: PMC7532672 DOI: 10.7150/thno.49168] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2020] [Accepted: 08/23/2020] [Indexed: 02/06/2023] Open
Abstract
Super enhancers (SEs) are large clusters of adjacent enhancers that drive the expression of genes which regulate cellular identity; SE regions can be enriched with a high density of transcription factors, co-factors, and enhancer-associated epigenetic modifications. Through enhanced activation of their target genes, SEs play an important role in various diseases and conditions, including cancer. Recent studies have shown that SEs not only activate the transcriptional expression of coding genes to directly regulate biological functions, but also drive the transcriptional expression of non-coding RNAs (ncRNAs) to indirectly regulate biological functions. SE-derived ncRNAs play critical roles in tumorigenesis, including malignant proliferation, metastasis, drug resistance, and inflammatory response. Moreover, the abnormal expression of SE-derived ncRNAs is closely related to the clinical and pathological characterization of tumors. In this review, we summarize the functions and roles of SE-derived ncRNAs in tumorigenesis and discuss their prospective applications in tumor therapy. A deeper understanding of the potential mechanism underlying the action of SE-derived ncRNAs in tumorigenesis may provide new strategies for the early diagnosis of tumors and targeted therapy.
Collapse
|
92
|
Pacal I, Karaboga D, Basturk A, Akay B, Nalbantoglu U. A comprehensive review of deep learning in colon cancer. Comput Biol Med 2020; 126:104003. [DOI: 10.1016/j.compbiomed.2020.104003] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2020] [Revised: 08/28/2020] [Accepted: 08/28/2020] [Indexed: 12/17/2022]
|
93
|
Wimmer G, Häfner M, Uhl A. Improving CNN training on endoscopic image data by extracting additionally training data from endoscopic videos. Comput Med Imaging Graph 2020; 86:101798. [PMID: 33075676 DOI: 10.1016/j.compmedimag.2020.101798] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Revised: 06/23/2020] [Accepted: 09/24/2020] [Indexed: 02/07/2023]
Abstract
In this work we present a technique to deal with one of the biggest problems for the application of convolutional neural networks (CNNs) in the area of computer assisted endoscopic image diagnosis, the insufficient amount of training data. Based on patches from endoscopic images of colonic polyps with given label information, our proposed technique acquires additional (labeled) training data by tracking the area shown in the patches through the corresponding endoscopic videos and by extracting additional image patches from frames of these areas. So similar to the widely used augmentation strategies, additional training data is produced by adding images with different orientations, scales and points of view than the original images. However, contrary to augmentation techniques, we do not artificially produce image data but use real image data from videos under different image recording conditions (different viewpoints and image qualities). By means of our proposed method and by filtering out all extracted images with insufficient image quality, we are able to increase the amount of labeled image data by factor 39. We will show that our proposed method clearly and continuously improves the performance of CNNs.
Collapse
Affiliation(s)
- Georg Wimmer
- University of Salzburg, Department of Computer Sciences, Jakob-Haringerstrasse 2, Salzburg 5020, Austria.
| | - Michael Häfner
- Department of Gastroenterologie and Hepatologie, St. Elisabeth Hospital, Landstraßer Hauptstraße 4a, Wien A-1030, Austria
| | - Andreas Uhl
- University of Salzburg, Department of Computer Sciences, Jakob-Haringerstrasse 2, Salzburg 5020, Austria
| |
Collapse
|
94
|
Attardo S, Chandrasekar VT, Spadaccini M, Maselli R, Patel HK, Desai M, Capogreco A, Badalamenti M, Galtieri PA, Pellegatta G, Fugazza A, Carrara S, Anderloni A, Occhipinti P, Hassan C, Sharma P, Repici A. Artificial intelligence technologies for the detection of colorectal lesions: The future is now. World J Gastroenterol 2020; 26:5606-5616. [PMID: 33088155 PMCID: PMC7545398 DOI: 10.3748/wjg.v26.i37.5606] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Revised: 06/30/2020] [Accepted: 09/15/2020] [Indexed: 02/06/2023] Open
Abstract
Several studies have shown a significant adenoma miss rate up to 35% during screening colonoscopy, especially in patients with diminutive adenomas. The use of artificial intelligence (AI) in colonoscopy has been gaining popularity by helping endoscopists in polyp detection, with the aim to increase their adenoma detection rate (ADR) and polyp detection rate (PDR) in order to reduce the incidence of interval cancers. The efficacy of deep convolutional neural network (DCNN)-based AI system for polyp detection has been trained and tested in ex vivo settings such as colonoscopy still images or videos. Recent trials have evaluated the real-time efficacy of DCNN-based systems showing promising results in term of improved ADR and PDR. In this review we reported data from the preliminary ex vivo experiences and summarized the results of the initial randomized controlled trials.
Collapse
Affiliation(s)
- Simona Attardo
- Department of Endoscopy and Digestive Disease, AOU Maggiore della Carità, Novara 28100, Italy
| | | | - Marco Spadaccini
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
- Department of Biomedical Sciences, Humanitas University, Rozzano 20089, Italy
| | - Roberta Maselli
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
| | - Harsh K Patel
- Department of Internal Medicine, Ochsner Clinic Foundation, New Orleans, LA 70124, United States
| | - Madhav Desai
- Department of Gastroenterology and Hepatology, Kansas City VA Medical Center, Kansas City, MO 66045, United States
| | - Antonio Capogreco
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
- Department of Biomedical Sciences, Humanitas University, Rozzano 20089, Italy
| | - Matteo Badalamenti
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
| | | | - Gaia Pellegatta
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
| | - Alessandro Fugazza
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
| | - Silvia Carrara
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
| | - Andrea Anderloni
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
| | - Pietro Occhipinti
- Department of Endoscopy and Digestive Disease, AOU Maggiore della Carità, Novara 28100, Italy
| | - Cesare Hassan
- Endoscopy Unit, Nuovo Regina Margherita Hospital, Roma 00153, Italy
| | - Prateek Sharma
- Department of Gastroenterology and Hepatology, Kansas City VA Medical Center, Kansas City, MO 66045, United States
| | - Alessandro Repici
- Department of Endoscopy, Humanitas Research Hospital, Rozzano 20089, Italy
- Department of Biomedical Sciences, Humanitas University, Rozzano 20089, Italy
| |
Collapse
|
95
|
Sierra F, Gutierrez Y, Martinez F. An online deep convolutional polyp lesion prediction over Narrow Band Imaging (NBI). ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:2412-2415. [PMID: 33018493 DOI: 10.1109/embc44109.2020.9176534] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Polyps, represented as abnormal protuberances along intestinal track, are the main biomarker to diagnose gastrointestinal cancer. During routine colonoscopies such polyps are localized and coarsely characterized according to microvascular and surface textural patterns. Narrow-band imaging (NBI) sequences have emerged as complementary technique to enhance description of suspicious mucosa surfaces according to blood vessels architectures. Nevertheless, a high number of misleading polyp characterization, together with expert dependency during evaluation, reduce the possibility of effective disease treatments. Additionally, challenges during colonoscopy, such as abrupt camera motions, changes of intensity and artifacts, difficult the diagnosis task. This work introduces a robust frame-level convolutional strategy with the capability to characterize and predict hyperplastic, adenomas and serrated polyps over NBI sequences. The proposed strategy was evaluated over a total of 76 videos achieving an average accuracy of 90,79% to distinguish among these three classes. Remarkably, the approach achieves a 100% of accuracy to differentiate intermediate serrated polyps, whose evaluation is challenging even for expert gastroenterologist. The approach was also favorable to support polyp resection decisions, achieving perfect score on evaluated dataset.Clinical relevance- The proposed approach supports observable hystological characterization of polyps during a routine colonoscopy avoiding misclassification of potential masses that could evolve in cancer.
Collapse
|
96
|
Laiz P, Vitrià J, Wenzek H, Malagelada C, Azpiroz F, Seguí S. WCE polyp detection with triplet based embeddings. Comput Med Imaging Graph 2020; 86:101794. [PMID: 33130417 DOI: 10.1016/j.compmedimag.2020.101794] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Revised: 09/17/2020] [Accepted: 09/18/2020] [Indexed: 12/20/2022]
Abstract
Wireless capsule endoscopy is a medical procedure used to visualize the entire gastrointestinal tract and to diagnose intestinal conditions, such as polyps or bleeding. Current analyses are performed by manually inspecting nearly each one of the frames of the video, a tedious and error-prone task. Automatic image analysis methods can be used to reduce the time needed for physicians to evaluate a capsule endoscopy video. However these methods are still in a research phase. In this paper we focus on computer-aided polyp detection in capsule endoscopy images. This is a challenging problem because of the diversity of polyp appearance, the imbalanced dataset structure and the scarcity of data. We have developed a new polyp computer-aided decision system that combines a deep convolutional neural network and metric learning. The key point of the method is the use of the Triplet Loss function with the aim of improving feature extraction from the images when having small dataset. The Triplet Loss function allows to train robust detectors by forcing images from the same category to be represented by similar embedding vectors while ensuring that images from different categories are represented by dissimilar vectors. Empirical results show a meaningful increase of AUC values compared to state-of-the-art methods. A good performance is not the only requirement when considering the adoption of this technology to clinical practice. Trust and explainability of decisions are as important as performance. With this purpose, we also provide a method to generate visual explanations of the outcome of our polyp detector. These explanations can be used to build a physician's trust in the system and also to convey information about the inner working of the method to the designer for debugging purposes.
Collapse
Affiliation(s)
- Pablo Laiz
- Department of Mathematics and Computer Science, Universitat de Barcelona, Barcelona, Spain.
| | - Jordi Vitrià
- Department of Mathematics and Computer Science, Universitat de Barcelona, Barcelona, Spain
| | | | - Carolina Malagelada
- Digestive System Research Unit, University Hospital Vall d'Hebron, Barcelona, Spain
| | - Fernando Azpiroz
- Digestive System Research Unit, University Hospital Vall d'Hebron, Barcelona, Spain
| | - Santi Seguí
- Department of Mathematics and Computer Science, Universitat de Barcelona, Barcelona, Spain
| |
Collapse
|
97
|
Wang KW, Dong M. Potential applications of artificial intelligence in colorectal polyps and cancer: Recent advances and prospects. World J Gastroenterol 2020; 26:5090-5100. [PMID: 32982111 PMCID: PMC7495038 DOI: 10.3748/wjg.v26.i34.5090] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 07/01/2020] [Accepted: 08/12/2020] [Indexed: 02/06/2023] Open
Abstract
Since the advent of artificial intelligence (AI) technology, it has been constantly studied and has achieved rapid development. The AI assistant system is expected to improve the quality of automatic polyp detection and classification. It could also help prevent endoscopists from missing polyps and make an accurate optical diagnosis. These functions provided by AI could result in a higher adenoma detection rate and decrease the cost of polypectomy for hyperplastic polyps. In addition, AI has good performance in the staging, diagnosis, and segmentation of colorectal cancer. This article provides an overview of recent research focusing on the application of AI in colorectal polyps and cancer and highlights the advances achieved.
Collapse
Affiliation(s)
- Ke-Wei Wang
- Department of Gastrointestinal Surgery, the First Affiliated Hospital of China Medical University, Shenyang 110001, Liaoning Province, China
| | - Ming Dong
- Department of Gastrointestinal Surgery, the First Affiliated Hospital of China Medical University, Shenyang 110001, Liaoning Province, China
| |
Collapse
|
98
|
Application of Deep Learning for Early Screening of Colorectal Precancerous Lesions under White Light Endoscopy. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2020; 2020:8374317. [PMID: 32952602 PMCID: PMC7480430 DOI: 10.1155/2020/8374317] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Accepted: 06/30/2020] [Indexed: 12/27/2022]
Abstract
Methods We collected and sorted out the white light endoscopic images of some patients undergoing colonoscopy. The convolutional neural network model is used to detect whether the image contains lesions: CRC, colorectal adenoma (CRA), and colorectal polyps. The accuracy, sensitivity, and specificity rates are used as indicators to evaluate the model. Then, the instance segmentation model is used to locate and classify the lesions on the images containing lesions, and mAP (mean average precision), AP50, and AP75 are used to evaluate the performance of an instance segmentation model. Results In the process of detecting whether the image contains lesions, we compared ResNet50 with the other four models, that is, AlexNet, VGG19, ResNet18, and GoogLeNet. The result is that ResNet50 performs better than several other models. It scored an accuracy of 93.0%, a sensitivity of 94.3%, and a specificity of 90.6%. In the process of localization and classification of the lesion in images containing lesions by Mask R-CNN, its mAP, AP50, and AP75 were 0.676, 0.903, and 0.833, respectively. Conclusion We developed and compared five models for the detection of lesions in white light endoscopic images. ResNet50 showed the optimal performance, and Mask R-CNN model could be used to locate and classify lesions in images containing lesions.
Collapse
|
99
|
Meng J, Xue L, Chang Y, Zhang J, Chang S, Liu K, Liu S, Wang B, Yang K. Automatic detection and segmentation of adenomatous colorectal polyps during colonoscopy using Mask R-CNN. Open Life Sci 2020; 15:588-596. [PMID: 33817247 PMCID: PMC7968546 DOI: 10.1515/biol-2020-0055] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2020] [Revised: 05/27/2020] [Accepted: 06/03/2020] [Indexed: 12/27/2022] Open
Abstract
Colorectal cancer (CRC) is one of the main alimentary tract system malignancies affecting people worldwide. Adenomatous polyps are precursors of CRC, and therefore, preventing the development of these lesions may also prevent subsequent malignancy. However, the adenoma detection rate (ADR), a measure of the ability of a colonoscopist to identify and remove precancerous colorectal polyps, varies significantly among endoscopists. Here, we attempt to use a convolutional neural network (CNN) to generate a unique computer-aided diagnosis (CAD) system by exploring in detail the multiple-scale performance of deep neural networks. We applied this system to 3,375 hand-labeled images from the screening colonoscopies of 1,197 patients; of whom, 3,045 were assigned to the training dataset and 330 to the testing dataset. The images were diagnosed simply as either an adenomatous or non-adenomatous polyp. When applied to the testing dataset, our CNN-CAD system achieved a mean average precision of 89.5%. We conclude that the proposed framework could increase the ADR and decrease the incidence of interval CRCs, although further validation through large multicenter trials is required.
Collapse
Affiliation(s)
- Jie Meng
- Department of Gastroenterology and Hepatology, Tianjin Medical University General Hospital, Tianjin 300052, China.,Department of Gastroenterology, Affiliated Hospital of Hebei University, Baoding 071000, China
| | - Linyan Xue
- College of Quality and Technical Supervision, Hebei University, Baoding 071002, China
| | - Ying Chang
- Department of Gastroenterology, Affiliated Hospital of Hebei University, Baoding 071000, China
| | - Jianguang Zhang
- Department of Gastroenterology, Affiliated Hospital of Hebei University, Baoding 071000, China
| | - Shilong Chang
- College of Quality and Technical Supervision, Hebei University, Baoding 071002, China
| | - Kun Liu
- College of Quality and Technical Supervision, Hebei University, Baoding 071002, China
| | - Shuang Liu
- College of Quality and Technical Supervision, Hebei University, Baoding 071002, China
| | - Bangmao Wang
- Department of Gastroenterology and Hepatology, Tianjin Medical University General Hospital, Tianjin 300052, China
| | - Kun Yang
- College of Quality and Technical Supervision, Hebei University, Baoding 071002, China
| |
Collapse
|
100
|
Kang H, Xia L, Yan F, Wan Z, Shi F, Yuan H, Jiang H, Wu D, Sui H, Zhang C, Shen D. Diagnosis of Coronavirus Disease 2019 (COVID-19) With Structured Latent Multi-View Representation Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:2606-2614. [PMID: 32386147 DOI: 10.1109/tmi.2020.2992546] [Citation(s) in RCA: 113] [Impact Index Per Article: 28.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/19/2023]
Abstract
Recently, the outbreak of Coronavirus Disease 2019 (COVID-19) has spread rapidly across the world. Due to the large number of infected patients and heavy labor for doctors, computer-aided diagnosis with machine learning algorithm is urgently needed, and could largely reduce the efforts of clinicians and accelerate the diagnosis process. Chest computed tomography (CT) has been recognized as an informative tool for diagnosis of the disease. In this study, we propose to conduct the diagnosis of COVID-19 with a series of features extracted from CT images. To fully explore multiple features describing CT images from different views, a unified latent representation is learned which can completely encode information from different aspects of features and is endowed with promising class structure for separability. Specifically, the completeness is guaranteed with a group of backward neural networks (each for one type of features), while by using class labels the representation is enforced to be compact within COVID-19/community-acquired pneumonia (CAP) and also a large margin is guaranteed between different types of pneumonia. In this way, our model can well avoid overfitting compared to the case of directly projecting high-dimensional features into classes. Extensive experimental results show that the proposed method outperforms all comparison methods, and rather stable performances are observed when varying the number of training data.
Collapse
|