Use of Endoscopic Impression, Artificial Intelligence, and Pathologist Interpretation to Resolve Discrepancies Between Endoscopy and Pathology Analyses of Diminutive Colorectal Polyps

An artificial intelligence-assisted system versus white light endoscopy alone for adenoma detection in individuals with Lynch syndrome (TIMELY): an international, multicentre, randomised controlled trial

BACKGROUND

Computer-aided detection (CADe) systems for colonoscopy have been shown to increase small polyp detection during colonoscopy in the general population. People with Lynch syndrome represent an ideal target population for CADe-assisted colonoscopy because adenomas, the primary cancer precursor lesions, are characterised by their small size and higher likelihood of showing advanced histology. We aimed to evaluate the performance of CADe-assisted colonoscopy in detecting adenomas in individuals with Lynch syndrome.

METHODS

TIMELY was an international, multicentre, parallel, randomised controlled trial done in 11 academic centres and six community centres in Belgium, Germany, Italy, and Spain. We enrolled individuals aged 18 years or older with pathogenic or likely pathogenic MLH1, MSH2, MSH6, or EPCAM variants. Participants were consecutively randomly assigned (1:1) to either CADe (GI Genius) assisted white light endoscopy (WLE) or WLE alone. A centre-stratified randomisation sequence was generated through a computer-generated system with a separate randomisation list for each centre according to block-permuted randomisation (block size 26 patients per centre). Allocation was automatically provided by the online AEG-REDCap database. Participants were masked to the random assignment but endoscopists were not. The primary outcome was the mean number of adenomas per colonoscopy, calculated by dividing the total number of adenomas detected by the total number of colonoscopies and assessed in the intention-to-treat population. This trial is registered with ClinicalTrials.gov, NCT04909671.

FINDINGS

Between Sept 13, 2021, and April 6, 2023, 456 participants were screened for eligibility, 430 of whom were randomly assigned to receive CADe-assisted colonoscopy (n=214) or WLE (n=216). 256 (60%) participants were female and 174 (40%) were male. In the intention-to-treat analysis, the mean number of adenomas per colonoscopy was 0·64 (SD 1·57) in the CADe group and 0·64 (1·17) in the WLE group (adjusted rate ratio 1·03 [95% CI 0·72-1·47); p=0·87). No adverse events were reported during the trial.

INTERPRETATION

In this multicentre international trial, CADe did not improve the detection of adenomas in individuals with Lynch syndrome. High-quality procedures and thorough inspection and exposure of the colonic mucosa remain the cornerstone in surveillance of Lynch syndrome.

FUNDING

Spanish Gastroenterology Association, Spanish Society of Digestive Endoscopy, European Society of Gastrointestinal Endoscopy, Societat Catalana de Digestologia, Instituto Carlos III, Beca de la Marato de TV3 2020. Co-funded by the European Union.

Computer-aided diagnosis system for optical diagnosis of colorectal polyps under white light imaging

OBJECTIVES

This study presents a novel computer-aided diagnosis (CADx) designed for optically diagnosing colorectal polyps using white light imaging (WLI).We aimed to evaluate the effectiveness of the CADx and its auxiliary role among endoscopists with different levels of expertise.

METHODS

We collected 2,324 neoplastic and 3,735 nonneoplastic polyp WLI images for model training, and 838 colorectal polyp images from 740 patients for model validation. We compared the diagnostic accuracy of the CADx with that of 15 endoscopists under WLI and narrow band imaging (NBI). The auxiliary benefits of CADx for endoscopists of different experience levels and for identifying different types of colorectal polyps was also evaluated.

RESULTS

The CADx demonstrated an optical diagnostic accuracy of 84.49%, showing considerable superiority over all endoscopists, irrespective of whether WLI or NBI was used (P < 0.001). Assistance from the CADx significantly improved the diagnostic accuracy of the endoscopists from 68.84% to 77.49% (P = 0.001), with the most significant impact observed among novice endoscopists. Notably, novices using CADx-assisted WLI outperform junior and expert endoscopists without such assistance.

CONCLUSIONS

The CADx demonstrated a crucial role in substantially enhancing the precision of optical diagnosis for colorectal polyps under WLI and showed the greatest auxiliary benefits for novice endoscopists.

Using Computer-aided Optical Diagnosis and Expert Review to Evaluate Colorectal Polyps Diagnosed as Normal Mucosa in Pathology

Pathological assessment of colorectal polyps is considered the current reference standard for histologic diagnosis. About 10% of polyps sent to the pathology lab are returned with the diagnosis of mucosal folds, mucosal prolapse, or normal mucosa.1,2 Two recent publications have indicated that disagreements between endoscopic optical diagnosis and the subsequent pathological diagnoses might be due to misdiagnosis in pathology.3,4 We were therefore interested in re-evaluating pathology-based diagnosis of "mucosal polyps" using expert endoscopists and computer-assisted diagnosis (CADx) evaluation.

The Role of Artificial Intelligence in Colorectal Cancer Screening: Lesion Detection and Lesion Characterization

Colorectal cancer remains a leading cause of cancer-related morbidity and mortality worldwide, despite the widespread uptake of population surveillance strategies. This is in part due to the persistent development of 'interval colorectal cancers', where patients develop colorectal cancer despite appropriate surveillance intervals, implying pre-malignant polyps were not resected at a prior colonoscopy. Multiple techniques have been developed to improve the sensitivity and accuracy of lesion detection and characterisation in an effort to improve the efficacy of colorectal cancer screening, thereby reducing the incidence of interval colorectal cancers. This article presents a comprehensive review of the transformative role of artificial intelligence (AI), which has recently emerged as one such solution for improving the quality of screening and surveillance colonoscopy. Firstly, AI-driven algorithms demonstrate remarkable potential in addressing the challenge of overlooked polyps, particularly polyp subtypes infamous for escaping human detection because of their inconspicuous appearance. Secondly, AI empowers gastroenterologists without exhaustive training in advanced mucosal imaging to characterise polyps with accuracy similar to that of expert interventionalists, reducing the dependence on pathologic evaluation and guiding appropriate resection techniques or referrals for more complex resections. AI in colonoscopy holds the potential to advance the detection and characterisation of polyps, addressing current limitations and improving patient outcomes. The integration of AI technologies into routine colonoscopy represents a promising step towards more effective colorectal cancer screening and prevention.

The Role of Artificial Intelligence in Prospective Real-Time Histological Prediction of Colorectal Lesions during Colonoscopy: A Systematic Review and Meta-Analysis

Artificial intelligence (AI) presents a novel platform for improving disease diagnosis. However, the clinical utility of AI remains limited to discovery studies, with poor translation to clinical practice. Current data suggests that 26% of diminutive pre-malignant lesions and 3.5% of colorectal cancers are missed during colonoscopies. The primary aim of this study was to explore the role of artificial intelligence in real-time histological prediction of colorectal lesions during colonoscopy. A systematic search using MeSH headings relating to "AI", "machine learning", "computer-aided", "colonoscopy", and "colon/rectum/colorectal" identified 2290 studies. Thirteen studies reporting real-time analysis were included. A total of 2958 patients with 5908 colorectal lesions were included. A meta-analysis of six studies reporting sensitivities (95% CI) demonstrated that endoscopist diagnosis was superior to a computer-assisted detection platform, although no statistical significance was reached (p = 0.43). AI applications have shown encouraging results in differentiating neoplastic and non-neoplastic lesions using narrow-band imaging, white light imaging, and blue light imaging. Other modalities include autofluorescence imaging and elastic scattering microscopy. The current literature demonstrates that despite the promise of new endoscopic AI models, they remain inferior to expert endoscopist diagnosis. There is a need to focus developments on real-time histological predictions prior to clinical translation to demonstrate improved diagnostic capabilities and time efficiency.

Varied Trainee Competence in Cold Snare Polypectomy: Results of the COMPLETE Randomized Controlled Trial

INTRODUCTION

Cold snare polypectomy (CSP) is strongly recommended as the optimal technique for the complete removal of small polyps. Though significant variability in polypectomy technique and quality has been established, the learning curve and impact of targeted training on CSP are unknown. Video feedback has shown promise as an effective pedagogy to improve performance among surgical trainees. We aimed to compare CSP performance between trainees who received video-based feedback and those who received conventional apprentice-based concurrent feedback. We hypothesized that video-based feedback would accelerate competence.

METHODS

We conducted a single-blinded, randomized controlled trial on competence for CSP of polyps <1 cm, comparing video-based feedback with conventional feedback. We randomly assigned deidentified consecutively recorded CSP videos to blinded raters to assess using the CSP Assessment Tool. We shared cumulative sum learning curves every 25 CSP with each trainee. The video feedback trainees also received biweekly individualized terminal feedback. Control trainees received conventional feedback during colonoscopy. The primary outcome was CSP competence. We also assessed competence across domains and change over polypectomy volume.

RESULTS

We enrolled and randomized 22 trainees, 12 to video-based feedback and 10 to conventional feedback, and evaluated 2,339 CSP. The learning curve was long; 2 trainees (16.7%) in the video feedback achieved competence, after a mean of 135 polyps, and no one in the control ( P = 0.481) achieved competence. Overall and in all steps of CSP, a higher percentage of the video feedback group met competence, increasing 3% every 20 CSP ( P = 0.0004).

DISCUSSION

Video feedback aided trainees to competence in CSP. However, the learning curve was long. Our findings strongly suggest that current training methods are not sufficient to support trainees to competency by the completion of their fellowship programs. The impact of new training methods, such as simulation-based mastery learning, should be assessed to determine whether such methods can result in achievement of competence at a faster rate; ClinicalTrials.gov : NCT03115008.

Spatio-temporal classification for polyp diagnosis

Colonoscopy remains the gold standard investigation for colorectal cancer screening as it offers the opportunity to both detect and resect pre-cancerous polyps. Computer-aided polyp characterisation can determine which polyps need polypectomy and recent deep learning-based approaches have shown promising results as clinical decision support tools. Yet polyp appearance during a procedure can vary, making automatic predictions unstable. In this paper, we investigate the use of spatio-temporal information to improve the performance of lesions classification as adenoma or non-adenoma. Two methods are implemented showing an increase in performance and robustness during extensive experiments both on internal and openly available benchmark datasets.

Detection of duodenal villous atrophy on endoscopic images using a deep learning algorithm

BACKGROUND AND AIMS

Celiac disease with its endoscopic manifestation of villous atrophy (VA) is underdiagnosed worldwide. The application of artificial intelligence (AI) for the macroscopic detection of VA at routine EGD may improve diagnostic performance.

METHODS

A dataset of 858 endoscopic images of 182 patients with VA and 846 images from 323 patients with normal duodenal mucosa was collected and used to train a ResNet18 deep learning model to detect VA. An external dataset was used to test the algorithm, in addition to 6 fellows and 4 board-certified gastroenterologists. Fellows could consult the AI algorithm's result during the test. From their consultation distribution, a stratification of test images into "easy" and "difficult" was performed and used for classified performance measurement.

RESULTS

External validation of the AI algorithm yielded values of 90%, 76%, and 84% for sensitivity, specificity, and accuracy, respectively. Fellows scored corresponding values of 63%, 72%, and 67% and experts scored 72%, 69%, and 71%, respectively. AI consultation significantly improved all trainee performance statistics. Although fellows and experts showed significantly lower performance for difficult images, the performance of the AI algorithm was stable.

CONCLUSIONS

In this study, an AI algorithm outperformed endoscopy fellows and experts in the detection of VA on endoscopic still images. AI decision support significantly improved the performance of nonexpert endoscopists. The stable performance on difficult images suggests a further positive add-on effect in challenging cases.

Improving quality control in the routine practice for histopathological interpretation of gastrointestinal endoscopic biopsies using artificial intelligence

BACKGROUND

Colorectal and gastric cancer are major causes of cancer-related deaths. In Korea, gastrointestinal (GI) endoscopic biopsy specimens account for a high percentage of histopathologic examinations. Lack of a sufficient pathologist workforce can cause an increase in human errors, threatening patient safety. Therefore, we developed a digital pathology total solution combining artificial intelligence (AI) classifier models and pathology laboratory information system for GI endoscopic biopsy specimens to establish a post-analytic daily fast quality control (QC) system, which was applied in clinical practice for a 3-month trial run by four pathologists.

METHODS AND FINDINGS

Our whole slide image (WSI) classification framework comprised patch-generator, patch-level classifier, and WSI-level classifier. The classifiers were both based on DenseNet (Dense Convolutional Network). In laboratory tests, the WSI classifier achieved accuracy rates of 95.8% and 96.0% in classifying histopathological WSIs of colorectal and gastric endoscopic biopsy specimens, respectively, into three classes (Negative for dysplasia, Dysplasia, and Malignant). Classification by pathologic diagnosis and AI prediction were compared and daily reviews were conducted, focusing on discordant cases for early detection of potential human errors by the pathologists, allowing immediate correction, before the pathology report error is conveyed to the patients. During the 3-month AI-assisted daily QC trial run period, approximately 7-10 times the number of slides compared to that in the conventional monthly QC (33 months) were reviewed by pathologists; nearly 100% of GI endoscopy biopsy slides were double-checked by the AI models. Further, approximately 17-30 times the number of potential human errors were detected within an average of 1.2 days.

CONCLUSIONS

The AI-assisted daily QC system that we developed and established demonstrated notable improvements in QC, in quantitative, qualitative, and time utility aspects. Ultimately, we developed an independent AI-assisted post-analytic daily fast QC system that was clinically applicable and influential, which could enhance patient safety.

Performance and comparison of artificial intelligence and human experts in the detection and classification of colonic polyps

OBJECTIVE

The main aim of this study was to analyze the performance of different artificial intelligence (AI) models in endoscopic colonic polyp detection and classification and compare them with doctors with different experience.

METHODS

We searched the studies on Colonoscopy, Colonic Polyps, Artificial Intelligence, Machine Learning, and Deep Learning published before May 2020 in PubMed, EMBASE, Cochrane, and the citation index of the conference proceedings. The quality of studies was assessed using the QUADAS-2 table of diagnostic test quality evaluation criteria. The random-effects model was calculated using Meta-DISC 1.4 and RevMan 5.3.

RESULTS

A total of 16 studies were included for meta-analysis. Only one study (1/16) presented externally validated results. The area under the curve (AUC) of AI group, expert group and non-expert group for detection and classification of colonic polyps were 0.940, 0.918, and 0.871, respectively. AI group had slightly lower pooled specificity than the expert group (79% vs. 86%, P < 0.05), but the pooled sensitivity was higher than the expert group (88% vs. 80%, P < 0.05). While the non-experts had less pooled specificity in polyp recognition than the experts (81% vs. 86%, P < 0.05), and higher pooled sensitivity than the experts (85% vs. 80%, P < 0.05).

CONCLUSION

The performance of AI in polyp detection and classification is similar to that of human experts, with high sensitivity and moderate specificity. Different tasks may have an impact on the performance of deep learning models and human experts, especially in terms of sensitivity and specificity.

Vessel and tissue recognition during third-space endoscopy using a deep learning algorithm

In this study, we aimed to develop an artificial intelligence clinical decision support solution to mitigate operator-dependent limitations during complex endoscopic procedures such as endoscopic submucosal dissection and peroral endoscopic myotomy, for example, bleeding and perforation. A DeepLabv3-based model was trained to delineate vessels, tissue structures and instruments on endoscopic still images from such procedures. The mean cross-validated Intersection over Union and Dice Score were 63% and 76%, respectively. Applied to standardised video clips from third-space endoscopic procedures, the algorithm showed a mean vessel detection rate of 85% with a false-positive rate of 0.75/min. These performance statistics suggest a potential clinical benefit for procedure safety, time and also training.

What size cutoff level should be used to implement optical polyp diagnosis?

BACKGROUND : The risk of advanced pathology increases with polyp size, as does the potential for mismanagement when optical diagnosis is used. This study aimed to evaluate the proportion of patients who would be assigned inadequate surveillance intervals when different size cutoffs are adopted for use of optical diagnosis. METHODS : In a post hoc analysis of three prospective studies, the use of optical diagnosis was evaluated for three polyp size groups: 1-3, 1-5, and 1-10 mm. The primary outcome was the proportion of patients in whom advanced adenomas were found and optical diagnosis resulted in delayed surveillance. Secondary outcomes included agreements between surveillance intervals based on high confidence optical diagnosis and pathology outcomes, reduction in histopathological examinations, and proportion of patients who could receive an immediate surveillance recommendation. RESULTS : We included 3374 patients (7291 polyps ≤ 10 mm) undergoing complete colonoscopies (median age 66.0 years, 75.2 % male, 29.6 % for screening). The percentage of patients with advanced adenomas and either 2- or 7-year delayed surveillance intervals (n = 79) was 3.8 %, 15.2 %, and 25.3 % for size cutoffs of 1-3, 1-5, and 1-10 mm polyps, respectively (P < 0.001). Surveillance interval agreements between pathology and optical diagnosis for the three groups were 97.2 %, 95.5 %, and 94.2 %, respectively. Total reductions in pathology examinations for the three groups were 33.5 %, 62.3 %, and 78.2 %, respectively. CONCLUSION : A 3-mm cutoff for clinical implementation of optical diagnosis resulted in a very low risk of delayed management of advanced neoplasia while showing high surveillance interval agreement with pathology and a one-third reduction in overall requirement for pathology examinations.

A novel AI device for real-time optical characterization of colorectal polyps

Accurate in-vivo optical characterization of colorectal polyps is key to select the optimal treatment regimen during colonoscopy. However, reported accuracies vary widely among endoscopists. We developed a novel intelligent medical device able to seamlessly operate in real-time using conventional white light (WL) endoscopy video stream without virtual chromoendoscopy (blue light, BL). In this work, we evaluated the standalone performance of this computer-aided diagnosis device (CADx) on a prospectively acquired dataset of unaltered colonoscopy videos. An international group of endoscopists performed optical characterization of each polyp acquired in a prospective study, blinded to both histology and CADx result, by means of an online platform enabling careful video assessment. Colorectal polyps were categorized by reviewers, subdivided into 10 experts and 11 non-experts endoscopists, and by the CADx as either “adenoma” or “non-adenoma”. A total of 513 polyps from 165 patients were assessed. CADx accuracy in WL was found comparable to the accuracy of expert endoscopists (CADx_WL/Exp; OR 1.211 [0.766–1.915]) using histopathology as the reference standard. Moreover, CADx accuracy in WL was found superior to the accuracy of non-expert endoscopists (CADx_WL/NonExp; OR 1.875 [1.191–2.953]), and CADx accuracy in BL was found comparable to it (CADx_BL/CADx_WL; OR 0.886 [0.612–1.282]). The proposed intelligent device shows the potential to support non-expert endoscopists in systematically reaching the performances of expert endoscopists in optical characterization.

Development of artificial intelligence technology in diagnosis, treatment, and prognosis of colorectal cancer

Artificial intelligence (AI) technology has made leaps and bounds since its invention. AI technology can be subdivided into many technologies such as machine learning and deep learning. The application scope and prospect of different technologies are also totally different. Currently, AI technologies play a pivotal role in the highly complex and wide-ranging medical field, such as medical image recognition, biotechnology, auxiliary diagnosis, drug research and development, and nutrition. Colorectal cancer (CRC) is a common gastrointestinal cancer that has a high mortality, posing a serious threat to human health. Many CRCs are caused by the malignant transformation of colorectal polyps. Therefore, early diagnosis and treatment are crucial to CRC prognosis. The methods of diagnosing CRC are divided into imaging diagnosis, endoscopy, and pathology diagnosis. Treatment methods are divided into endoscopic treatment, surgical treatment, and drug treatment. AI technology is in the weak era and does not have communication capabilities. Therefore, the current AI technology is mainly used for image recognition and auxiliary analysis without in-depth communication with patients. This article reviews the application of AI in the diagnosis, treatment, and prognosis of CRC and provides the prospects for the broader application of AI in CRC.

Deep learning driven colorectal lesion detection in gastrointestinal endoscopic and pathological imaging

Colorectal cancer has the second highest incidence of malignant tumors and is the fourth leading cause of cancer deaths in China. Early diagnosis and treatment of colorectal cancer will lead to an improvement in the 5-year survival rate, which will reduce medical costs. The current diagnostic methods for early colorectal cancer include excreta, blood, endoscopy, and computer-aided endoscopy. In this paper, research on image analysis and prediction of colorectal cancer lesions based on deep learning is reviewed with the goal of providing a reference for the early diagnosis of colorectal cancer lesions by combining computer technology, 3D modeling, 5G remote technology, endoscopic robot technology, and surgical navigation technology. The findings will supplement the research and provide insights to improve the cure rate and reduce the mortality of colorectal cancer.

Optical diagnosis of colorectal polyps using convolutional neural networks

Colonoscopy remains the gold standard investigation for colorectal cancer screening as it offers the opportunity to both detect and resect pre-malignant and neoplastic polyps. Although technologies for image-enhanced endoscopy are widely available, optical diagnosis has not been incorporated into routine clinical practice, mainly due to significant inter-operator variability. In recent years, there has been a growing number of studies demonstrating the potential of convolutional neural networks (CNN) to enhance optical diagnosis of polyps. Data suggest that the use of CNNs might mitigate the inter-operator variability amongst endoscopists, potentially enabling a “resect and discard“ or ”leave in“ strategy to be adopted in real-time. This would have significant financial benefits for healthcare systems, avoid unnecessary polypectomies of non-neoplastic polyps and improve the efficiency of colonoscopy. Here, we review advances in CNN for the optical diagnosis of colorectal polyps, current limitations and future directions.

Computer-Aided Diagnosis of Diminutive Colorectal Polyps in Endoscopic Images: Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Background

Most colorectal polyps are diminutive and benign, especially those in the rectosigmoid colon, and the resection of these polyps is not cost-effective. Advancements in image-enhanced endoscopy have improved the optical prediction of colorectal polyp histology. However, subjective interpretability and inter- and intraobserver variability prohibits widespread implementation. The number of studies on computer-aided diagnosis (CAD) is increasing; however, their small sample sizes limit statistical significance.

Objective

This review aims to evaluate the diagnostic test accuracy of CAD models in predicting the histology of diminutive colorectal polyps by using endoscopic images.

Methods

Core databases were searched for studies that were based on endoscopic imaging, used CAD models for the histologic diagnosis of diminutive colorectal polyps, and presented data on diagnostic performance. A systematic review and diagnostic test accuracy meta-analysis were performed.

Results

Overall, 13 studies were included. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD models for the diagnosis of diminutive colorectal polyps (adenomatous or neoplastic vs nonadenomatous or nonneoplastic) were 0.96 (95% CI 0.93-0.97), 0.93 (95% CI 0.91-0.95), 0.87 (95% CI 0.76-0.93), and 87 (95% CI 38-201), respectively. The meta-regression analysis showed no heterogeneity, and no publication bias was detected. Subgroup analyses showed robust results. The negative predictive value of CAD models for the diagnosis of adenomatous polyps in the rectosigmoid colon was 0.96 (95% CI 0.95-0.97), and this value exceeded the threshold of the diagnosis and leave strategy.

Conclusions

CAD models show potential for the optical histological diagnosis of diminutive colorectal polyps via the use of endoscopic images.

Trial Registration

PROSPERO CRD42021232189; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=232189

Artificial intelligence for identification and characterization of colonic polyps

Colonoscopy remains the gold standard exam for colorectal cancer screening due to its ability to detect and resect pre-cancerous lesions in the colon. However, its performance is greatly operator dependent. Studies have shown that up to one-quarter of colorectal polyps can be missed on a single colonoscopy, leading to high rates of interval colorectal cancer. In addition, the American Society for Gastrointestinal Endoscopy has proposed the “resect-and-discard” and “diagnose-and-leave” strategies for diminutive colorectal polyps to reduce the costs of unnecessary polyp resection and pathology evaluation. However, the performance of optical biopsy has been suboptimal in community practice. With recent improvements in machine-learning techniques, artificial intelligence–assisted computer-aided detection and diagnosis have been increasingly utilized by endoscopists. The application of computer-aided design on real-time colonoscopy has been shown to increase the adenoma detection rate while decreasing the withdrawal time and improve endoscopists’ optical biopsy accuracy, while reducing the time to make the diagnosis. These are promising steps toward standardization and improvement of colonoscopy quality, and implementation of “resect-and-discard” and “diagnose-and-leave” strategies. Yet, issues such as real-world applications and regulatory approval need to be addressed before artificial intelligence models can be successfully implemented in clinical practice. In this review, we summarize the recent literature on the application of artificial intelligence for detection and characterization of colorectal polyps and review the limitation of existing artificial intelligence technologies and future directions for this field.

Advances in the application of artificial intelligence in solid tumor imaging

Early diagnosis and timely treatment are crucial in reducing cancer-related mortality. Artificial intelligence (AI) has greatly relieved clinical workloads and changed the current medical workflows. We searched for recent studies, reports and reviews referring to AI and solid tumors; many reviews have summarized AI applications in the diagnosis and treatment of a single tumor type. We herein systematically review the advances of AI application in multiple solid tumors including esophagus, stomach, intestine, breast, thyroid, prostate, lung, liver, cervix, pancreas and kidney with a specific focus on the continual improvement on model performance in imaging practice.

Colorectal polyp characterization with standard endoscopy: Will Artificial Intelligence succeed where human eyes failed?

The American Society for Gastrointestinal Endoscopy (ASGE) has proposed the "resect-and-discard" and "diagnose-and-leave" strategies for diminutive colorectal polyps to reduce the costs of unnecessary polyp resection and pathology evaluation. However, the diagnostic thresholds set by these guidelines are not always met in community practice. To overcome this sub-optimal performance, artificial intelligence (AI) has been applied to the field of endoscopy. The incorporation of deep learning algorithms with AI models resulted in highly accurate systems that match the expert endoscopists' optical biopsy and exceed the ASGE recommended thresholds. Recent studies have demonstrated that the integration of AI in clinical practice results in significant improvement in endoscopists' diagnostic accuracy while reducing the time to make a diagnosis. Yet, several points need to be addressed before AI models can be successfully implemented in clinical practice. In this review, we summarize the recent literature on the application of AI for characterization of colorectal polyps, and review the current limitation and future directions for this field.

Artificial Intelligence-Aided Colonoscopy for Polyp Detection: A Systematic Review and Meta-Analysis of Randomized Clinical Trials

Background: This study aimed to compare artificial intelligence (AI)-aided colonoscopy with conventional colonoscopy for polyp detection. Methods: A systematic literature search was performed in PubMed and Ovid for randomized clinical trials (RCTs) comparing AI-aided colonoscopy with conventional colonoscopy for polyp detection. The last search was performed on July 22, 2020. The primary outcome was polyp detection rate (PDR) and adenoma detection rate (ADR). Results: Seven RCTs published between 2019 and 2020 with a total of 5427 individuals were included. When compared with conventional colonoscopy, AI-aided colonoscopy significantly improved PDR (P < .001, odds ratio [OR] = 1.95, 95% confidence interval [CI]: 1.75 to 2.19, I² = 0%) and ADR (P < .001, OR = 1.72, 95% CI: 1.52 to 1.95, I² = 33%). Besides, polyps in the AI-aided group were significantly smaller in size than those in conventional group (P = .004, weighted mean difference = -0.48, 95% CI: -0.81 to -0.15, I² = 0%). In addition, AI-aided group detected significantly less proportion of advanced adenoma (P = .03, OR = 0.70, 95% CI: 0.50 to 0.97, I² = 46%), pedicle polyps (P < .001, OR = 0.64, 95% CI: 0.49 to 0.83, I² = 0%), and pedicle adenomas (P < .001, OR = 0.60, 95% CI: 0.44 to 0.80, I² = 0%). Conclusion: AI-aided colonoscopy could significantly increase the PDR and ADR, especially for those with small size. Besides, the shape and pathology recognition of the AI technique should be further improved in the future.

Amalgamation of cloud-based colonoscopy videos with patient-level metadata to facilitate large-scale machine learning

Background and study aims  Storage of full-length endoscopic procedures is becoming increasingly popular. To facilitate large-scale machine learning (ML) focused on clinical outcomes, these videos must be merged with the patient-level data in the electronic health record (EHR). Our aim was to present a method of accurately linking patient-level EHR data with cloud stored colonoscopy videos. Methods  This study was conducted at a single academic medical center. Most procedure videos are automatically uploaded to the cloud server but are identified only by procedure time and procedure room. We developed and then tested an algorithm to match recorded videos with corresponding exams in the EHR based upon procedure time and room and subsequently extract frames of interest. Results  Among 28,611 total colonoscopies performed over the study period, 21,170 colonoscopy videos in 20,420 unique patients (54.2 % male, median age 58) were matched to EHR data. Of 100 randomly sampled videos, appropriate matching was manually confirmed in all. In total, these videos represented 489,721 minutes of colonoscopy performed by 50 endoscopists (median 214 colonoscopies per endoscopist). The most common procedure indications were polyp screening (47.3 %), surveillance (28.9 %) and inflammatory bowel disease (9.4 %). From these videos, we extracted procedure highlights (identified by image capture; mean 8.5 per colonoscopy) and surrounding frames. Conclusions  We report the successful merging of a large database of endoscopy videos stored with limited identifiers to rich patient-level data in a highly accurate manner. This technique facilitates the development of ML algorithms based upon relevant patient outcomes.

Artificial Intelligence in Lower Gastrointestinal Endoscopy: The Current Status and Future Perspective

The present manuscript aims to review the history, recent advances, evidence, and challenges of artificial intelligence (AI) in colonoscopy. Although it is mainly focused on polyp detection and characterization, it also considers other potential applications (i.e., inflammatory bowel disease) and future perspectives. Some of the most recent algorithms show promising results that are similar to human expert performance. The integration of AI in routine clinical practice will be challenging, with significant issues to overcome (i.e., regulatory, reimbursement). Medico-legal issues will also need to be addressed. With the exception of an AI system that is already available in selected countries (GI Genius; Medtronic, Minneapolis, MN, USA), the majority of the technology is still in its infancy and has not yet been proven to reach a sufficient diagnostic performance to be adopted in the clinical practice. However, larger players will enter the arena of AI in the next few months.

Artificial intelligence in gastrointestinal endoscopy

Background and Aims

Artificial intelligence (AI)-based applications have transformed several industries and are widely used in various consumer products and services. In medicine, AI is primarily being used for image classification and natural language processing and has great potential to affect image-based specialties such as radiology, pathology, and gastroenterology (GE). This document reviews the reported applications of AI in GE, focusing on endoscopic image analysis.

Methods

The MEDLINE database was searched through May 2020 for relevant articles by using key words such as machine learning, deep learning, artificial intelligence, computer-aided diagnosis, convolutional neural networks, GI endoscopy, and endoscopic image analysis. References and citations of the retrieved articles were also evaluated to identify pertinent studies. The manuscript was drafted by 2 authors and reviewed in person by members of the American Society for Gastrointestinal Endoscopy Technology Committee and subsequently by the American Society for Gastrointestinal Endoscopy Governing Board.

Results

Deep learning techniques such as convolutional neural networks have been used in several areas of GI endoscopy, including colorectal polyp detection and classification, analysis of endoscopic images for diagnosis of Helicobacter pylori infection, detection and depth assessment of early gastric cancer, dysplasia in Barrett’s esophagus, and detection of various abnormalities in wireless capsule endoscopy images.

Conclusions

The implementation of AI technologies across multiple GI endoscopic applications has the potential to transform clinical practice favorably and improve the efficiency and accuracy of current diagnostic methods.

A Gratifying Step forward for the Application of Artificial Intelligence in the Field of Endoscopy: A Narrative Review

Endoscopy is the optimal choice of diagnosis of gastrointestinal (GI) diseases. Following the advancements made in medical technology, different kinds of novel endoscopy-methods have emerged. Although the significant progress in the penetration of endoscopic tools that have markedly improved the diagnostic rate of GI diseases, there are still some limitations, including instability of human diagnostic performance caused by intensive labor burden and high missed diagnosis rate of subtle lesions. Recently, artificial intelligence (AI) has been applied gradually to assist endoscopists in addressing these issues.

Potential applications of artificial intelligence in colorectal polyps and cancer: Recent advances and prospects

Since the advent of artificial intelligence (AI) technology, it has been constantly studied and has achieved rapid development. The AI assistant system is expected to improve the quality of automatic polyp detection and classification. It could also help prevent endoscopists from missing polyps and make an accurate optical diagnosis. These functions provided by AI could result in a higher adenoma detection rate and decrease the cost of polypectomy for hyperplastic polyps. In addition, AI has good performance in the staging, diagnosis, and segmentation of colorectal cancer. This article provides an overview of recent research focusing on the application of AI in colorectal polyps and cancer and highlights the advances achieved.

How Artificial Intelligence Will Impact Colonoscopy and Colorectal Screening

Artificial intelligence may improve value in colonoscopy-based colorectal screening and surveillance by improving quality and decreasing unnecessary costs. The quality of screening and surveillance as measured by adenoma detection rates can be improved through real-time computer-assisted detection of polyps. Unnecessary costs can be decreased with optical biopsies to identify low-risk polyps using computer-assisted diagnosis that can undergo the resect-and-discard or diagnose-and-leave strategy. Key challenges include the clinical integration of artificial intelligence-based technology into the endoscopists' workflow, the effect of this technology on endoscopy center efficiency, and the interpretability of the underlying deep learning algorithms. The future for image-based artificial intelligence in gastroenterology will include applications to improve the diagnosis and treatment of cancers throughout the gastrointestinal tract.

Improved Accuracy in Optical Diagnosis of Colorectal Polyps Using Convolutional Neural Networks with Visual Explanations

BACKGROUND & AIMS

Narrow-band imaging (NBI) can be used to determine whether colorectal polyps are adenomatous or hyperplastic. We investigated whether an artificial intelligence (AI) system can increase the accuracy of characterizations of polyps by endoscopists of different skill levels.

METHODS

We developed convolutional neural networks (CNNs) for evaluation of diminutive colorectal polyps, based on efficient neural architecture searches via parameter sharing with augmentation using NBIs of diminutive (≤5 mm) polyps, collected from October 2015 through October 2017 at the Seoul National University Hospital, Healthcare System Gangnam Center (training set). We trained the CNN using images from 1100 adenomatous polyps and 1050 hyperplastic polyps from 1379 patients. We then tested the system using 300 images of 180 adenomatous polyps and 120 hyperplastic polyps, obtained from January 2018 to May 2019. We compared the accuracy of 22 endoscopists of different skill levels (7 novices, 4 experts, and 11 NBI-trained experts) vs the CNN in evaluation of images (adenomatous vs hyperplastic) from 180 adenomatous and 120 hyperplastic polyps. The endoscopists then evaluated the polyp images with knowledge of the CNN-processed results. We conducted mixed-effect logistic and linear regression analyses to determine the effects of AI assistance on the accuracy of analysis of diminutive colorectal polyps by endoscopists (primary outcome).

RESULTS

The CNN distinguished adenomatous vs hyperplastic diminutive polyps with 86.7% accuracy, based on histologic analysis as the reference standard. Endoscopists distinguished adenomatous vs hyperplastic diminutive polyps with 82.5% overall accuracy (novices, 73.8% accuracy; experts, 83.8% accuracy; and NBI-trained experts, 87.6% accuracy). With knowledge of the CNN-processed results, the overall accuracy of the endoscopists increased to 88.5% (P < .05). With knowledge of the CNN-processed results, the accuracy of novice endoscopists increased to 85.6% (P < .05). The CNN-processed results significantly reduced endoscopist time of diagnosis (from 3.92 to 3.37 seconds per polyp, P = .042).

CONCLUSIONS

We developed a CNN that significantly increases the accuracy of evaluation of diminutive colorectal polyps (as adenomatous vs hyperplastic) and reduces the time of diagnosis by endoscopists. This AI assistance system significantly increased the accuracy of analysis by novice endoscopists, who achieved near-expert levels of accuracy without extra training. The CNN assistance system can reduce the skill-level dependence of endoscopists and costs.

Optical evaluation: the crux for effective management of colorectal neoplasia

Advances in minimally invasive tissue resection techniques now allow for the majority of early colorectal neoplasia to be managed endoscopically. To optimize their respective risk-benefit profiles, and, therefore, appropriately select between endoscopic mucosal resection, endoscopic submucosal dissection, and surgery, the endoscopist must accurately predict the risk of submucosal invasive cancer and estimate depth of invasion. Herein, we discuss the evidence and our approach for optical evaluation of large (⩾ 20 mm) colorectal laterally spreading lesions.