Artificial intelligence technologies for the detection of colorectal lesions: The future is now

Predictors of early colorectal cancer metastasis to lymph nodes: providing rationale for therapy decisions

With the improvement of national health awareness and the popularization of a series of screening methods, the number of patients with early colorectal cancer is gradually increasing, and accurate prediction of lymph node metastasis of T1 colorectal cancer is the key to determining the optimal therapeutic solutions. Whether patients with T1 colorectal cancer undergoing endoscopic resection require additional surgery and regional lymph node dissection is inconclusive in current guidelines. However, we can be sure that in early colorectal cancer without lymph node metastasis, endoscopic resection alone does not affect the prognosis, and it greatly improves the quality of life and reduces the incidence of surgical complications while preserving organ integrity. Therefore, it is vital to discriminate patients without lymph node metastasis in T1 colorectal cancer, and this requires accurate predictors. This paper briefly explains the significance and shortcomings of traditional pathological factors, then extends and states the new pathological factors, clinical test factors, molecular biomarkers, and the risk assessment models of lymph node metastasis based on artificial intelligence.

Artificial Intelligence-assisted colonoscopy and colorectal cancer screening: Where are we going?

Colorectal cancer is a significant global health concern, necessitating effective screening strategies to reduce its incidence and mortality rates. Colonoscopy plays a crucial role in the detection and removal of colorectal neoplastic precursors. However, there are limitations and variations in the performance of endoscopists, leading to missed lesions and suboptimal outcomes. The emergence of artificial intelligence (AI) in endoscopy offers promising opportunities to improve the quality and efficacy of screening colonoscopies. In particular, AI applications, including computer-aided detection (CADe) and computer-aided characterization (CADx), have demonstrated the potential to enhance adenoma detection and optical diagnosis accuracy. Additionally, AI-assisted quality control systems aim to standardize the endoscopic examination process. This narrative review provides an overview of AI principles and discusses the current knowledge on AI-assisted endoscopy in the context of screening colonoscopies. It highlights the significant role of AI in improving lesion detection, characterization, and quality assurance during colonoscopy. However, further well-designed studies are needed to validate the clinical impact and cost-effectiveness of AI-assisted colonoscopy before its widespread implementation.

The imitation game: a review of the use of artificial intelligence in colonoscopy, and endoscopists' perceptions thereof

The development of deep learning systems in artificial intelligence (AI) has enabled advances in endoscopy, and AI-aided colonoscopy has recently been ushered into clinical practice as a clinical decision-support tool. This has enabled real-time AI-aided detection of polyps with a higher sensitivity than the average endoscopist, and evidence to support its use has been promising thus far. This review article provides a summary of currently published data relating to AI-aided colonoscopy, discusses current clinical applications, and introduces ongoing research directions. We also explore endoscopists' perceptions and attitudes toward the use of this technology, and discuss factors influencing its uptake in clinical practice.

Knowledge, perceptions and behaviours of endoscopists towards the use of artificial intelligence-aided colonoscopy

BACKGROUND

Recent developments in artificial intelligence (AI) systems have enabled advancements in endoscopy. Deep learning systems, using convolutional neural networks, have allowed for real-time AI-aided detection of polyps with higher sensitivity than the average endoscopist. However, not all endoscopists welcome the advent of AI systems.

METHODS

We conducted a survey on the knowledge of AI, perceptions of AI in medicine, and behaviours regarding use of AI-aided colonoscopy, in a single centre 2 months after the implementation of Medtronic's GI Genius in colonoscopy. We obtained a response rate of 66.7% (16/24) amongst consultant-grade endoscopists. Fisher's exact test was used to calculate the significance of correlations.

RESULTS

Knowledge of AI varied widely amongst endoscopists. Most endoscopists were optimistic about AI's capabilities in performing objective administrative and clinical tasks, but reserved about AI providing personalised, empathetic care. 68.8% (n = 11) of endoscopists agreed or strongly agreed that GI Genius should be used as an adjunct in colonoscopy. In analysing the 31.3% (n = 5) of endoscopists who disagreed or were ambivalent about its use, there was no significant correlation with their knowledge or perceptions of AI, but a significant number did not enjoy using the programme (p-value = 0.0128) and did not think it improved the quality of colonoscopy (p-value = 0.033).

CONCLUSIONS

Acceptance of AI-aided colonoscopy systems is more related to the endoscopist's experience with using the programme, rather than general knowledge or perceptions towards AI. Uptake of such systems will rely greatly on how the device is delivered to the end user.

Artificial Intelligence-Aided Endoscopy and Colorectal Cancer Screening

Colorectal cancer (CRC) is the third most common cancer worldwide, with the highest incidence reported in high-income countries. However, because of the slow progression of neoplastic precursors, along with the opportunity for their endoscopic detection and resection, a well-designed endoscopic screening program is expected to strongly decrease colorectal cancer incidence and mortality. In this regard, quality of colonoscopy has been clearly related with the risk of post-colonoscopy colorectal cancer. Recently, the development of artificial intelligence (AI) applications in the medical field has been growing in interest. Through machine learning processes, and, more recently, deep learning, if a very high numbers of learning samples are available, AI systems may automatically extract specific features from endoscopic images/videos without human intervention, helping the endoscopists in different aspects of their daily practice. The aim of this review is to summarize the current knowledge on AI-aided endoscopy, and to outline its potential role in colorectal cancer prevention.

The role of artificial intelligence based systems for cost optimization in colorectal cancer prevention programs

Colorectal Cancer (CRC) has seen a dramatic increase in incidence globally. In 2019, colorectal cancer accounted for 1.15 million deaths and 24.28 million disability-adjusted life-years (DALYs) worldwide. In India, the annual incidence rates (AARs) for colon cancer was 4.4 per 100,000. There has been a steady rise in the prevalence of CRC in India which may be attributed to urbanization, mass migration of population, westernization of diet and lifestyle practices and a rise of obesity and metabolic risk factors that place the population at a higher risk of CRC. Moreoever, CRC in India differs from that described in the Western countries, with a higher proportion of young patients and more patients presenting with an advanced stage. This may be due to poor access to specialized healthcare and socio-economic factors. Early identification of adenomatous colonic polyps, which are well-recognized pre-cancerous lesions, at the time of screening colonoscopy has been shown to be the most effective measure used for CRC prevention. However, colonic polyps are frequently missed during colonoscopy and moreover, these screening programs necessitate man-power, time and resources for processing resected polyps, that may hamper penetration and efficacy in mid- to low-income countries. In the last decade, there has been significant progress made in the automatic detection of colonic polyps by multiple AI-based systems. With the advent of better AI methodology, the focus has shifted from mere detection to accurate discrimination and diagnosis of colonic polyps. These systems, once validated, could usher in a new era in Colorectal Cancer (CRC) prevention programs which would center around “Leave in-situ” and “Resect and discard” strategies. These new strategies hinge around the specificity and accuracy of AI based systems in correctly identifying the pathological diagnosis of the polyps, thereby providing the endoscopist with real-time information in order to make a clinical decision of either leaving the lesion in-situ (mucosal polyps) or resecting and discarding the polyp (hyperplastic polyps). The major advantage of employing these strategies would be in cost optimization of CRC prevention programs while ensuring good clinical outcomes. The adoption of these AI-based systems in the national cancer prevention program of India in accordance with the mandate to increase technology integration could prove to be cost-effective and enable implementation of CRC prevention programs at the population level. This level of penetration could potentially reduce the incidence of CRC and improve patient survival by enabling early diagnosis and treatment. In this review, we will highlight key advancements made in the field of AI in the identification of polyps during colonoscopy and explore the role of AI based systems in cost optimization during the universal implementation of CRC prevention programs in the context of mid-income countries like India.

Development and validation of a deep learning-based algorithm for colonoscopy quality assessment

BACKGROUND

Quality indicators should be assessed and monitored to improve colonoscopy quality in clinical practice. Endoscopists must enter relevant information in the endoscopy reporting system to facilitate data collection, which may be inaccurate. The current study aimed to develop a full deep learning-based algorithm to identify and analyze intra-procedural colonoscopy quality indicators based on endoscopy images obtained during the procedure.

METHODS

A deep learning system for classifying colonoscopy images for quality assurance purposes was developed and its performance was assessed with an independent dataset. The system was utilized to analyze captured images and results were compared with those of real-world reports.

RESULTS

In total, 10,417 images from the hospital endoscopy database and 3157 from Hyper-Kvasir open dataset were utilized to develop the quality assurance algorithm. The overall accuracy of the algorithm was 96.72% and that of the independent test dataset was 94.71%. Moreover, 761 real-world reports and colonoscopy images were analyzed. The accuracy of electronic reports about cecal intubation rate was 99.34% and that of the algorithm was 98.95%. The agreement rate for the assessment of polypectomy rates using the electronic reports and the algorithm was 0.87 (95% confidence interval 0.83-0.90). A good correlation was found between the withdrawal time calculated using the algorithm and that entered by the physician (correlation coefficient r = 0.959, p < 0.0001).

CONCLUSION

We proposed a novel deep learning-based algorithm that used colonoscopy images for quality assurance purposes. This model can be used to automatically assess intra-procedural colonoscopy quality indicators in clinical practice.

Artificial intelligence in endoscopy: More than what meets the eye in screening colonoscopy and endosonographic evaluation of pancreatic lesions

Artificial intelligence (AI)-based tools have ushered in a new era of innovation in the field of gastrointestinal (GI) endoscopy. Despite vast improvements in endoscopic techniques and equipment, diagnostic endoscopy remains heavily operator-dependent, in particular, colonoscopy and endoscopic ultrasound (EUS). Recent reports have shown that as much as 25% of colonic adenomas may be missed at colonoscopy. This can result in an increased incidence of interval colon cancer. Similarly, EUS has been shown to have high inter-observer variability, overlap in diagnoses with a relatively low specificity for pancreatic lesions. Our understanding of Machine-learning (ML) techniques in AI have evolved over the last decade and its application in AI–based tools for endoscopic detection and diagnosis is being actively investigated at several centers. ML is an aspect of AI that is based on neural networks, and is widely used for image classification, object detection, and semantic segmentation which are key functional aspects of AI-related computer aided diagnostic systems. In this review, current status and limitations of ML, specifically for adenoma detection and endosonographic diagnosis of pancreatic lesions, will be summarized from existing literature. This will help to better understand its role as viewed through the prism of real world application in the field of GI endoscopy.

Polyp detection in video colonoscopy using deep learning

Video colonoscopy automatic processing is a challenge and further development of computer assisted diagnosis is very helpful in correctness assessment of the exam, in e-learning and training, for statistics on polyps’ malignity or in polyps’ survey. New devices and programming languages are emerging and deep learning begun already to furnish astonishing results, in the quest for high speed and optimal polyp detection software. This paper presents a successful attempt in detecting the intestinal polyps in real time video colonoscopy with deep learning, using Mobile Net.

Deep learning-based endoscopic anatomy classification: an accelerated approach for data preparation and model validation

BACKGROUND

Photodocumentation during endoscopy procedures is one of the indicators for endoscopy performance quality; however, this indicator is difficult to measure and audit in the endoscopy unit. Emerging artificial intelligence technology may solve this problem, which requires a large amount of material for model development. We developed a deep learning-based endoscopic anatomy classification system through convolutional neural networks with an accelerated data preparation approach.

PATIENTS AND METHODS

We retrospectively collected 8,041 images from esophagogastroduodenoscopy (EGD) procedures and labeled them using two experts for nine anatomical locations of the upper gastrointestinal tract. A base model for EGD image multiclass classification was first developed, and an additional 6,091 images were enrolled and classified by the base model. A total of 5,963 images were manually confirmed and added to develop the subsequent enhanced model. Additional internal and external endoscopy image datasets were used to test the model performance.

RESULTS

The base model achieved total accuracy of 96.29%. For the enhanced model, the total accuracy was 96.64%. The overall accuracy improved with the enhanced model compared with the base model for the internal test dataset without narrowband images (93.05% vs. 91.25%, p < 0.01) or with narrowband images (92.74% vs. 90.46%, p < 0.01). The total accuracy was 92.56% of the enhanced model on the external test dataset.

CONCLUSIONS

We constructed a deep learning-based model with an accelerated approach that can be used for quality control in endoscopy units. The model was also validated with both internal and external datasets with high accuracy.

Comparing the number and relevance of false activations between 2 artificial intelligence computer-aided detection systems: the NOISE study

BACKGROUND AND AIMS

Artificial intelligence has been shown to be effective in polyp detection, and multiple computer-aided detection (CADe) systems have been developed. False-positive (FP) activation emerged as a possible way to benchmark CADe performance in clinical practice. The aim of this study was to validate a previously developed classification of FPs comparing the performances of different brands of approved CADe systems.

METHODS

We compared 2 different consecutive video libraries (40 video per arm) collected at Humanitas Research Hospital with 2 different CADe system brands (CADe A and CADe B). For each video, the number of CADe false activations, cause, and time spent by the endoscopist to examine the area erroneously highlighted were reported. The FP activations were classified according to the previously developed classification of FPs (the NOISE classification) according to their cause and relevance.

RESULTS

In CADe A 1021 FP activations were registered across the 40 videos (25.5 ± 12.2 FPs per colonoscopy), whereas in CADe B 1028 were identified (25.7 ± 13.2 FPs per colonoscopy; P = .53). Among them, 22.9 ± 9.9 (89.8% in CADe A) and 22.1 ± 10.0 (86.0% in CADe B) were because of artifacts from the bowel wall. Conversely, 2.6 ± 1.9 (10.2% in CADe A) and 3.5 ± 2.1 (14% in CADe B) were caused by bowel content (P = .45). Within CADe A each false activation required .2 ± .9 seconds, with 1.6 ± 1.0 FPs (6.3%) requiring additional time for endoscopic assessment. Comparable results were reported within CADe B with .2 ± .8 seconds spent per false activation and 1.8 ± 1.2 FPs per colonoscopy requiring additional inspection.

CONCLUSIONS

The use of a standardized nomenclature provided comparable results with either of the 2 recently approved CADe systems. (Clinical trial registration number: NCT04399590.).

Discovering the first US FDA-approved computer-aided polyp detection system

Colorectal cancer is the third most common cancer worldwide. Because of the slow progression of the precancerous precursors, an efficient endoscopic surveillance strategy may be expected. It seems that around one-fourth of colorectal malignancies are still missed during colonoscopy. Several endoscopic technologies have been introduced, without radical changes. Interest in the development of artificial intelligence applications in the medical field has grown in the past decade. Artificial intelligence can help to highlight a specific region of interest that needs closer examination for the identification of polyps. The aim of this review is to report the first clinical experiences with the first US FDA-approved, real-time, deep-learning, computer-aided detection system (GI Genius™, Medtronic).

Advanced imaging and artificial intelligence for Barrett's esophagus: What we should and soon will do

Barrett’s esophagus (BE) is a well-established risk factor for esophageal adenocarcinoma. It is recommended that patients have regular endoscopic surveillance, with the ultimate goal of detecting early-stage neoplastic lesions before they can progress to invasive carcinoma. Detection of both dysplasia or early adenocarcinoma permits curative endoscopic treatments, and with this aim, thorough endoscopic assessment is crucial and improves outcomes. The burden of missed neoplasia in BE is still far from being negligible, likely due to inappropriate endoscopic surveillance. Over the last two decades, advanced imaging techniques, moving from traditional dye-spray chromoendoscopy to more practical virtual chromoendoscopy technologies, have been introduced with the aim to enhance neoplasia detection in BE. As witnessed in other fields, artificial intelligence (AI) has revolutionized the field of diagnostic endoscopy and is set to cover a pivotal role in BE as well. The aim of this commentary is to comprehensively summarize present evidence, recent research advances, and future perspectives regarding advanced imaging technology and AI in BE; the combination of computer-aided diagnosis to a widespread adoption of advanced imaging technologies is eagerly awaited. It will also provide a useful step-by-step approach for performing high-quality endoscopy in BE, in order to increase the diagnostic yield of endoscopy in clinical practice.

A promising deep learning-assistive algorithm for histopathological screening of colorectal cancer

Colorectal cancer is one of the most common cancers worldwide, accounting for an annual estimated 1.8 million incident cases. With the increasing number of colonoscopies being performed, colorectal biopsies make up a large proportion of any histopathology laboratory workload. We trained and validated a unique artificial intelligence (AI) deep learning model as an assistive tool to screen for colonic malignancies in colorectal specimens, in order to improve cancer detection and classification; enabling busy pathologists to focus on higher order decision-making tasks. The study cohort consists of Whole Slide Images (WSI) obtained from 294 colorectal specimens. Qritive’s unique composite algorithm comprises both a deep learning model based on a Faster Region Based Convolutional Neural Network (Faster-RCNN) architecture for instance segmentation with a ResNet-101 feature extraction backbone that provides glandular segmentation, and a classical machine learning classifier. The initial training used pathologists’ annotations on a cohort of 66,191 image tiles extracted from 39 WSIs. A subsequent application of a classical machine learning-based slide classifier sorted the WSIs into ‘low risk’ (benign, inflammation) and ‘high risk’ (dysplasia, malignancy) categories. We further trained the composite AI-model’s performance on a larger cohort of 105 resections WSIs and then validated our findings on a cohort of 150 biopsies WSIs against the classifications of two independently blinded pathologists. We evaluated the area under the receiver-operator characteristic curve (AUC) and other performance metrics. The AI model achieved an AUC of 0.917 in the validation cohort, with excellent sensitivity (97.4%) in detection of high risk features of dysplasia and malignancy. We demonstrate an unique composite AI-model incorporating both a glandular segmentation deep learning model and a classical machine learning classifier, with excellent sensitivity in picking up high risk colorectal features. As such, AI plays a role as a potential screening tool in assisting busy pathologists by outlining the dysplastic and malignant glands.

Artificial intelligence applications in gastroenterology: steps ahead

Artificial intelligence (AI) applications are used in gastroenterology for automatic imaging diagnostic methods such as ultrasonography, computer tomography, magnetic resonance imaging, but also in endoscopy, capsule endoscopy and biopsy followed by automatic digital pathology evaluation. The accuracy of AI-based systems is superior to human expertise. Furthermore, in reality, a very small percentage of the patients are being investigated by a human expert in endoscopy, so implementing AI in this investigation would only increase the diagnostic accuracy. The existence of an unimaginable number of digital images and different types of medical information made possible the analysis and training of convolutional neural network (CNN), which consists of multilayers of artificial neural networks (ANN) with step-by-step minimal processing, creating a fundamental resource for any AI-based system to learn by itself how to automatically perform medical tasks, which were performed only by human experts in the past. The main objectives for AI applications used in gastroenterology are to improve the medical procedures with enhanced precision, to reduce the number of medical errors and to perform repetitive tasks.

Usefulness of artificial intelligence in gastric neoplasms

Recently, studies in many medical fields have reported that image analysis based on artificial intelligence (AI) can be used to analyze structures or features that are difficult to identify with human eyes. To diagnose early gastric cancer, related efforts such as narrow-band imaging technology are on-going. However, diagnosis is often difficult. Therefore, a diagnostic method based on AI for endoscopic imaging was developed and its effectiveness was confirmed in many studies. The gastric cancer diagnostic program based on AI showed relatively high diagnostic accuracy and could differentially diagnose non-neoplastic lesions including benign gastric ulcers and dysplasia. An AI system has also been developed that helps to predict the invasion depth of gastric cancer through endoscopic images and observe the stomach during endoscopy without blind spots. Therefore, if AI is used in the field of endoscopy, it is expected to aid in the diagnosis of gastric neoplasms and determine the application of endoscopic therapy by predicting the invasion depth.

Performance Comparison of the Deep Learning and the Human Endoscopist for Bleeding Peptic Ulcer Disease

Purpose

Management of peptic ulcer bleeding is clinically challenging. Accurate characterization of the bleeding during endoscopy is key for endoscopic therapy. This study aimed to assess whether a deep learning model can aid in the classification of bleeding peptic ulcer disease.

Methods

Endoscopic still images of patients (n = 1694) with peptic ulcer bleeding for the last 5 years were retrieved and reviewed. Overall, 2289 images were collected for deep learning model training, and 449 images were validated for the performance test. Two expert endoscopists classified the images into different classes based on their appearance. Four deep learning models, including Mobile Net V2, VGG16, Inception V4, and ResNet50, were proposed and pre-trained by ImageNet with the established convolutional neural network algorithm. A comparison of the endoscopists and trained deep learning model was performed to evaluate the model’s performance on a dataset of 449 testing images.

Results

The results first presented the performance comparisons of four deep learning models. The Mobile Net V2 presented the optimal performance of the proposal models. The Mobile Net V2 was chosen for further comparing the performance with the diagnostic results obtained by one senior and one novice endoscopists. The sensitivity and specificity were acceptable for the prediction of “normal” lesions in both 3-class and 4-class classifications. For the 3-class category, the sensitivity and specificity were 94.83% and 92.36%, respectively. For the 4-class category, the sensitivity and specificity were 95.40% and 92.70%, respectively. The interobserver agreement of the testing dataset of the model was moderate to substantial with the senior endoscopist. The accuracy of the determination of endoscopic therapy required and high-risk endoscopic therapy of the deep learning model was higher than that of the novice endoscopist.

Conclusions

In this study, the deep learning model performed better than inexperienced endoscopists. Further improvement of the model may aid in clinical decision-making during clinical practice, especially for trainee endoscopist.

A novel lightweight deep convolutional neural network for early detection of oral cancer

OBJECTIVES

To develop a lightweight deep convolutional neural network (CNN) for binary classification of oral lesions into benign and malignant or potentially malignant using standard real-time clinical images.

METHODS

A small deep CNN, that uses a pretrained EfficientNet-B0 as a lightweight transfer learning model, was proposed. A data set of 716 clinical images was used to train and test the proposed model. Accuracy, specificity, sensitivity, receiver operating characteristics (ROC) and area under curve (AUC) were used to evaluate performance. Bootstrapping with 120 repetitions was used to calculate arithmetic means and 95% confidence intervals (CIs).

RESULTS

The proposed CNN model achieved an accuracy of 85.0% (95% CI: 81.0%-90.0%), a specificity of 84.5% (95% CI: 78.9%-91.5%), a sensitivity of 86.7% (95% CI: 80.4%-93.3%) and an AUC of 0.928 (95% CI: 0.88-0.96).

CONCLUSIONS

Deep CNNs can be an effective method to build low-budget embedded vision devices with limited computation power and memory capacity for diagnosis of oral cancer. Artificial intelligence (AI) can improve the quality and reach of oral cancer screening and early detection.