Artificial Intelligence in Gastrointestinal Endoscopy

Prompt-based polyp segmentation during endoscopy

Accurate judgment and identification of polyp size is crucial in endoscopic diagnosis. However, the indistinct boundaries of polyps lead to missegmentation and missed cancer diagnoses. In this paper, a prompt-based polyp segmentation method (PPSM) is proposed to assist in early-stage cancer diagnosis during endoscopy. It combines endoscopists' experience and artificial intelligence technology. Firstly, a prompt-based polyp segmentation network (PPSN) is presented, which contains the prompt encoding module (PEM), the feature extraction encoding module (FEEM), and the mask decoding module (MDM). The PEM encodes prompts to guide the FEEM for feature extracting and the MDM for mask generating. So that PPSN can segment polyps efficiently. Secondly, endoscopists' ocular attention data (gazes) are used as prompts, which can enhance PPSN's accuracy for segmenting polyps and obtain prompt data effectively in real-world. To reinforce the PPSN's stability, non-uniform dot matrix prompts are generated to compensate for frame loss during the eye-tracking. Moreover, a data augmentation method based on the segment anything model (SAM) is introduced to enrich the prompt dataset and improve the PPSN's adaptability. Experiments demonstrate the PPSM's accuracy and real-time capability. The results from cross-training and cross-testing on four datasets show the PPSM's generalization. Based on the research results, a disposable electronic endoscope with the real-time auxiliary diagnosis function for early cancer and an image processor have been developed. Part of the code and the method for generating the prompts dataset are available at https://github.com/XinZhenRen/PPSM.

Deep Learning-Based Semantic Segmentation for Objective Colonoscopy Quality Assessment

Background: This study aims to objectively evaluate the overall quality of colonoscopies using a specially trained deep learning-based semantic segmentation neural network. This represents a modern and valuable approach for the analysis of colonoscopy frames. Methods: We collected thousands of colonoscopy frames extracted from a set of video colonoscopy files. A color-based image processing method was used to extract color features from specific regions of each colonoscopy frame, namely, the intestinal mucosa, residues, artifacts, and lumen. With these features, we automatically annotated all the colonoscopy frames and then selected the best of them to train a semantic segmentation network. This trained network was used to classify the four region types in a different set of test colonoscopy frames and extract pixel statistics that are relevant to quality evaluation. The test colonoscopies were also evaluated by colonoscopy experts using the Boston scale. Results: The deep learning semantic segmentation method obtained good results, in terms of classifying the four key regions in colonoscopy frames, and produced pixel statistics that are efficient in terms of objective quality assessment. The Spearman correlation results were as follows: BBPS vs. pixel scores: 0.69; BBPS vs. mucosa pixel percentage: 0.63; BBPS vs. residue pixel percentage: -0.47; BBPS vs. Artifact Pixel Percentage: -0.65. The agreement analysis using Cohen's Kappa yielded a value of 0.28. The colonoscopy evaluation based on the extracted pixel statistics showed a fair level of compatibility with the experts' evaluations. Conclusions: Our proposed deep learning semantic segmentation approach is shown to be a promising tool for evaluating the overall quality of colonoscopies and goes beyond the Boston Bowel Preparation Scale in terms of assessing colonoscopy quality. In particular, while the Boston scale focuses solely on the amount of residual content, our method can identify and quantify the percentage of colonic mucosa, residues, and artifacts, providing a more comprehensive and objective evaluation.

Consensus statements on the current landscape of artificial intelligence applications in endoscopy, addressing roadblocks, and advancing artificial intelligence in gastroenterology

BACKGROUND AND AIMS

The American Society for Gastrointestinal Endoscopy (ASGE) AI Task Force along with experts in endoscopy, technology space, regulatory authorities, and other medical subspecialties initiated a consensus process that analyzed the current literature, highlighted potential areas, and outlined the necessary research in artificial intelligence (AI) to allow a clearer understanding of AI as it pertains to endoscopy currently.

METHODS

A modified Delphi process was used to develop these consensus statements.

RESULTS

Statement 1: Current advances in AI allow for the development of AI-based algorithms that can be applied to endoscopy to augment endoscopist performance in detection and characterization of endoscopic lesions. Statement 2: Computer vision-based algorithms provide opportunities to redefine quality metrics in endoscopy using AI, which can be standardized and can reduce subjectivity in reporting quality metrics. Natural language processing-based algorithms can help with the data abstraction needed for reporting current quality metrics in GI endoscopy effortlessly. Statement 3: AI technologies can support smart endoscopy suites, which may help optimize workflows in the endoscopy suite, including automated documentation. Statement 4: Using AI and machine learning helps in predictive modeling, diagnosis, and prognostication. High-quality data with multidimensionality are needed for risk prediction, prognostication of specific clinical conditions, and their outcomes when using machine learning methods. Statement 5: Big data and cloud-based tools can help advance clinical research in gastroenterology. Multimodal data are key to understanding the maximal extent of the disease state and unlocking treatment options. Statement 6: Understanding how to evaluate AI algorithms in the gastroenterology literature and clinical trials is important for gastroenterologists, trainees, and researchers, and hence education efforts by GI societies are needed. Statement 7: Several challenges regarding integrating AI solutions into the clinical practice of endoscopy exist, including understanding the role of human-AI interaction. Transparency, interpretability, and explainability of AI algorithms play a key role in their clinical adoption in GI endoscopy. Developing appropriate AI governance, data procurement, and tools needed for the AI lifecycle are critical for the successful implementation of AI into clinical practice. Statement 8: For payment of AI in endoscopy, a thorough evaluation of the potential value proposition for AI systems may help guide purchasing decisions in endoscopy. Reliable cost-effectiveness studies to guide reimbursement are needed. Statement 9: Relevant clinical outcomes and performance metrics for AI in gastroenterology are currently not well defined. To improve the quality and interpretability of research in the field, steps need to be taken to define these evidence standards. Statement 10: A balanced view of AI technologies and active collaboration between the medical technology industry, computer scientists, gastroenterologists, and researchers are critical for the meaningful advancement of AI in gastroenterology.

CONCLUSIONS

The consensus process led by the ASGE AI Task Force and experts from various disciplines has shed light on the potential of AI in endoscopy and gastroenterology. AI-based algorithms have shown promise in augmenting endoscopist performance, redefining quality metrics, optimizing workflows, and aiding in predictive modeling and diagnosis. However, challenges remain in evaluating AI algorithms, ensuring transparency and interpretability, addressing governance and data procurement, determining payment models, defining relevant clinical outcomes, and fostering collaboration between stakeholders. Addressing these challenges while maintaining a balanced perspective is crucial for the meaningful advancement of AI in gastroenterology.

[Status Quo of Surgical Navigation]

Surgical navigation, also referred to as computer-assisted or image-guided surgery, is a technique that employs a variety of methods - such as 3D imaging, tracking systems, specialised software, and robotics to support surgeons during surgical interventions. These emerging technologies aim not only to enhance the accuracy and precision of surgical procedures, but also to enable less invasive approaches, with the objective of reducing complications and improving operative outcomes for patients. By harnessing the integration of emerging digital technologies, surgical navigation holds the promise of assisting complex procedures across various medical disciplines. In recent years, the field of surgical navigation has witnessed significant advances. Abdominal surgical navigation, particularly endoscopy, laparoscopic, and robot-assisted surgery, is currently undergoing a phase of rapid evolution. Emphases include image-guided navigation, instrument tracking, and the potential integration of augmented and mixed reality (AR, MR). This article will comprehensively delve into the latest developments in surgical navigation, spanning state-of-the-art intraoperative technologies like hyperspectral and fluorescent imaging, to the integration of preoperative radiological imaging within the intraoperative setting.

Artificial intelligence-aided diagnostic imaging: A state-of-the-art technique in precancerous screening

BACKGROUND AND AIM

Chromoendoscopy with the use of indigo carmine (IC) dye is a crucial endoscopic technique to identify gastrointestinal neoplasms. However, its performance is limited by the endoscopist's skill, and no standards are available for lesion identification. Thus, we developed an artificial intelligence (AI) model to replace chromoendoscopy.

METHODS

This pilot study assessed the feasibility of our novel AI model in the conversion of white-light images (WLI) into virtual IC-dyed images based on a generative adversarial network. The predictions of our AI model were evaluated against the assessments of five endoscopic experts who were blinded to the purpose of this study with a staining quality rating from 1 (unacceptable) to 4 (excellent).

RESULTS

The AI model successfully transformed the WLI of polyps with different morphologies and different types of lesions in the gastrointestinal tract into virtual IC-dyed images. The quality ratings of the real IC-dyed and AI images did not significantly differ concerning surface structure (AI vs IC: 3.08 vs 3.00), lesion border (3.04 vs 2.98), and overall contrast (3.14 vs 3.02) from 10 sets of images (10 AI images and 10 real IC-dyed images). Although the score depended significantly on the evaluator, the staining methods (AI or real IC) and evaluators had no significant interaction (P > 0.05) with each other.

CONCLUSION

Our results demonstrated the feasibility of employing AI model's virtual IC staining, increasing the possibility of being employed in daily practice. This novel technology may facilitate gastrointestinal lesion identification in the future.

Development of convolutional neural network models that recognize normal anatomic structures during real-time radial-array and linear-array EUS (with videos)

BACKGROUND AND AIMS

EUS is a high-skill technique that requires numerous procedures to achieve competence. However, training facilities are limited worldwide. Convolutional neural network (CNN) models have been previously implemented for object detection. We developed 2 EUS-based CNN models for normal anatomic structure recognition during real-time linear- and radial-array EUS evaluations.

METHODS

The study was performed from February 2020 to June 2022. Consecutive patient videos of linear- and radial-array EUS videos were recorded. Expert endosonographers identified and labeled 20 normal anatomic structures within the videos for training and validation of the CNN models. Initial CNN models (CNNv1) were developed from 45 videos and the improved models (CNNv2) from an additional 102 videos. CNN model performance was compared with that of 2 expert endosonographers.

RESULTS

CNNv1 used 45,034 linear-array EUS frames and 21,063 radial-array EUS frames. CNNv2 used 148,980 linear-array EUS frames and 128,871 radial-array EUS frames. Linear-array CNNv1 and radial-array CNNv1 achieved a 75.65% and 71.36% mean average precision (mAP) with a total loss of .19 and .18, respectively. Linear-array CNNv2 obtained an 88.7% mAP with a .06 total loss, whereas radial-array CNNv2 achieved an 83.5% mAP with a .07 total loss. CNNv2 accurately detected all studied normal anatomic structures with a >98% observed agreement during clinical validation.

CONCLUSIONS

The proposed CNN models accurately recognize the normal anatomic structures in prerecorded videos and real-time EUS. Prospective trials are needed to evaluate the impact of these models on the learning curves of EUS trainees.

Deep learning-based clinical decision support system for gastric neoplasms in real-time endoscopy: development and validation study

BACKGROUND : Deep learning models have previously been established to predict the histopathology and invasion depth of gastric lesions using endoscopic images. This study aimed to establish and validate a deep learning-based clinical decision support system (CDSS) for the automated detection and classification (diagnosis and invasion depth prediction) of gastric neoplasms in real-time endoscopy. METHODS : The same 5017 endoscopic images that were employed to establish previous models were used for the training data. The primary outcomes were: (i) the lesion detection rate for the detection model, and (ii) the lesion classification accuracy for the classification model. For performance validation of the lesion detection model, 2524 real-time procedures were tested in a randomized pilot study. Consecutive patients were allocated either to CDSS-assisted or conventional screening endoscopy. The lesion detection rate was compared between the groups. For performance validation of the lesion classification model, a prospective multicenter external test was conducted using 3976 novel images from five institutions. RESULTS : The lesion detection rate was 95.6 % (internal test). On performance validation, CDSS-assisted endoscopy showed a higher lesion detection rate than conventional screening endoscopy, although statistically not significant (2.0 % vs. 1.3 %; P = 0.21) (randomized study). The lesion classification rate was 89.7 % in the four-class classification (advanced gastric cancer, early gastric cancer, dysplasia, and non-neoplastic) and 89.2 % in the invasion depth prediction (mucosa confined or submucosa invaded; internal test). On performance validation, the CDSS reached 81.5 % accuracy in the four-class classification and 86.4 % accuracy in the binary classification (prospective multicenter external test). CONCLUSIONS : The CDSS demonstrated its potential for real-life clinical application and high performance in terms of lesion detection and classification of detected lesions in the stomach.

Artificial intelligence in pancreatic cancer: diagnosis, limitations, and the future prospects-a narrative review

PURPOSE

This review aims to explore the role of AI in the application of pancreatic cancer management and make recommendations to minimize the impact of the limitations to provide further benefits from AI use in the future.

METHODS

A comprehensive review of the literature was conducted using a combination of MeSH keywords, including "Artificial intelligence", "Pancreatic cancer", "Diagnosis", and "Limitations".

RESULTS

The beneficial implications of AI in the detection of biomarkers, diagnosis, and prognosis of pancreatic cancer have been explored. In addition, current drawbacks of AI use have been divided into subcategories encompassing statistical, training, and knowledge limitations; data handling, ethical and medicolegal aspects; and clinical integration and implementation.

CONCLUSION

Artificial intelligence (AI) refers to computational machine systems that accomplish a set of given tasks by imitating human intelligence in an exponential learning pattern. AI in gastrointestinal oncology has continued to provide significant advancements in the clinical, molecular, and radiological diagnosis and intervention techniques required to improve the prognosis of many gastrointestinal cancer types, particularly pancreatic cancer.

Automated Detection of Bowel Preparation Scoring and Adequacy With Deep Convolutional Neural Networks

INTRODUCTION

Adequate bowel preparation is integral to effective colonoscopy. Inadequate bowel preparation has been associated with reduced adenoma detection rate and increased post-colonoscopy colorectal cancer (PCCRC). As a result, the USMSTF recommends early interval reevaluation for colonoscopies with inadequate bowel preparation. However, bowel preparation documentation is highly variable with subjective interpretation. In this study, we developed deep convolutional neural networks (DCNN) to objectively ascertain bowel preparation.

METHODS

Bowel preparation scores were assigned using the Boston Bowel Preparation Scale (BBPS). Bowel preparation adequacy and inadequacy were defined as BBPS ≥2 and BBPS <2, respectively. A total of 38523 images were extracted from 28 colonoscopy videos and split into 26966 images for training, 7704 for validation, and 3853 for testing. Two DCNNs were created using a Densenet-169 backbone in PyTorch library evaluating BBPS score and bowel preparation adequacy. We used Adam optimiser with an initial learning rate of 3 × 10-4 and a scheduler to decay the learning rate of each parameter group by 0.1 every 7 epochs along with focal loss as our criterion for both classifiers.

RESULTS

The overall accuracy for BBPS subclassification and determination of adequacy was 91% and 98%, respectively. The accuracy for BBPS 0, BBPS 1, BBPS 2, and BBPS 3 was 84%, 91%, 85%, and 96%, respectively.

CONCLUSION

We developed DCCNs capable of assessing bowel preparation adequacy and scoring with a high degree of accuracy. However, this algorithm will require further research to assess its efficacy in real-time colonoscopy.

Artificial Intelligence for Colonoscopy: Past, Present, and Future

During the past decades, many automated image analysis methods have been developed for colonoscopy. Real-time implementation of the most promising methods during colonoscopy has been tested in clinical trials, including several recent multi-center studies. All trials have shown results that may contribute to prevention of colorectal cancer. We summarize the past and present development of colonoscopy video analysis methods, focusing on two categories of artificial intelligence (AI) technologies used in clinical trials. These are (1) analysis and feedback for improving colonoscopy quality and (2) detection of abnormalities. Our survey includes methods that use traditional machine learning algorithms on carefully designed hand-crafted features as well as recent deep-learning methods. Lastly, we present the gap between current state-of-the-art technology and desirable clinical features and conclude with future directions of endoscopic AI technology development that will bridge the current gap.

Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy

Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N = 29,506 gastroscopy and N = 18,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities.

Artificial intelligence in colorectal cancer screening in patients with inflammatory bowel disease

Artificial intelligence (AI) is a branch of computer science that develops intelligent machines. In recent years, medicine has been contemplated with this recent modality to aid in the diagnosis of diseases in several specialties, including gastroenterology and gastrointestinal endoscopy. This new technology has superior ability to perform tasks mimicking human behavior and can identify possible pathological alterations, such as pre-malignant lesions and dysplasia, precursor lesions of colorectal cancer (CRC), and support medical decision-making. CRC is among the three most prevalent cancer types, and the second most common cause of cancer-related deaths worldwide; in addition, it is a leading cause of death in patients with inflammatory bowel disease (IBD). Patients with IBD tend to have greater inflammatory cell activity in the intestinal mucosa, which can favor cell proliferation and CRC development. AI can contribute to the detection of pre-neoplastic lesions in patients at risk of CRC development, such as those with extensive IBD or when additional CRC risk factors, such as smoking, are present. In fact, AI systems could improve all aspects of care related to both the detection of pre-malignant and malignant lesions and the screening of patients with IBD. In this review, we aimed to show the benefits and innovations of AI in the screening of CRC in patients with IBD. The promising applications of AI have the potential to revolutionize clinical practice and gastrointestinal endoscopy, especially in at-risk patients, such as those with IBD.

Assessment of the Risk of Severe Dengue Using Intrahost Viral Population in Dengue Virus Serotype 2 Patients via Machine Learning

Dengue virus, a positive-sense single-stranded RNA virus, continuously threatens human health. Although several criteria for evaluation of severe dengue have been recently established, the ability to prognose the risk of severe outcomes for dengue patients remains limited. Mutant spectra of RNA viruses, including single nucleotide variants (SNVs) and defective virus genomes (DVGs), contribute to viral virulence and growth. Here, we determine the potency of intrahost viral population in dengue patients with primary infection that progresses into severe dengue. A total of 65 dengue virus serotype 2 infected patients in primary infection including 17 severe cases were enrolled. We utilized deep sequencing to directly define the frequency of SNVs and detection times of DVGs in sera of dengue patients and analyzed their associations with severe dengue. Among the detected SNVs and DVGs, the frequencies of 9 SNVs and the detection time of 1 DVG exhibited statistically significant differences between patients with dengue fever and those with severe dengue. By utilizing the detected frequencies/times of the selected SNVs/DVG as features, the machine learning model showed high average with a value of area under the receiver operating characteristic curve (AUROC, 0.966 ± 0.064). The elevation of the frequency of SNVs at E (nucleotide position 995 and 2216), NS2A (nucleotide position 4105), NS3 (nucleotide position 4536, 4606), and NS5 protein (nucleotide position 7643 and 10067) and the detection times of the selected DVG that had a deletion junction in the E protein region (nucleotide positions of the junction: between 969 and 1022) increased the possibility of dengue patients for severe dengue. In summary, we demonstrated the detected frequencies/times of SNVs/DVG in dengue patients associated with severe disease and successfully utilized them to discriminate severe patients using machine learning algorithm. The identified SNVs and DVGs that are associated with severe dengue will expand our understanding of intrahost viral population in dengue pathogenesis.

Choledochoscopy: An update

Choledochoscopy, or cholangioscopy, is an endoscopic procedure for direct visualization within the biliary tract for diagnostic or therapeutic purposes. Since its conception in 1879, many variations and improvements are made to ensure relevance in diagnosing and managing a range of intrahepatic and extrahepatic biliary pathologies. This ranges from improved visual impression and optical guided biopsies of indeterminate biliary strictures and clinically indistinguishable pathologies to therapeutic uses in stone fragmentation and other ablative therapies. Furthermore, with the evolving understanding of biliary disorders, there are significant innovative ideas and techniques to fill this void, such as nuanced instances of biliary stenting and retrieving migrated ductal stents. With this in mind, we present a review of the current advancements in choledo-choscopy with new supporting evidence that further delineates the role of choledochoscopy in various diagnostic and therapeutic interventions, complications, limitations and put forth areas for further study.

Digital surgery for gastroenterological diseases

Advances in machine learning, computer vision and artificial intelligence methods, in combination with those in processing and cloud computing capability, portend the advent of true decision support during interventions in real-time and soon perhaps in automated surgical steps. Such capability, deployed alongside technology intraoperatively, is termed digital surgery and can be delivered without the need for high-end capital robotic investment. An area close to clinical usefulness right now harnesses advances in near infrared endolaparoscopy and fluorescence guidance for tissue characterisation through the use of biophysics-inspired algorithms. This represents a potential synergistic methodology for the deep learning methods currently advancing in ophthalmology, radiology, and recently gastroenterology via colonoscopy. As databanks of more general surgical videos are created, greater analytic insights can be derived across the operative spectrum of gastroenterological disease and operations (including instrumentation and operative step sequencing and recognition, followed over time by surgeon and instrument performance assessment) and linked to value-based outcomes. However, issues of legality, ethics and even morality need consideration, as do the limiting effects of monopolies, cartels and isolated data silos. Furthermore, the role of the surgeon, surgical societies and healthcare institutions in this evolving field needs active deliberation, as the default risks relegation to bystander or passive recipient. This editorial provides insight into this accelerating field by illuminating the near-future and next decade evolutionary steps towards widespread clinical integration for patient and societal benefit.

Artificial Intelligence in Capsule Endoscopy: A Practical Guide to Its Past and Future Challenges

Artificial intelligence (AI) has revolutionized the medical diagnostic process of various diseases. Since the manual reading of capsule endoscopy videos is a time-intensive, error-prone process, computerized algorithms have been introduced to automate this process. Over the past decade, the evolution of convolutional neural network (CNN) enabled AI to detect multiple lesions simultaneously with increasing accuracy and sensitivity. Difficulty in validating CNN performance and unique characteristics of capsule endoscopy images make computer-aided reading systems in capsule endoscopy still on a preclinical level. Although AI technology can be used as an auxiliary second observer in capsule endoscopy, it is expected that in the near future, it will effectively reduce the reading time and ultimately become an independent, integrated reading system.

Editors' Choice of Noteworthy Clinical Endoscopy Publications in the First Decade

This is a special review to celebrate the 10^th anniversary of Clinical Endoscopy. Each deputy editor has selected articles from one’s subspecialty that are significant in terms of the number of downloads, citations, and clinical importance. The articles included original articles, review articles, systematic reviews, and meta-analyses.

Applications of Artificial Intelligence for the Diagnosis of Gastrointestinal Diseases

The development of convolutional neural networks has achieved impressive advances of machine learning in recent years, leading to an increasing use of artificial intelligence (AI) in the field of gastrointestinal (GI) diseases. AI networks have been trained to differentiate benign from malignant lesions, analyze endoscopic and radiological GI images, and assess histological diagnoses, obtaining excellent results and high overall diagnostic accuracy. Nevertheless, there data are lacking on side effects of AI in the gastroenterology field, and high-quality studies comparing the performance of AI networks to health care professionals are still limited. Thus, large, controlled trials in real-time clinical settings are warranted to assess the role of AI in daily clinical practice. This narrative review gives an overview of some of the most relevant potential applications of AI for gastrointestinal diseases, highlighting advantages and main limitations and providing considerations for future development.

Computer-Aided Diagnosis of Diminutive Colorectal Polyps in Endoscopic Images: Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Background

Most colorectal polyps are diminutive and benign, especially those in the rectosigmoid colon, and the resection of these polyps is not cost-effective. Advancements in image-enhanced endoscopy have improved the optical prediction of colorectal polyp histology. However, subjective interpretability and inter- and intraobserver variability prohibits widespread implementation. The number of studies on computer-aided diagnosis (CAD) is increasing; however, their small sample sizes limit statistical significance.

Objective

This review aims to evaluate the diagnostic test accuracy of CAD models in predicting the histology of diminutive colorectal polyps by using endoscopic images.

Methods

Core databases were searched for studies that were based on endoscopic imaging, used CAD models for the histologic diagnosis of diminutive colorectal polyps, and presented data on diagnostic performance. A systematic review and diagnostic test accuracy meta-analysis were performed.

Results

Overall, 13 studies were included. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD models for the diagnosis of diminutive colorectal polyps (adenomatous or neoplastic vs nonadenomatous or nonneoplastic) were 0.96 (95% CI 0.93-0.97), 0.93 (95% CI 0.91-0.95), 0.87 (95% CI 0.76-0.93), and 87 (95% CI 38-201), respectively. The meta-regression analysis showed no heterogeneity, and no publication bias was detected. Subgroup analyses showed robust results. The negative predictive value of CAD models for the diagnosis of adenomatous polyps in the rectosigmoid colon was 0.96 (95% CI 0.95-0.97), and this value exceeded the threshold of the diagnosis and leave strategy.

Conclusions

CAD models show potential for the optical histological diagnosis of diminutive colorectal polyps via the use of endoscopic images.

Trial Registration

PROSPERO CRD42021232189; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=232189

Endoscopic diagnosis and treatment of gastric dysplasia and early cancer: Current evidence and what the future may hold

Gastric cancer accounts for a significant proportion of worldwide cancer-related morbidity and mortality. The well documented precancerous cascade provides an opportunity for clinicians to detect and treat gastric cancers at an endoscopically curable stage. In high prevalence regions such as Japan and Korea, this has led to the implementation of population screening programs. However, guidelines remain ambiguous in lower prevalence regions. In recent years, there have been many advances in the endoscopic diagnosis and treatment of early gastric cancer and precancerous lesions. More advanced endoscopic imaging has led to improved detection and characterization of gastric lesions as well as superior accuracy for delineation of margins prior to resection. In addition, promising early data on artificial intelligence in gastroscopy suggests a future role for this technology in maximizing the yield of advanced endoscopic imaging. Data on endoscopic resection (ER) are particularly robust in Japan and Korea, with high rates of curative ER and markedly reduced procedural morbidity. However, there is a shortage of data in other regions to support the applicability of protocols from these high prevalence countries. Future advances in endoscopic therapeutics will likely lead to further expansion of the current indications for ER, as both technology and proceduralist expertise continue to grow.

Use of Artificial Intelligence to Improve the Quality Control of Gastrointestinal Endoscopy

With the rapid development of science and technology, artificial intelligence (AI) systems are becoming ubiquitous, and their utility in gastroenteroscopy is beginning to be recognized. Digestive endoscopy is a conventional and reliable method of examining and diagnosing digestive tract diseases. However, with the increase in the number and types of endoscopy, problems such as a lack of skilled endoscopists and difference in the professional skill of doctors with different degrees of experience have become increasingly apparent. Most studies thus far have focused on using computers to detect and diagnose lesions, but improving the quality of endoscopic examination process itself is the basis for improving the detection rate and correctly diagnosing diseases. In the present study, we mainly reviewed the role of AI in monitoring systems, mainly through the endoscopic examination time, reducing the blind spot rate, improving the success rate for detecting high-risk lesions, evaluating intestinal preparation, increasing the detection rate of polyps, automatically collecting maps and writing reports. AI can even perform quality control evaluations for endoscopists, improve the detection rate of endoscopic lesions and reduce the burden on endoscopists.

Deep neural network approaches for detecting gastric polyps in endoscopic images

Gastrointestinal endoscopy is the primary method used for the diagnosis and treatment of gastric polyps. The early detection and removal of polyps is vitally important in preventing cancer development. Many studies indicate that a high workload can contribute to misdiagnosing gastric polyps, even for experienced physicians. In this study, we aimed to establish a deep learning-based computer-aided diagnosis system for automatic gastric polyp detection. A private gastric polyp dataset was generated for this purpose consisting of 2195 endoscopic images and 3031 polyp labels. Retrospective gastrointestinal endoscopy data from the Karadeniz Technical University, Farabi Hospital, were used in the study. YOLOv4, CenterNet, EfficientNet, Cross Stage ResNext50-SPP, YOLOv3, YOLOv3-SPP, Single Shot Detection, and Faster Regional CNN deep learning models were implemented and assessed to determine the most efficient model for precancerous gastric polyp detection. The dataset was split 70% and 30% for training and testing all the implemented models. YOLOv4 was determined to be the most accurate model, with an 87.95% mean average precision. We also evaluated all the deep learning models using a public gastric polyp dataset as the test data. The results show that YOLOv4 has significant potential applicability in detecting gastric polyps and can be used effectively in gastrointestinal CAD systems. Gastric Polyp Detection Process using Deep Learning with Private Dataset.

Role of Artificial Intelligence in Video Capsule Endoscopy

Capsule endoscopy (CE) has been increasingly utilised in recent years as a minimally invasive tool to investigate the whole gastrointestinal (GI) tract and a range of capsules are currently available for evaluation of upper GI, small bowel, and lower GI pathology. Although CE is undoubtedly an invaluable test for the investigation of small bowel pathology, it presents considerable challenges and limitations, such as long and laborious reading times, risk of missing lesions, lack of bowel cleansing score and lack of locomotion. Artificial intelligence (AI) seems to be a promising tool that may help improve the performance metrics of CE, and consequently translate to better patient care. In the last decade, significant progress has been made to apply AI in the field of endoscopy, including CE. Although it is certain that AI will find soon its place in day-to-day endoscopy clinical practice, there are still some open questions and barriers limiting its widespread application. In this review, we provide some general information about AI, and outline recent advances in AI and CE, issues around implementation of AI in medical practice and potential future applications of AI-aided CE.

Use of artificial intelligence in endoscopic ultrasound evaluation of pancreatic pathologies

The application of artificial intelligence (AI) using deep learning and machine learning approaches in modern medicine is rapidly expanding. Within the field of Gastroenterology, AI is being evaluated across a breadth of clinical and diagnostic applications including identification of pathology, differentiation of disease processes, and even automated procedure report generation. Many pancreatic pathologies can have overlapping features creating a diagnostic dilemma that provides a window for AI-assisted improvement in current evaluation and diagnosis, particularly using endoscopic ultrasound. This topic highlight will review the basics of AI, history of AI in gastrointestinal endoscopy, and prospects for AI in the evaluation of autoimmune pancreatitis, pancreatic ductal adenocarcinoma, chronic pancreatitis and intraductal papillary mucinous neoplasm.

Usefulness of artificial intelligence in gastric neoplasms

Recently, studies in many medical fields have reported that image analysis based on artificial intelligence (AI) can be used to analyze structures or features that are difficult to identify with human eyes. To diagnose early gastric cancer, related efforts such as narrow-band imaging technology are on-going. However, diagnosis is often difficult. Therefore, a diagnostic method based on AI for endoscopic imaging was developed and its effectiveness was confirmed in many studies. The gastric cancer diagnostic program based on AI showed relatively high diagnostic accuracy and could differentially diagnose non-neoplastic lesions including benign gastric ulcers and dysplasia. An AI system has also been developed that helps to predict the invasion depth of gastric cancer through endoscopic images and observe the stomach during endoscopy without blind spots. Therefore, if AI is used in the field of endoscopy, it is expected to aid in the diagnosis of gastric neoplasms and determine the application of endoscopic therapy by predicting the invasion depth.

Predicting Colorectal Cancer Occurrence in IBD

Patients with colonic inflammatory bowel disease (IBD) are at an increased risk of developing colorectal cancer (CRC), and are therefore enrolled into a surveillance programme aimed at detecting dysplasia or early cancer. Current surveillance programmes are guided by clinical, endoscopic or histological predictors of colitis-associated CRC (CA-CRC). We have seen great progress in our understanding of these predictors of disease progression, and advances in endoscopic technique and management, along with improved medical care, has been mirrored by the falling incidence of CA-CRC over the last 50 years. However, more could be done to improve our molecular understanding of CA-CRC progression and enable better risk stratification for patients with IBD. This review summarises the known risk factors associated with CA-CRC and explores the molecular landscape that has the potential to complement and optimise the existing IBD surveillance programme.

Artificial intelligence in brachytherapy: a summary of recent developments

Artificial intelligence (AI) applications, in the form of machine learning and deep learning, are being incorporated into practice in various aspects of medicine, including radiation oncology. Ample evidence from recent publications explores its utility and future use in external beam radiotherapy. However, the discussion on its role in brachytherapy is sparse. This article summarizes available current literature and discusses potential uses of AI in brachytherapy, including future directions. AI has been applied for brachytherapy procedures during almost all steps, starting from decision-making till treatment completion. AI use has led to improvement in efficiency and accuracy by reducing the human errors and saving time in certain aspects. Apart from direct use in brachytherapy, AI also contributes to contemporary advancements in radiology and associated sciences that can affect brachytherapy decisions and treatment. There is a renewal of interest in brachytherapy as a technique in recent years, contributed largely by the understanding that contemporary advances such as intensity modulated radiotherapy and stereotactic external beam radiotherapy cannot match the geometric gains and conformality of brachytherapy, and the integrated efforts of international brachytherapy societies to promote brachytherapy training and awareness. Use of AI technologies may consolidate it further by reducing human effort and time. Prospective validation over larger studies and incorporation of AI technologies for a larger patient population would help improve the efficiency and acceptance of brachytherapy. The enthusiasm favoring AI needs to be balanced against the short duration and quantum of experience with AI in limited patient subsets, need for constant learning and re-learning to train the AI algorithms, and the inevitability of humans having to take responsibility for the correctness and safety of treatments.

Artificial Intelligence in Colorectal Cancer Screening, Diagnosis and Treatment. A New Era

The development of artificial intelligence (AI) algorithms has permeated the medical field with great success. The widespread use of AI technology in diagnosing and treating several types of cancer, especially colorectal cancer (CRC), is now attracting substantial attention. CRC, which represents the third most commonly diagnosed malignancy in both men and women, is considered a leading cause of cancer-related deaths globally. Our review herein aims to provide in-depth knowledge and analysis of the AI applications in CRC screening, diagnosis, and treatment based on current literature. We also explore the role of recent advances in AI systems regarding medical diagnosis and therapy, with several promising results. CRC is a highly preventable disease, and AI-assisted techniques in routine screening represent a pivotal step in declining incidence rates of this malignancy. So far, computer-aided detection and characterization systems have been developed to increase the detection rate of adenomas. Furthermore, CRC treatment enters a new era with robotic surgery and novel computer-assisted drug delivery techniques. At the same time, healthcare is rapidly moving toward precision or personalized medicine. Machine learning models have the potential to contribute to individual-based cancer care and transform the future of medicine.

Use of Endoscopic Images in the Prediction of Submucosal Invasion of Gastric Neoplasms: Automated Deep Learning Model Development and Usability Study

BACKGROUND

In a previous study, we examined the use of deep learning models to classify the invasion depth (mucosa-confined versus submucosa-invaded) of gastric neoplasms using endoscopic images. The external test accuracy reached 77.3%. However, model establishment is labor intense, requiring high performance. Automated deep learning (AutoDL) models, which enable fast searching of optimal neural architectures and hyperparameters without complex coding, have been developed.

OBJECTIVE

The objective of this study was to establish AutoDL models to classify the invasion depth of gastric neoplasms. Additionally, endoscopist-artificial intelligence interactions were explored.

METHODS

The same 2899 endoscopic images that were employed to establish the previous model were used. A prospective multicenter validation using 206 and 1597 novel images was conducted. The primary outcome was external test accuracy. Neuro-T, Create ML Image Classifier, and AutoML Vision were used in establishing the models. Three doctors with different levels of endoscopy expertise were asked to classify the invasion depth of gastric neoplasms for each image without AutoDL support, with faulty AutoDL support, and with best performance AutoDL support in sequence.

RESULTS

The Neuro-T-based model reached 89.3% (95% CI 85.1%-93.5%) external test accuracy. For the model establishment time, Create ML Image Classifier showed the fastest time of 13 minutes while reaching 82.0% (95% CI 76.8%-87.2%) external test accuracy. While the expert endoscopist's decisions were not influenced by AutoDL, the faulty AutoDL misled the endoscopy trainee and the general physician. However, this was corrected by the support of the best performance AutoDL model. The trainee gained the most benefit from the AutoDL support.

CONCLUSIONS

AutoDL is deemed useful for the on-site establishment of customized deep learning models. An inexperienced endoscopist with at least a certain level of expertise can benefit from AutoDL support.

Challenges in Crohn's Disease Management after Gastrointestinal Cancer Diagnosis

Simple Summary

Crohn’s disease (CD) is a chronic inflammatory bowel disease affecting both young and elderly patients, involving the entire gastrointestinal tract from the mouth to anus. The chronic transmural inflammation can lead to several complications, among which gastrointestinal cancers represent one of the most life-threatening, with a higher risk of onset as compared to the general population. Moreover, diagnostic and therapeutic strategies in this subset of patients still represent a significant challenge for physicians. Thus, the aim of this review is to provide a comprehensive overview of the current evidence for an adequate diagnostic pathway and medical and surgical management of CD patients after gastrointestinal cancer onset.

Abstract

Crohn’s disease (CD) is a chronic inflammatory bowel disease with a progressive course, potentially affecting the entire gastrointestinal tract from mouth to anus. Several studies have shown an increased risk of both intestinal and extra-intestinal cancer in patients with CD, due to long-standing transmural inflammation and damage accumulation. The similarity of symptoms among CD, its related complications and the de novo onset of gastrointestinal cancer raises difficulties in the differential diagnosis. In addition, once a cancer diagnosis in CD patients is made, selecting the appropriate treatment can be particularly challenging. Indeed, both surgical and oncological treatments are not always the same as that of the general population, due to the inflammatory context of the gastrointestinal tract and the potential exacerbation of gastrointestinal symptoms of patients with CD; moreover, the overlap of the neoplastic disease could lead to adjustments in the pharmacological treatment of the underlying CD, especially with regard to immunosuppressive drugs. For these reasons, a case-by-case analysis in a multidisciplinary approach is often appropriate for the best diagnostic and therapeutic evaluation of patients with CD after gastrointestinal cancer onset.

Assessing the risk of dengue severity using demographic information and laboratory test results with machine learning

BACKGROUND

Dengue virus causes a wide spectrum of disease, which ranges from subclinical disease to severe dengue shock syndrome. However, estimating the risk of severe outcomes using clinical presentation or laboratory test results for rapid patient triage remains a challenge. Here, we aimed to develop prognostic models for severe dengue using machine learning, according to demographic information and clinical laboratory data of patients with dengue.

METHODOLOGY/PRINCIPAL FINDINGS

Out of 1,581 patients in the National Cheng Kung University Hospital with suspected dengue infections and subjected to NS1 antigen, IgM and IgG, and qRT-PCR tests, 798 patients including 138 severe cases were enrolled in the study. The primary target outcome was severe dengue. Machine learning models were trained and tested using the patient dataset that included demographic information and qualitative laboratory test results collected on day 1 when they sought medical advice. To develop prognostic models, we applied various machine learning methods, including logistic regression, random forest, gradient boosting machine, support vector classifier, and artificial neural network, and compared the performance of the methods. The artificial neural network showed the highest average discrimination area under the receiver operating characteristic curve (0.8324 ± 0.0268) and balance accuracy (0.7523 ± 0.0273). According to the model explainer that analyzed the contributions/co-contributions of the different factors, patient age and dengue NS1 antigenemia were the two most important risk factors associated with severe dengue. Additionally, co-existence of anti-dengue IgM and IgG in patients with dengue increased the probability of severe dengue.

CONCLUSIONS/SIGNIFICANCE

We developed prognostic models for the prediction of dengue severity in patients, using machine learning. The discriminative ability of the artificial neural network exhibited good performance for severe dengue prognosis. This model could help clinicians obtain a rapid prognosis during dengue outbreaks. However, the model requires further validation using external cohorts in future studies.

Emerging use of artificial intelligence in inflammatory bowel disease

Inflammatory bowel disease (IBD) is a complex, immune-mediated gastrointestinal disorder with ill-defined etiology, multifaceted diagnostic criteria, and unpredictable treatment response. Innovations in IBD diagnostics, including developments in genomic sequencing and molecular analytics, have generated tremendous interest in leveraging these large data platforms into clinically meaningful tools. Artificial intelligence, through machine learning facilitates the interpretation of large arrays of data, and may provide insight to improving IBD outcomes. While potential applications of machine learning models are vast, further research is needed to generate standardized models that can be adapted to target IBD populations. 

Techniques to integrate artificial intelligence systems with medical information in gastroenterology

Gastrointestinal (GI) endoscopy is the central element in contemporary gastroenterology as it provides direct evidence to guide targeted therapy. To increase the accuracy of GI endoscopy and to reduce human-related errors, artificial intelligence (AI) has been applied in GI endoscopy, which has been proved to be effective in diagnosing and treating numerous diseases. Therefore, we review current research on the efficacy of AI-assisted GI endoscopy in order to assess its functions, advantages and how the design can be improved.

The Impact of Artificial Intelligence in the Endoscopic Assessment of Premalignant and Malignant Esophageal Lesions: Present and Future

In the gastroenterology field, the impact of artificial intelligence was investigated for the purposes of diagnostics, risk stratification of patients, improvement in quality of endoscopic procedures and early detection of neoplastic diseases, implementation of the best treatment strategy, and optimization of patient prognosis. Computer-assisted diagnostic systems to evaluate upper endoscopy images have recently emerged as a supporting tool in endoscopy due to the risks of misdiagnosis related to standard endoscopy and different expertise levels of endoscopists, time-consuming procedures, lack of availability of advanced procedures, increasing workloads, and development of endoscopic mass screening programs. Recent research has tended toward computerized, automatic, and real-time detection of lesions, which are approaches that offer utility in daily practice. Despite promising results, certain studies might overexaggerate the diagnostic accuracy of artificial systems, and several limitations remain to be overcome in the future. Therefore, additional multicenter randomized trials and the development of existent database platforms are needed to certify clinical implementation. This paper presents an overview of the literature and the current knowledge of the usefulness of different types of machine learning systems in the assessment of premalignant and malignant esophageal lesions via conventional and advanced endoscopic procedures. This study makes a presentation of the artificial intelligence terminology and refers also to the most prominent recent research on computer-assisted diagnosis of neoplasia on Barrett’s esophagus and early esophageal squamous cell carcinoma, and prediction of invasion depth in esophageal neoplasms. Furthermore, this review highlights the main directions of future doctor–computer collaborations in which machines are expected to improve the quality of medical action and routine clinical workflow, thus reducing the burden on physicians.

Prediction of Submucosal Invasion for Gastric Neoplasms in Endoscopic Images Using Deep-Learning

Endoscopic resection is recommended for gastric neoplasms confined to mucosa or superficial submucosa. The determination of invasion depth is based on gross morphology assessed in endoscopic images, or on endoscopic ultrasound. These methods have limited accuracy and pose an inter-observer variability. Several studies developed deep-learning (DL) algorithms classifying invasion depth of gastric cancers. Nevertheless, these algorithms are intended to be used after definite diagnosis of gastric cancers, which is not always feasible in various gastric neoplasms. This study aimed to establish a DL algorithm for accurately predicting submucosal invasion in endoscopic images of gastric neoplasms. Pre-trained convolutional neural network models were fine-tuned with 2899 white-light endoscopic images. The prediction models were subsequently validated with an external dataset of 206 images. In the internal test, the mean area under the curve discriminating submucosal invasion was 0.887 (95% confidence interval: 0.849–0.924) by DenseNet−161 network. In the external test, the mean area under the curve reached 0.887 (0.863–0.910). Clinical simulation showed that 6.7% of patients who underwent gastrectomy in the external test were accurately qualified by the established algorithm for potential endoscopic resection, avoiding unnecessary operation. The established DL algorithm proves useful for the prediction of submucosal invasion in endoscopic images of gastric neoplasms.