Gorilla in the room: Even experts can miss polyps at colonoscopy and how AI helps complex visual perception tasks

Efficient artificial intelligence-based assessment of the gastroesophageal valve with Hill classification through active learning

Standardized assessment of the gastroesophageal valve during endoscopy, attainable via the Hill classification, is important for clinical assessment and therapeutic decision making. The Hill classification is associated with the presence of hiatal hernia (HH), a common endoscopic finding connected to gastro-esophageal reflux disease. A novel efficient medical artificial intelligence (AI) training pipeline using active learning (AL) is designed. We identified 21,970 gastroscopic images as training data and used our AL to train a model for predicting the Hill classification and detecting HH. Performance of the AL and traditionally trained models were evaluated on an external expert-annotated image collection. The AL model achieved accuracy of 76%. A traditionally trained model with 125% more training data achieved 77% accuracy. Furthermore, the AL model achieved higher precision than the traditional one for rare classes, with 0.54 versus 0.39 (p < 0.05) for grade 3 and 0.72 versus 0.61 (p < 0.05) for grade 4. In detecting HH, the AL model achieved 94% accuracy, 0.72 precision and 0.74 recall. Our AL pipeline is more efficient than traditional methods in training AI for endoscopy.

Assessing clinical efficacy of polyp detection models using open-access datasets

Background

Ensuring accurate polyp detection during colonoscopy is essential for preventing colorectal cancer (CRC). Recent advances in deep learning-based computer-aided detection (CADe) systems have shown promise in enhancing endoscopists' performances. Effective CADe systems must achieve high polyp detection rates from the initial seconds of polyp appearance while maintaining low false positive (FP) detection rates throughout the procedure.

Method

We integrated four open-access datasets into a unified platform containing over 340,000 images from various centers, including 380 annotated polyps, with distinct data splits for comprehensive model development and benchmarking. The REAL-Colon dataset, comprising 60 full-procedure colonoscopy videos from six centers, is used as the fifth dataset of the platform to simulate clinical conditions for model evaluation on unseen center data. Performance assessment includes traditional object detection metrics and new metrics that better meet clinical needs. Specifically, by defining detection events as sequences of consecutive detections, we compute per-polyp recall at early detection stages and average per-patient FPs, enabling the generation of Free-Response Receiver Operating Characteristic (FROC) curves.

Results

Using YOLOv7, we trained and tested several models across the proposed data splits, showcasing the robustness of our open-access platform for CADe system development and benchmarking. The introduction of new metrics allows for the optimization of CADe operational parameters based on clinically relevant criteria, such as per-patient FPs and early polyp detection. Our findings also reveal that omitting full-procedure videos leads to non-realistic assessments and that detecting small polyp bounding boxes poses the greatest challenge.

Conclusion

This study demonstrates how newly available open-access data supports ongoing research progress in environments that closely mimic clinical settings. The introduced metrics and FROC curves illustrate CADe clinical efficacy and can aid in tuning CADe hyperparameters.

Enhancing human-AI collaboration: The case of colonoscopy

Diagnostic errors impact patient health and healthcare costs. Artificial Intelligence (AI) shows promise in mitigating this burden by supporting Medical Doctors in decision-making. However, the mere display of excellent or even superhuman performance by AI in specific tasks does not guarantee a positive impact on medical practice. Effective AI assistance should target the primary causes of human errors and foster effective collaborative decision-making with human experts who remain the ultimate decision-makers. In this narrative review, we apply these principles to the specific scenario of AI assistance during colonoscopy. By unraveling the neurocognitive foundations of the colonoscopy procedure, we identify multiple bottlenecks in perception, attention, and decision-making that contribute to diagnostic errors, shedding light on potential interventions to mitigate them. Furthermore, we explored how existing AI devices fare in clinical practice and whether they achieved an optimal integration with the human decision-maker. We argue that to foster optimal Human-AI collaboration, future research should expand our knowledge of factors influencing AI's impact, establish evidence-based cognitive models, and develop training programs based on them. These efforts will enhance human-AI collaboration, ultimately improving diagnostic accuracy and patient outcomes. The principles illuminated in this review hold more general value, extending their relevance to a wide array of medical procedures and beyond.

Artificial Intelligence-assisted colonoscopy and colorectal cancer screening: Where are we going?

Colorectal cancer is a significant global health concern, necessitating effective screening strategies to reduce its incidence and mortality rates. Colonoscopy plays a crucial role in the detection and removal of colorectal neoplastic precursors. However, there are limitations and variations in the performance of endoscopists, leading to missed lesions and suboptimal outcomes. The emergence of artificial intelligence (AI) in endoscopy offers promising opportunities to improve the quality and efficacy of screening colonoscopies. In particular, AI applications, including computer-aided detection (CADe) and computer-aided characterization (CADx), have demonstrated the potential to enhance adenoma detection and optical diagnosis accuracy. Additionally, AI-assisted quality control systems aim to standardize the endoscopic examination process. This narrative review provides an overview of AI principles and discusses the current knowledge on AI-assisted endoscopy in the context of screening colonoscopies. It highlights the significant role of AI in improving lesion detection, characterization, and quality assurance during colonoscopy. However, further well-designed studies are needed to validate the clinical impact and cost-effectiveness of AI-assisted colonoscopy before its widespread implementation.

REAL-Colon: A dataset for developing real-world AI applications in colonoscopy

Detection and diagnosis of colon polyps are key to preventing colorectal cancer. Recent evidence suggests that AI-based computer-aided detection (CADe) and computer-aided diagnosis (CADx) systems can enhance endoscopists' performance and boost colonoscopy effectiveness. However, most available public datasets primarily consist of still images or video clips, often at a down-sampled resolution, and do not accurately represent real-world colonoscopy procedures. We introduce the REAL-Colon (Real-world multi-center Endoscopy Annotated video Library) dataset: a compilation of 2.7 M native video frames from sixty full-resolution, real-world colonoscopy recordings across multiple centers. The dataset contains 350k bounding-box annotations, each created under the supervision of expert gastroenterologists. Comprehensive patient clinical data, colonoscopy acquisition information, and polyp histopathological information are also included in each video. With its unprecedented size, quality, and heterogeneity, the REAL-Colon dataset is a unique resource for researchers and developers aiming to advance AI research in colonoscopy. Its openness and transparency facilitate rigorous and reproducible research, fostering the development and benchmarking of more accurate and reliable colonoscopy-related algorithms and models.

A Review of the Technology, Training, and Assessment Methods for the First Real-Time AI-Enhanced Medical Device for Endoscopy

Artificial intelligence (AI) has the potential to assist in endoscopy and improve decision making, particularly in situations where humans may make inconsistent judgments. The performance assessment of the medical devices operating in this context is a complex combination of bench tests, randomized controlled trials, and studies on the interaction between physicians and AI. We review the scientific evidence published about GI Genius, the first AI-powered medical device for colonoscopy to enter the market, and the device that is most widely tested by the scientific community. We provide an overview of its technical architecture, AI training and testing strategies, and regulatory path. In addition, we discuss the strengths and limitations of the current platform and its potential impact on clinical practice. The details of the algorithm architecture and the data that were used to train the AI device have been disclosed to the scientific community in the pursuit of a transparent AI. Overall, the first AI-enabled medical device for real-time video analysis represents a significant advancement in the use of AI for endoscopies and has the potential to improve the accuracy and efficiency of colonoscopy procedures.