Novel Developments in Endoscopic Mucosal Imaging

Using reflectometry to minimize the dependence of fluorescence intensity on optical absorption and scattering

The total diffuse reflectance RT and the effective attenuation coefficient µeff of an optically diffuse medium map uniquely onto its absorption and reduced scattering coefficients. Using this premise, we developed a methodology where RT and the slope of the logarithmic spatially resolved reflectance, a quantity related to µeff, are the inputs of a look-up table to correct the dependence of fluorescent signals on the media's optical properties. This methodology does not require an estimation of the medium's optical property, avoiding elaborate simulations and their errors to offer accurate and fast corrections. The experimental demonstration of our method yielded a mean relative error in fluorophore concentrations of less than 4% over a wide range of optical property variations. We discuss how the method developed can be employed to improve image fidelity and fluorochrome quantification in fluorescence molecular imaging clinical applications.

In Vivo Staging the Progression of Colitis and Associated Cancer by Concurrent Microimaging of Key Biomarkers

Currently colorectal cancer (CRC) staging (colitis, adenoma, and carcinoma) mainly relies on ex vivo pathologic analysis requiring an invasive surgical process with limited sample collection and increased metastatic risk. Thus, in vivo noninvasive pathological diagnosis is extremely demanded. By verifying the samples of clinical patients and CRC mouse models, it was found that vascular endothelial growth factor receptor 2 (VEGFR2) was barely expressed in the colitis stage and only appeared in adenoma and carcinoma stages with obvious elevation, while prostaglandin E receptor 4 (PTGER4) could be observed from colitis to adenoma and carcinoma stages with a gradient increase of expression. VEGFR2 and PTGER4 were further chosen as key biomarkers for molecular pathological diagnosis in vivo and corresponding molecular probes were constructed. The feasibility of in vivo noninvasive CRC staging by concurrent microimaging of dual biomarkers using confocal laser endoscopy (CLE) was verified in CRC mouse models and further confirmed by ex vivo pathological analysis. In vivo CLE imaging exhibited the correlation of severe colonic crypt structural alteration with a higher biomarker expression in adenoma and carcinoma stages. This strategy shows promise in benefiting patients undergoing CRC progression with in-time, noninvasive, and precise pathological staging, thus providing valuable guidance for selecting therapeutic strategies.

Development of Advanced Imaging and Molecular Imaging for Barrett's Neoplasia

Barrett esophagus (BE) is a precursor to a life-threatening esophageal adenocarcinoma (EAC). Surveillance endoscopy with random biopsies is recommended for early intervention against EAC, but its adherence in the clinical setting is poor. Dysplastic lesions with flat architecture and patchy distribution in BE are hardly detected by high-resolution endoscopy, and the surveillance protocol entails issues of time and labor and suboptimal interobserver agreement for diagnosing dysplasia. Therefore, the development of advanced imaging technologies is necessary for Barrett's surveillance. Recently, non-endoscopic or endoscopic technologies, such as cytosponge, endocytoscopy, confocal laser endomicroscopy, autofluorescence imaging, and optical coherence tomography/volumetric laser endomicroscopy, were developed, but most of them are not clinically available due to the limited view field, expense of the equipment, and significant time for the learning curve. Another strategy is focused on the development of molecular biomarkers, which are also not ready to use. However, a combination of advanced imaging techniques together with specific biomarkers is expected to identify morphological abnormalities and biological disorders at an early stage in the surveillance. Here, we review recent developments in advanced imaging and molecular imaging for Barrett's neoplasia. Further developments in multiple biomarker panels specific for Barrett's HGD/EAC include wide-field imaging systems for targeting 'red flags', a high-resolution imaging system for optical biopsy, and a computer-aided diagnosis system with artificial intelligence, all of which enable a real-time and accurate diagnosis of dysplastic BE in Barrett's surveillance and provide information for precision medicine.

Artificial intelligence in endoscopy: More than what meets the eye in screening colonoscopy and endosonographic evaluation of pancreatic lesions

Artificial intelligence (AI)-based tools have ushered in a new era of innovation in the field of gastrointestinal (GI) endoscopy. Despite vast improvements in endoscopic techniques and equipment, diagnostic endoscopy remains heavily operator-dependent, in particular, colonoscopy and endoscopic ultrasound (EUS). Recent reports have shown that as much as 25% of colonic adenomas may be missed at colonoscopy. This can result in an increased incidence of interval colon cancer. Similarly, EUS has been shown to have high inter-observer variability, overlap in diagnoses with a relatively low specificity for pancreatic lesions. Our understanding of Machine-learning (ML) techniques in AI have evolved over the last decade and its application in AI–based tools for endoscopic detection and diagnosis is being actively investigated at several centers. ML is an aspect of AI that is based on neural networks, and is widely used for image classification, object detection, and semantic segmentation which are key functional aspects of AI-related computer aided diagnostic systems. In this review, current status and limitations of ML, specifically for adenoma detection and endosonographic diagnosis of pancreatic lesions, will be summarized from existing literature. This will help to better understand its role as viewed through the prism of real world application in the field of GI endoscopy.

Today’s Mistakes and Tomorrow’s Wisdom in Endoscopic Imaging of Barrett’s Esophagus

<b><i>Background:</i></b> Esophageal adenocarcinoma (EAC) is one of the main causes of cancer-related deaths worldwide and its incidence is rising. Barrett’s esophagus (BE) can develop low- and high-grade dysplasia which can progress to EAC overtime. The golden standard to detect dysplastic BE (DBE) or EAC is surveillance with high-definition white-light endoscopy (HD-WLE) and random biopsies according to the Seattle protocol. However, this method is time-consuming and associated with a remarkable miss rate. Therefore, there is great need for the development of novel reliable techniques to optimize surveillance strategies and improve detection rates. <b><i>Summary:</i></b> Optical chromoendoscopy (OC) techniques like narrow-band imaging have shown improved detection of DBE and EAC compared to HD-WLE and random biopsies. Most recent OC techniques, including the iSCAN optical enhancement system and linked color imaging, showed improved characterization of DBE and EAC retrospectively. Fluorescence molecular endoscopy (FME) presented promising results to highlight DBE and EAC. Moreover, with the establishment of well-performing delineation computer-aided detection (CAD) algorithms and the first real-time CAD system for EAC, we expect clinical application of CAD in the near future. <b><i>Key Messages:</i></b> Despite impressive progress made in the development of advanced endoscopic techniques, combined HD-WLE/OC followed by random biopsies remains the golden standard for BE surveillance. Surveillance depends on appropriate mucosal cleansing, sufficient inspection time, and competence of the performing gastroenterologist to improve detection of EAC. In addition, to facilitate the clinical implementation of advanced endoscopic techniques, multicenter prospective clinical studies are demanded for OC and FME. Meanwhile, further optimization of CAD algorithms, the education of gastroenterologists, and analysis of the interaction between the clinician and the computer should be performed.

Molecular Imaging of Ulex Europaeus Agglutinin in Colorectal Cancer Using Confocal Laser Endomicroscopy (With Video)

Background and Study Aims

Previous studies have identified that colorectal cancer has different fucosylation levels compared to the normal colon. Ulex europaeus agglutinin-I (UEA-I), which specifically combines with α1-2 fucose glycan, is usually used to detect fucosylation levels. Therefore, we used confocal laser endomicroscopy (CLE) to investigate fluorescently labeled UEA-Fluorescein isothiocyanate (FITC) for detecting colonic cancer.

Patients and Methods

We stained frozen mouse colon tissue sections of normal, adenoma, and adenocarcinoma species with UEA-FITC to detect fucosylation levels in different groups. White light endoscopy and endocytoscopy were first used to detect the lesions. The UEA-FITC was then stained in the mice and human colon tissues in vitro. The CLE was used to detect the UEA-FITC levels of the corresponding lesions, and videos were recorded for quantitation analysis. The diagnostic accuracy of UEA-FITC using CLE was evaluated in terms of sensitivity and specificity.

Results

The UEA expression level in colorectal cancer was lower than that in normal intestinal epithelium. The fluorescence intensity ratio of UEA-FITC in colorectal cancer was significantly lower than that in normal tissue detected by CLE in both mice and humans. The combination of UEA-FITC and CLE presented a good diagnostic accuracy with a sensitivity of 95.6% and a specificity of 97.7% for detecting colorectal cancer. The positive and negative predictive values were 91.6% and 95.6%, respectively. Overall, 95.6% of the sites were correctly classified by CLE.

Conclusions

We developed a new imaging strategy to improve the diagnostic efficacy of CLE by using UEA-FITC.

The use of deep learning on endoscopic images to assess the response of rectal cancer after chemoradiation

Background

Accurate response evaluation is necessary to select complete responders (CRs) for a watch-and-wait approach. Deep learning may aid in this process, but so far has never been evaluated for this purpose. The aim was to evaluate the accuracy to assess response with deep learning methods based on endoscopic images in rectal cancer patients after neoadjuvant therapy.

Methods

Rectal cancer patients diagnosed between January 2012 and December 2015 and treated with neoadjuvant (chemo)radiotherapy were retrospectively selected from a single institute. All patients underwent flexible endoscopy for response evaluation. Diagnostic performance (accuracy, area under the receiver operator characteristics curve (AUC), positive- and negative predictive values, sensitivities and specificities) of different open accessible deep learning networks was calculated. Reference standard was histology after surgery, or long-term outcome (>2 years of follow-up) in a watch-and-wait policy.

Results

226 patients were included for the study (117(52%) were non-CRs; 109(48%) were CRs). The accuracy, AUC, positive- and negative predictive values, sensitivity and specificity of the different models varied from 0.67–0.75%, 0.76–0.83%, 67–74%, 70–78%, 68–79% to 66–75%, respectively. Overall, EfficientNet-B2 was the most successful model with the highest diagnostic performance.

Conclusions

This pilot study shows that deep learning has a modest accuracy (AUCs 0.76-0.83). This is not accurate enough for clinical decision making, and lower than what is generally reported by experienced endoscopists. Deep learning models can however be further improved and may become useful to assist endoscopists in evaluating the response. More well-designed prospective studies are required.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-021-08685-7.

The evaluation of follow-up strategies of watch-and-wait patients with a complete response after neoadjuvant therapy in rectal cancer

AIM

Many of the current follow-up schedules in a watch-and-wait approach include very frequent MRI and endoscopy examinations to ensure early detection of local regrowth (LR). The aim of this study was to analyse the occurrence and detection of LR in a watch-and-wait cohort and to suggest a more efficient follow-up schedule.

METHOD

Rectal cancer patients with a clinical complete response after neoadjuvant therapy were prospectively and retrospectively included in a multicentre watch-and-wait registry between 2004 and 2018, with the current follow-up schedule with 3-monthly endoscopy and MRI in the first year and 6 monthly thereafter. A theoretical comparison was constructed for the detection of LR in the current follow-up schedule against four other hypothetical schedules.

RESULTS

In all, 50/304 (16%) of patients developed a LR. The majority (98%) were detected at ≤2 years, located in the lumen (94%) and were visible on endoscopy (88%). The theoretical comparison of the different hypothetical schedules suggests that the optimal follow-up schedule should focus on the first 2 years with 3-monthly endoscopy and 3-6 monthly MRI. Longer intervals in the first 2 years will cause delays in diagnosis of LR ranging from 0 to 5 months. After 2 years, increasing the interval from 6 to 12 months did not cause important delays.

CONCLUSION

The optimal follow-up schedule for a watch-and-wait policy in patients with a clinical complete response after chemoradiation for rectal cancer should include frequent endoscopy and to a lesser degree MRI in the first 2 years. Longer intervals, up to 12 months, can be considered after 2 years.

Clinically Available Optical Imaging Technologies in Endoscopic Lesion Detection: Current Status and Future Perspective

Endoscopic optical imaging technologies for the detection and evaluation of dysplasia and early cancer have made great strides in recent decades. With the capacity of in vivo early detection of subtle lesions, they allow modern endoscopists to provide accurate and effective optical diagnosis in real time. This review mainly analyzes the current status of clinically available endoscopic optical imaging techniques, with emphasis on the latest updates of existing techniques. We summarize current coverage of these technologies in major hospital departments such as gastroenterology, urology, gynecology, otolaryngology, pneumology, and laparoscopic surgery. In order to promote a broader understanding, we further cover the underlying principles of these technologies and analyze their performance. Moreover, we provide a brief overview of future perspectives in related technologies, such as computer-assisted diagnosis (CAD) algorithms dealing with exploring endoscopic video data. We believe all these efforts will benefit the healthcare of the community, help endoscopists improve the accuracy of diagnosis, and relieve patients' suffering.

Endoscopic imaging in inflammatory bowel disease: current developments and emerging strategies

INTRODUCTION

Developments in enhanced and magnified endoscopy have signified major advances in endoscopic imaging of ileocolonic pathology in inflammatory bowel disease (IBD). Artificial intelligence is increasingly being used to augment the benefits of these advanced techniques. Nevertheless, treatment of IBD patients is frustrated by high rates of non-response to therapy, while delayed detection and failures to detect neoplastic lesions impede successful surveillance. A possible solution is offered by molecular imaging, which adds functional imaging data to mucosal morphology assessment through visualizing biological parameters. Other label-free modalities enable visualization beyond the mucosal surface without the need of tracers.

AREAS COVERED

A literature search up to May 2020 was conducted in PubMed/MEDLINE in order to find relevant articles that involve the (pre-)clinical application of high-definition white light endoscopy, chromoendoscopy, artificial intelligence, confocal laser endomicroscopy, endocytoscopy, molecular imaging, optical coherence tomography, and Raman spectroscopy in IBD.

EXPERT OPINION

Enhanced and magnified endoscopy have enabled an improved assessment of the ileocolonic mucosa. Implementing molecular imaging in endoscopy could overcome the remaining clinical challenges by giving practitioners a real-time in vivo view of targeted biomarkers. Label-free modalities could help optimize the endoscopic assessment of mucosal healing and dysplasia detection in IBD patients.

Advanced Imaging and Sampling in Barrett's Esophagus: Artificial Intelligence to the Rescue?

Because the current Barrett's esophagus (BE) surveillance protocol suffers from sampling error of random biopsies and a high miss-rate of early neoplastic lesions, many new endoscopic imaging and sampling techniques have been developed. None of these techniques, however, have significantly increased the diagnostic yield of BE neoplasia. In fact, these techniques have led to an increase in the amount of visible information, yet endoscopists and pathologists inevitably suffer from variations in intra- and interobserver agreement. Artificial intelligence systems have the potential to overcome these endoscopist-dependent limitations.

Polypectomy Techniques

Polyps in the upper and lower gastrointestinal tract can be premalignant or malignant lesions that can be treated endoscopically in early stages to prevent morbidity and more invasive procedures. This article critically reviews the techniques available and provides recommendations for endoscopic polypectomy.

Machine learning in GI endoscopy: practical guidance in how to interpret a novel field

There has been a vast increase in GI literature focused on the use of machine learning in endoscopy. The relative novelty of this field poses a challenge for reviewers and readers of GI journals. To appreciate scientific quality and novelty of machine learning studies, understanding of the technical basis and commonly used techniques is required. Clinicians often lack this technical background, while machine learning experts may be unfamiliar with clinical relevance and implications for daily practice. Therefore, there is an increasing need for a multidisciplinary, international evaluation on how to perform high-quality machine learning research in endoscopy. This review aims to provide guidance for readers and reviewers of peer-reviewed GI journals to allow critical appraisal of the most relevant quality requirements of machine learning studies. The paper provides an overview of common trends and their potential pitfalls and proposes comprehensive quality requirements in six overarching themes: terminology, data, algorithm description, experimental setup, interpretation of results and machine learning in clinical practice.

A Facile and Reproducible Synthesis of Near-Infrared Fluorescent Conjugates with Small Targeting Molecules for Microbial Infection Imaging

Optical imaging of microbial infections, based on the detection of targeted fluorescent probes, offers high sensitivity and resolution with a relatively simple and portable setup. As the absorbance of near-infrared (NIR) light by human tissues is minimal, using respective tracers, such as IRdye800CW, enables imaging deeper target sites in the body. Herein, we present a general strategy for the conjugation of IRdye800CW and IRdye700DX to small molecules (vancomycin and amphotericin B) to provide conjugates targeted toward bacterial and fungal infections for optical imaging and photodynamic therapy. In particular, we present how the use of coupling agents (such as HBTU or HATU) leads to high yields (over 50%) in the reactions of amines and IRDye-NHS esters and how precipitation can be used as a convenient purification strategy to remove excess of the targeting molecule after the reaction. The high selectivity of the synthesized model compound Vanco-800CW has been proven in vitro, and the development of analogous agents opens up new possibilities for diagnostic and theranostic purposes. In times of increasing microbial resistance, this research gives us access to a platform of new fluorescent tracers for the imaging of infections, enabling early diagnosis and respective treatment.

Personalizing Treatment in IBD: Hype or Reality in 2020? Can We Predict Response to Anti-TNF?

The advent of anti-TNF agents as the first approved targeted therapy in the treatment of inflammatory bowel disease (IBD) patients has made a major impact on our existing therapeutic algorithms. They have not only been approved for induction and maintenance treatment in IBD patients, but have also enabled us to define and achieve novel therapeutic outcomes, such as combination of clinical symptom control and endoscopic remission, as well as mucosal healing. Nevertheless, approximately one third of treated patients do not respond to initiated anti-TNF therapy and these treatments are associated with sometimes severe systemic side-effects. There is therefore the currently unmet clinical need do establish predictive markers of response to identify the subgroup of IBD patients, that have a heightened probability of response. There have so far been approaches from different fields of IBD research, to descry markers that would empower us to apply TNF-inhibitors in a more rational manner. These markers encompass findings from disease-related and clinical factors, pharmacokinetics, biochemical markers, blood and stool derived parameters, pharmacogenomics, microbial species, metabolic compounds, and mucosal factors. Furthermore, changes in the intestinal immune cell composition in response to therapeutic pressure of anti-TNF treatment have recently been implicated in the process of molecular resistance to these drugs. Insights into factors that determine resistance to anti-TNF therapy give reasonable hope, that a more targeted approach can then be utilized in these non-responders. Here, IL-23 could be identified as one of the key factors determining resistance to TNF-inhibitors. Growing insights into the molecular mechanism of action of TNF-inhibitors might also enable us to derive critical molecular markers that not only mediate the clinical effects of anti-TNF therapy, but which level of expression might also correlate with its therapeutic efficacy. In this narrative review, we present an overview of currently identified possible predictive markers for successful anti-TNF therapy and discuss identified molecular pathways that drive resistance to these substances. We will also point out the necessity and difficulty of developing and validating a diagnostic marker concerning clinically relevant outcome parameters, before they can finally enter daily clinical practice and enable a more personalized therapeutic approach.

Deep-Learning System Detects Neoplasia in Patients With Barrett's Esophagus With Higher Accuracy Than Endoscopists in a Multistep Training and Validation Study With Benchmarking

BACKGROUND & AIMS

We aimed to develop and validate a deep-learning computer-aided detection (CAD) system, suitable for use in real time in clinical practice, to improve endoscopic detection of early neoplasia in patients with Barrett's esophagus (BE).

METHODS

We developed a hybrid ResNet-UNet model CAD system using 5 independent endoscopy data sets. We performed pretraining using 494,364 labeled endoscopic images collected from all intestinal segments. Then, we used 1704 unique esophageal high-resolution images of rigorously confirmed early-stage neoplasia in BE and nondysplastic BE, derived from 669 patients. System performance was assessed by using data sets 4 and 5. Data set 5 was also scored by 53 general endoscopists with a wide range of experience from 4 countries to benchmark CAD system performance. Coupled with histopathology findings, scoring of images that contained early-stage neoplasia in data sets 2-5 were delineated in detail for neoplasm position and extent by multiple experts whose evaluations served as the ground truth for segmentation.

RESULTS

The CAD system classified images as containing neoplasms or nondysplastic BE with 89% accuracy, 90% sensitivity, and 88% specificity (data set 4, 80 patients and images). In data set 5 (80 patients and images) values for the CAD system vs those of the general endoscopists were 88% vs 73% accuracy, 93% vs 72% sensitivity, and 83% vs 74% specificity. The CAD system achieved higher accuracy than any of the individual 53 nonexpert endoscopists, with comparable delineation performance. CAD delineations of the area of neoplasm overlapped with those from the BE experts in all detected neoplasia in data sets 4 and 5. The CAD system identified the optimal site for biopsy of detected neoplasia in 97% and 92% of cases (data sets 4 and 5, respectively).

CONCLUSIONS

We developed, validated, and benchmarked a deep-learning computer-aided system for primary detection of neoplasia in patients with BE. The system detected neoplasia with high accuracy and near-perfect delineation performance. The Netherlands National Trials Registry, Number: NTR7072.

Real-time colorectal cancer diagnosis using PR-OCT with deep learning

Prior reports have shown optical coherence tomography (OCT) can differentiate normal colonic mucosa from neoplasia, potentially offering an alternative technique to endoscopic biopsy - the current gold-standard colorectal cancer screening and surveillance modality. To help clinical translation limited by processing the large volume of generated data, we designed a deep learning-based pattern recognition (PR) OCT system that automates image processing and provides accurate diagnosis potentially in real-time. Method: OCT is an emerging imaging technique to obtain 3-dimensional (3D) "optical biopsies" of biological samples with high resolution. We designed a convolutional neural network to capture the structure patterns in human colon OCT images. The network is trained and tested using around 26,000 OCT images acquired from 20 tumor areas, 16 benign areas, and 6 other abnormal areas. Results: The trained network successfully detected patterns that identify normal and neoplastic colorectal tissue. Experimental diagnoses predicted by the PR-OCT system were compared to the known histologic findings and quantitatively evaluated. A sensitivity of 100% and specificity of 99.7% can be reached. Further, the area under the receiver operating characteristic (ROC) curves (AUC) of 0.998 is achieved. Conclusions: Our results demonstrate that PR-OCT can be used to give an accurate real-time computer-aided diagnosis of colonic neoplastic mucosa. Future development of this system as an "optical biopsy" tool to assist doctors in real-time for early mucosal neoplasms screening and treatment evaluation following initial oncologic therapy is planned.

Deep principal dimension encoding for the classification of early neoplasia in Barrett's Esophagus with volumetric laser endomicroscopy

Barrett cancer is a treatable disease when detected at an early stage. However, current screening protocols are often not effective at finding the disease early. Volumetric Laser Endomicroscopy (VLE) is a promising new imaging tool for finding dysplasia in Barrett's esophagus (BE) at an early stage, by acquiring cross-sectional images of the microscopic structure of BE up to 3-mm deep. However, interpretation of VLE scans is difficult for medical doctors due to both the size and subtlety of the gray-scale data. Therefore, algorithms that can accurately find cancerous regions are very valuable for the interpretation of VLE data. In this study, we propose a fully-automatic multi-step Computer-Aided Detection (CAD) algorithm that optimally leverages the effectiveness of deep learning strategies by encoding the principal dimension in VLE data. Additionally, we show that combining the encoded dimensions with conventional machine learning techniques further improves results while maintaining interpretability. Furthermore, we train and validate our algorithm on a new histopathologically validated set of in-vivo VLE snapshots. Additionally, an independent test set is used to assess the performance of the model. Finally, we compare the performance of our algorithm against previous state-of-the-art systems. With the encoded principal dimension, we obtain an Area Under the Curve (AUC) and F₁ score of 0.93 and 87.4% on the test set respectively. We show this is a significant improvement compared to the state-of-the-art of 0.89 and 83.1%, respectively, thereby demonstrating the effectiveness of our approach.

Molecular endoscopic imaging: the future is bright

The prediction and final survival rate of gastrointestinal cancers are dependent on the stage of disease. The ideal would be to detect those gastrointestinal lesions at early stage or even premalignant forms which are difficult to detect by conventional endoscopy with white light optical imaging as they show minimum or no changes in morphological characteristics and are thus left untreated. The introduction of molecular imaging has greatly changed the pattern for detecting gastrointestinal lesions from purely macroscopic structural imaging to the molecular level. It allows microscopic examination of the gastrointestinal mucosa with endoscopy after the topical or systemic application of molecular probes. In recent years, major advancements in endoscopic instruments and specific molecular probes have been achieved. This review focuses on the current status of endoscopic imaging and highlights the application of molecular imaging in gastrointestinal and hepatic disease in the context of diagnosis and therapy based on recently published literature in this field. We also discuss the challenges of molecular endoscopic imaging, its future directions and potential that could have a tremendous impact on endoscopic research and clinical practice in future.