Pre-Cancerous Stomach Lesion Detections with Multispectral-Augmented Endoscopic Prototype

Optical biopsy mapping on endoscopic image mosaics with a marker-free probe

Gastric cancer is the second leading cause of cancer-related deaths worldwide. Early diagnosis significantly increases the chances of survival; therefore, improved assisted exploration and screening techniques are necessary. Previously, we made use of an augmented multi-spectral endoscope by inserting an optical probe into the instrumentation channel. However, the limited field of view and the lack of markings left by optical biopsies on the tissue complicate the navigation and revisit of the suspect areas probed in-vivo. In this contribution two innovative tools are introduced to significantly increase the traceability and monitoring of patients in clinical practice: (i) video mosaicing to build a more comprehensive and panoramic view of large gastric areas; (ii) optical biopsy targeting and registration with the endoscopic images. The proposed optical flow-based mosaicing technique selects images that minimize texture discontinuities and is robust despite the lack of texture and illumination variations. The optical biopsy targeting is based on automatic tracking of a free-marker probe in the endoscopic view using deep learning to dynamically estimate its pose during exploration. The accuracy of pose estimation is sufficient to ensure a precise overlapping of the standard white-light color image and the hyperspectral probe image, assuming that the small target area of the organ is almost flat. This allows the mapping of all spatio-temporally tracked biopsy sites onto the panoramic mosaic. Experimental validations are carried out from videos acquired on patients in hospital. The proposed technique is purely software-based and therefore easily integrable into clinical practice. It is also generic and compatible to any imaging modality that connects to a fiberscope.

Multimodal imaging as optical biopsy system for gastritis diagnosis in humans, and input of the mouse model

BACKGROUND

Gastric inflammation is a major risk factor for gastric cancer. Current endoscopic methods are not able to efficiently detect and characterize gastric inflammation, leading to a sub-optimal patients' care. New non-invasive methods are needed. Reflectance mucosal light analysis is of particular interest in this context. The aim of our study was to analyze reflectance light and specific autofluorescence signals, both in humans and in a mouse model of gastritis.

METHODS

We recruited patients undergoing gastroendoscopic procedure during which reflectance was analysed with a multispectral camera. In parallel, the gastritis mouse model of Helicobacter pylori infection was used to investigate reflectance from ex vivo gastric samples using a spectrometer. In both cases, autofluorescence signals were measured using a confocal microscope.

FINDINGS

In gastritis patients, reflectance modifications were significant in near-infrared spectrum, with a decrease between 610 and 725 nm and an increase between 750 and 840 nm. Autofluorescence was also modified, showing variations around 550 nm of emission. In H. pylori infected mice developing gastric inflammatory lesions, we observed significant reflectance modifications 18 months after infection, with increased intensity between 617 and 672 nm. Autofluorescence was significantly modified after 1, 3 and 6 months around 550 and 630 nm. Both in human and in mouse, these reflectance data can be considered as biomarkers and accurately predicted inflammatory state.

INTERPRETATION

In this pilot study, using a practical measuring device, we identified in humans, modification of reflectance spectra in the visible spectrum and for the first time in near-infrared, associated with inflammatory gastric states. Furthermore, both in the mouse model and humans, we also observed modifications of autofluorescence associated with gastric inflammation. These innovative data pave the way to deeper validation studies on larger cohorts, for further development of an optical biopsy system to detect gastritis and finally to better surveil this important gastric cancer risk factor.

FUNDING

The project was funded by the ANR EMMIE (ANR-15-CE17-0015) and the French Gastroenterology Society (SNFGE).

A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging

Gastrointestinal (GI) endoscopy has been an active field of research motivated by the large number of highly lethal GI cancers. Early GI cancer precursors are often missed during the endoscopic surveillance. The high missed rate of such abnormalities during endoscopy is thus a critical bottleneck. Lack of attentiveness due to tiring procedures, and requirement of training are few contributing factors. An automatic GI disease classification system can help reduce such risks by flagging suspicious frames and lesions. GI endoscopy consists of several multi-organ surveillance, therefore, there is need to develop methods that can generalize to various endoscopic findings. In this realm, we present a comprehensive analysis of the Medico GI challenges: Medical Multimedia Task at MediaEval 2017, Medico Multimedia Task at MediaEval 2018, and BioMedia ACM MM Grand Challenge 2019. These challenges are initiative to set-up a benchmark for different computer vision methods applied to the multi-class endoscopic images and promote to build new approaches that could reliably be used in clinics. We report the performance of 21 participating teams over a period of three consecutive years and provide a detailed analysis of the methods used by the participants, highlighting the challenges and shortcomings of the current approaches and dissect their credibility for the use in clinical settings. Our analysis revealed that the participants achieved an improvement on maximum Mathew correlation coefficient (MCC) from 82.68% in 2017 to 93.98% in 2018 and 95.20% in 2019 challenges, and a significant increase in computational speed over consecutive years.