3D reconstruction of small bowel lesions using stereo camera-based capsule endoscopy

QRNet: A Quaternion-Based Retinex Framework for Enhanced Wireless Capsule Endoscopy Image Quality

Wireless capsule endoscopy (WCE) offers a non-invasive diagnostic alternative for the gastrointestinal tract using a battery-powered capsule. Despite advantages, WCE encounters issues with video quality and diagnostic accuracy, often resulting in missing rates of 1-20%. These challenges stem from weak texture characteristics due to non-Lambertian tissue reflections, uneven illumination, and the necessity of color fidelity. Traditional Retinex-based methods used for image enhancement are suboptimal for endoscopy, as they frequently compromise anatomical detail while distorting color. To address these limitations, we introduce QRNet, a novel quaternion-based Retinex framework. QRNet performs image decomposition into reflectance and illumination components within hypercomplex space, maintaining inter-channel relationships that preserve color fidelity. A quaternion wavelet attention mechanism refines essential features while suppressing noise, balancing enhancement and fidelity through an innovative loss function. Experiments on Kvasir-Capsule and Red Lesion Endoscopy datasets demonstrate notable improvements in metrics such as PSNR (+2.3 dB), SSIM (+0.089), and LPIPS (-0.126). Moreover, lesion segmentation accuracy increases by up to 5%, indicating the framework's potential for improving early-stage lesion detection. Ablation studies highlight the quaternion representation's pivotal role in maintaining color consistency, confirming the promise of this advanced approach for clinical settings.

Robotic wireless capsule endoscopy: recent advances and upcoming technologies

Wireless capsule endoscopy (WCE) offers a non-invasive evaluation of the digestive system, eliminating the need for sedation and the risks associated with conventional endoscopic procedures. Its significance lies in diagnosing gastrointestinal tissue irregularities, especially in the small intestine. However, existing commercial WCE devices face limitations, such as the absence of autonomous lesion detection and treatment capabilities. Recent advancements in micro-electromechanical fabrication and computational methods have led to extensive research in sophisticated technology integration into commercial capsule endoscopes, intending to supersede wired endoscopes. This Review discusses the future requirements for intelligent capsule robots, providing a comparative evaluation of various methods' merits and disadvantages, and highlighting recent developments in six technologies relevant to WCE. These include near-field wireless power transmission, magnetic field active drive, ultra-wideband/intrabody communication, hybrid localization, AI-based autonomous lesion detection, and magnetic-controlled diagnosis and treatment. Moreover, we explore the feasibility for future "capsule surgeons".

Efficacy and safety of three-dimensional magnetically assisted capsule endoscopy for upper gastrointestinal and small bowel examination

BACKGROUND

Magnetically assisted capsule endoscopy (MACE) showed the feasibility for upper gastrointestinal examination. To further enhance the performance of conventional MACE, it is necessary to provide quality-improved and three-dimensional images. The aim of this clinical study was to determine the efficacy and safety of novel three-dimensional MACE (3D MACE) for upper gastrointestinal and small bowel examination at once.

METHODS

This was a prospective, single-center, non-randomized, and sequential examination study (KCT0007114) at Dongguk University Ilsan Hospital. Adult patients who visited for upper endoscopy were included. The study protocol was conducted in two stages. First, upper gastrointestinal examination was performed using 3D MACE, and a continuous small bowel examination was performed by conventional method of capsule endoscopy. Two hours later, an upper endoscopy was performed for comparison with 3D MACE examination. The primary outcome was confirmation of major gastric structures (esophagogastric junction, cardia/fundus, body, angle, antrum, and pylorus). Secondary outcomes were confirmation of esophagus and duodenal bulb, accuracy for gastric lesions, completion of small bowel examination, 3D image reconstruction of gastric lesion, and safety.

RESULTS

Fifty-five patients were finally enrolled. The examination time of 3D MACE was 14.84 ± 3.02 minutes and upper endoscopy was 5.22 ± 2.39 minutes. The confirmation rate of the six major gastric structures was 98.6% in 3D MACE and 100% in upper endoscopy. Gastric lesions were identified in 43 patients during 3D MACE, and 40 patients during upper endoscopy (Sensitivity 0.97). 3D reconstructed images were acquired for all lesions inspected by 3D MACE. The continuous small bowel examination by 3D MACE was completed in 94.5%. 3D MACE showed better overall satisfaction (3D MACE 9.55 ± 0.79 and upper endoscopy 7.75 ± 2.34, p<0.0001). There were no aspiration or significant adverse event or capsule retention in the 3D MACE examination.

CONCLUSIONS

Novel 3D MACE system is more advanced diagnostic modality than the conventional MACE. And it is possible to perform serial upper gastrointestinal and small bowel examination as a non-invasive and one-step test. It would be also served as a bridge to pan-endoscopy.

Challenges of 3D Surface Reconstruction in Capsule Endoscopy

Essential for improving the accuracy and reliability of bowel cancer screening, three-dimensional (3D) surface reconstruction using capsule endoscopy (CE) images remains challenging due to CE hardware and software limitations. This report generally focuses on challenges associated with 3D visualization and specifically investigates the impact of the indeterminate selection of the angle of the line-of-sight on 3D surfaces. Furthermore, it demonstrates that impact through 3D surfaces viewed at the same azimuth angles and different elevation angles of the line-of-sight. The report concludes that 3D printing of reconstructed 3D surfaces can potentially overcome line-of-sight indeterminate selection and 2D screen visual restriction-related errors.

Small Bowel Detection for Wireless Capsule Endoscopy Using Convolutional Neural Networks with Temporal Filtering

By automatically classifying the stomach, small bowel, and colon, the reading time of the wireless capsule endoscopy (WCE) can be reduced. In addition, it is an essential first preprocessing step to localize the small bowel in order to apply automated small bowel lesion detection algorithms based on deep learning. The purpose of the study was to develop an automated small bowel detection method from long untrimmed videos captured from WCE. Through this, the stomach and colon can also be distinguished. The proposed method is based on a convolutional neural network (CNN) with a temporal filtering on the predicted probabilities from the CNN. For CNN, we use a ResNet50 model to classify three organs including stomach, small bowel, and colon. The hybrid temporal filter consisting of a Savitzky–Golay filter and a median filter is applied to the temporal probabilities for the “small bowel” class. After filtering, the small bowel and the other two organs are differentiated with thresholding. The study was conducted on dataset of 200 patients (100 normal and 100 abnormal WCE cases), which was divided into a training set of 140 cases, a validation set of 20 cases, and a test set of 40 cases. For the test set of 40 patients (20 normal and 20 abnormal WCE cases), the proposed method showed accuracy of 99.8% in binary classification for the small bowel. Transition time errors for gastrointestinal tracts were only 38.8 ± 25.8 seconds for the transition between stomach and small bowel and 32.0 ± 19.1 seconds for the transition between small bowel and colon, compared to the ground truth organ transition points marked by two experienced gastroenterologists.

Preparation of image databases for artificial intelligence algorithm development in gastrointestinal endoscopy

Over the past decade, technological advances in deep learning have led to the introduction of artificial intelligence (AI) in medical imaging. The most commonly used structure in image recognition is the convolutional neural network, which mimics the action of the human visual cortex. The applications of AI in gastrointestinal endoscopy are diverse. Computer-aided diagnosis has achieved remarkable outcomes with recent improvements in machine-learning techniques and advances in computer performance. Despite some hurdles, the implementation of AI-assisted clinical practice is expected to aid endoscopists in real-time decision-making. In this summary, we reviewed state-of-the-art AI in the field of gastrointestinal endoscopy and offered a practical guide for building a learning image dataset for algorithm development.

Gastric examination using a novel three-dimensional magnetically assisted capsule endoscope and a hand-held magnetic controller: A porcine model study

Magnetically assisted capsule endoscopy (MACE) is a noninvasive procedure and can overcome passive capsule movement that limits gastric examination. MACE has been studied in many trials as an alternative to upper endoscopy. However, to increase diagnostic accuracy of various gastric lesions, MACE should be able to provide stereoscopic, clear images and to measure the size of a lesion. So, we conducted the animal experiment using a novel three-dimensional (3D) MACE and a new hand-held magnetic controller for gastric examination. The purpose of this study is to assess the performance and safety of 3D MACE and hand-held magnetic controller through the animal experiment. Subsequently, via the dedicated viewer, we evaluate whether 3D reconstruction images and clear images can be obtained and accurate lesion size can be measured. During real-time gastric examination, the maneuverability and visualization of 3D MACE were adequate. A polypoid mass lesion was incidentally observed at the lesser curvature side of the prepyloric antrum. The mass lesion was estimated to be 10.9 x 11.5 mm in the dedicated viewer, nearly the same size and shape as confirmed by upper endoscopy and postmortem examination. Also, 3D and clear images of the lesion were successfully reconstructed. This animal experiment demonstrates the accuracy and safety of 3D MACE. Further clinical studies are warranted to confirm the feasibility of 3D MACE for human gastric examination.

Diagnostic Yields and Clinical Impacts of Capsule Endoscopy

Observing the entire small bowel is difficult due to the presence of complex loops and a long length. Capsule endoscopy (CE) provides a noninvasive and patient-friendly method for visualizing the small bowel and colon. Small bowel capsule endoscopy (SBCE) has a critical role in the diagnosis of small bowel disorders through the direct observation of the entire small bowel mucosa and is becoming the primary diagnostic tool for small bowel diseases. Recently, colon capsule endoscopy (CCE) was also considered safe and feasible for obtaining sufficient colonic images in patients with incomplete colonoscopy, in the absence of bowel obstruction. This review article assesses the current status of CE in terms of the diagnostic yield and the clinical impact of SBCE in patients with obscure gastrointestinal bleeding, who have known or suspected Crohn's disease, small bowel tumor and inherited polyposis syndrome, celiac disease, and those who have undergone CCE.

Capsule Endoscopy: Pitfalls and Approaches to Overcome

Capsule endoscopy of the gastrointestinal tract is an innovative technology that serves to replace conventional endoscopy. Wireless capsule endoscopy, which is mainly used for small bowel examination, has recently been used to examine the entire gastrointestinal tract. This method is promising for its usefulness and development potential and enhances convenience by reducing the side effects and discomfort that may occur during conventional endoscopy. However, capsule endoscopy has fundamental limitations, including passive movement via bowel peristalsis and space restriction. This article reviews the current scientific aspects of capsule endoscopy and discusses the pitfalls and approaches to overcome its limitations. This review includes the latest research results on the role and potential of capsule endoscopy as a non-invasive diagnostic and therapeutic device.

Examination of Entire Gastrointestinal Tract: A Perspective of Mouth to Anus (M2A) Capsule Endoscopy

Capsule endoscopy (CE) is the only non-invasive diagnostic tool that enables the direct visualization of the gastrointestinal (GI) tract. Even though CE was initially developed for small-bowel investigation, its clinical application is expanding, and technological advances continue. The final iteration of CE will be a mouth to anus (M2A) capsule that investigates the entire GI tract by the ingestion of a single capsule. This narrative review describes the current developmental status of CE and discusses the possibility of realizing an M2A capsule and what needs to be overcome in the future.

Development and Verification of a Deep Learning Algorithm to Evaluate Small-Bowel Preparation Quality

Capsule endoscopy (CE) quality control requires an objective scoring system to evaluate the preparation of the small bowel (SB). We propose a deep learning algorithm to calculate SB cleansing scores and verify the algorithm’s performance. A 5-point scoring system based on clarity of mucosal visualization was used to develop the deep learning algorithm (400,000 frames; 280,000 for training and 120,000 for testing). External validation was performed using additional CE cases (n = 50), and average cleansing scores (1.0 to 5.0) calculated using the algorithm were compared to clinical grades (A to C) assigned by clinicians. Test results obtained using 120,000 frames exhibited 93% accuracy. The separate CE case exhibited substantial agreement between the deep learning algorithm scores and clinicians’ assessments (Cohen’s kappa: 0.672). In the external validation, the cleansing score decreased with worsening clinical grade (scores of 3.9, 3.2, and 2.5 for grades A, B, and C, respectively, p < 0.001). Receiver operating characteristic curve analysis revealed that a cleansing score cut-off of 2.95 indicated clinically adequate preparation. This algorithm provides an objective and automated cleansing score for evaluating SB preparation for CE. The results of this study will serve as clinical evidence supporting the practical use of deep learning algorithms for evaluating SB preparation quality.

X-ray Imaging for Gastrointestinal Tracking of Microscale Oral Drug Delivery Devices

Microscale devices are promising tools to overcome specific challenges within oral drug delivery. Despite the availability of advanced high-quality imaging techniques, visualization and tracking of microscale devices in the gastrointestinal (GI) tract is still a challenge. This work explores the possibilities of applying planar X-ray imaging and computed tomography (CT) scanning for visualization and tracking of microscale devices in the GI tract of rats. Microcontainers (MCs) are an example of microscale devices that have shown great potential as an oral drug delivery system. Barium sulfate (BaSO₄) loaded into the cavity of the MCs increases their overall X-ray contrast, which allows them to be easily tracked. The BaSO₄-loaded MCs are quantitatively tracked throughout the entire GI tract of rats by planar X-ray imaging and visualized in 3D by CT scanning. The majority of the BaSO₄-loaded MCs are observed to retain in the stomach for 0.5-2 h, enter the cecum after 3-4 h, and leave the cecum and colon 8-10 h post-administration. The imaging approaches can be adopted and used with other types of microscale devices when investigating GI behavior in, for example, preclinical trials and potential clinical studies.

Development of a deep learning-based software for calculating cleansing score in small bowel capsule endoscopy

A standardized small bowel (SB) cleansing scale is currently not available. The aim of this study was to develop an automated calculation software for SB cleansing score using deep learning. Consecutively performed capsule endoscopy cases were enrolled from three hospitals. A 5-step scoring system based on mucosal visibility was trained for deep learning in the training set. Performance of the trained software was evaluated in the validation set. Average cleansing score (1.0 to 5.0) by deep learning was compared to clinical grading (A to C) reviewed by clinicians. Cleansing scores decreased as clinical grading worsened (scores of 4.1, 3.5, and 2.9 for grades A, B, and C, respectively, P < 0.001). Adequate preparation was achieved for 91.7% of validation cases. The average cleansing score was significantly different between adequate and inadequate group (4.0 vs. 2.9, P < 0.001). ROC curve analysis revealed that a cut-off value of cleansing score at 3.25 had an AUC of 0.977. Diagnostic yields for small, hard-to-find lesions were associated with high cleansing scores (4.3 vs. 3.8, P < 0.001). We developed a novel scoring software which calculates objective, automated cleansing scores for SB preparation. The cut-off value we suggested provides a standard criterion for adequate bowel preparation as a quality indicator.

Optimization Design for Receiving Coil with Novel Structure Based on Mutual Coupling Model in Wireless Power Transmission for Capsule Endoscope

Wireless capsule endoscope (WCE) is a promising technology for noninvasive and painless imaging detection on gastrointestinal (GI) diseases. On the other hand, conventional endoscopes with wires could discomfort patients and cause them to vomit and aerosolize coronavirus if the patients are infected with COVID-19. However, there stands a technical bottleneck on power supply for the WCE. With the help of wireless power transmission technology, a hollow receiving coil (RC) is proposed to supply sufficient power and also minimize the size of WCE. A model on mutual inductance between transmitting and receiving coils is proposed to evaluate receiving power when the RC is in a different position and direction of patient’s GI tract. Based on the model, an optimal RC is built to obtain sufficient and stable power. Measurement of mutual inductance with the optimal RC validates high accuracy of the proposed model. The standard deviation of receiving power is very low. WCE with optimum RC gets sufficient power and captures images stably in live pig’s intestine tract. Additionally, the model is little affected by biological tissues. It ensures reliable performance of WCE and makes popular clinical application of WCE possible, which is also a relief to reduce epidemics like COVID-19.

A New Active Locomotion Capsule Endoscopy under Magnetic Control and Automated Reading Program

Capsule endoscopy (CE) is the first-line diagnostic modality for detecting small bowel lesions. CE is non-invasive and does not require sedation, but its movements cannot be controlled, it requires a long time for interpretation, and it has lower image quality compared to wired endoscopy. With the rapid advancement of technology, several methods to solve these problems have been developed. This article describes the ongoing developments regarding external CE locomotion using magnetic force, artificial intelligence-based interpretation, and image-enhancing technologies with the CE system.