Software for enhanced video capsule endoscopy: challenges for essential progress

Robotic wireless capsule endoscopy: recent advances and upcoming technologies

Wireless capsule endoscopy (WCE) offers a non-invasive evaluation of the digestive system, eliminating the need for sedation and the risks associated with conventional endoscopic procedures. Its significance lies in diagnosing gastrointestinal tissue irregularities, especially in the small intestine. However, existing commercial WCE devices face limitations, such as the absence of autonomous lesion detection and treatment capabilities. Recent advancements in micro-electromechanical fabrication and computational methods have led to extensive research in sophisticated technology integration into commercial capsule endoscopes, intending to supersede wired endoscopes. This Review discusses the future requirements for intelligent capsule robots, providing a comparative evaluation of various methods' merits and disadvantages, and highlighting recent developments in six technologies relevant to WCE. These include near-field wireless power transmission, magnetic field active drive, ultra-wideband/intrabody communication, hybrid localization, AI-based autonomous lesion detection, and magnetic-controlled diagnosis and treatment. Moreover, we explore the feasibility for future "capsule surgeons".

An efficient computational framework for gastrointestinal disorder prediction using attention-based transfer learning

Diagnosing gastrointestinal (GI) disorders, which affect parts of the digestive system such as the stomach and intestines, can be difficult even for experienced gastroenterologists due to the variety of ways these conditions present. Early diagnosis is critical for successful treatment, but the review process is time-consuming and labor-intensive. Computer-aided diagnostic (CAD) methods provide a solution by automating diagnosis, saving time, reducing workload, and lowering the likelihood of missing critical signs. In recent years, machine learning and deep learning approaches have been used to develop many CAD systems to address this issue. However, existing systems need to be improved for better safety and reliability on larger datasets before they can be used in medical diagnostics. In our study, we developed an effective CAD system for classifying eight types of GI images by combining transfer learning with an attention mechanism. Our experimental results show that ConvNeXt is an effective pre-trained network for feature extraction, and ConvNeXt+Attention (our proposed method) is a robust CAD system that outperforms other cutting-edge approaches. Our proposed method had an area under the receiver operating characteristic curve of 0.9997 and an area under the precision-recall curve of 0.9973, indicating excellent performance. The conclusion regarding the effectiveness of the system was also supported by the values of other evaluation metrics.

Polypoid Lesion Segmentation Using YOLO-V8 Network in Wireless Video Capsule Endoscopy Images

Gastrointestinal (GI) tract disorders are a significant public health issue. They are becoming more common and can cause serious health problems and high healthcare costs. Small bowel tumours (SBTs) and colorectal cancer (CRC) are both becoming more prevalent, especially among younger adults. Early detection and removal of polyps (precursors of malignancy) is essential for prevention. Wireless Capsule Endoscopy (WCE) is a procedure that utilises swallowable camera devices that capture images of the GI tract. Because WCE generates a large number of images, automated polyp segmentation is crucial. This paper reviews computer-aided approaches to polyp detection using WCE imagery and evaluates them using a dataset of labelled anomalies and findings. The study focuses on YOLO-V8, an improved deep learning model, for polyp segmentation and finds that it performs better than existing methods, achieving high precision and recall. The present study underscores the potential of automated detection systems in improving GI polyp identification.

Enteroscopy versus Video Capsule Endoscopy for Automatic Diagnosis of Small Bowel Disorders-A Comparative Analysis of Artificial Intelligence Applications

BACKGROUND

Small bowel disorders present a diagnostic challenge due to the limited accessibility of the small intestine. Accurate diagnosis is made with the aid of specific procedures, like capsule endoscopy or double-ballon enteroscopy, but they are not usually solicited and not widely accessible. This study aims to assess and compare the diagnostic effectiveness of enteroscopy and video capsule endoscopy (VCE) when combined with artificial intelligence (AI) algorithms for the automatic detection of small bowel diseases.

MATERIALS AND METHODS

We performed an extensive literature search for relevant studies about AI applications capable of identifying small bowel disorders using enteroscopy and VCE, published between 2012 and 2023, employing PubMed, Cochrane Library, Google Scholar, Embase, Scopus, and ClinicalTrials.gov databases.

RESULTS

Our investigation discovered a total of 27 publications, out of which 21 studies assessed the application of VCE, while the remaining 6 articles analyzed the enteroscopy procedure. The included studies portrayed that both investigations, enhanced by AI, exhibited a high level of diagnostic accuracy. Enteroscopy demonstrated superior diagnostic capability, providing precise identification of small bowel pathologies with the added advantage of enabling immediate therapeutic intervention. The choice between these modalities should be guided by clinical context, patient preference, and resource availability. Studies with larger sample sizes and prospective designs are warranted to validate these results and optimize the integration of AI in small bowel diagnostics.

CONCLUSIONS

The current analysis demonstrates that both enteroscopy and VCE with AI augmentation exhibit comparable diagnostic performance for the automatic detection of small bowel disorders.

Deep learning-based prediction model for diagnosing gastrointestinal diseases using endoscopy images

BACKGROUND

Gastrointestinal (GI) infections are quite common today around the world. Colonoscopy or wireless capsule endoscopy (WCE) are noninvasive methods for examining the whole GI tract for abnormalities. Nevertheless, it requires a great deal of time and effort for doctors to visualize a large number of images, and diagnosis is prone to human error. As a result, developing automated artificial intelligence (AI) based GI disease diagnosis methods is a crucial and emerging research area. AI-based prediction models may lead to improvements in the early diagnosis of gastrointestinal disorders, assessing severity, and healthcare systems for the benefit of patients as well as clinicians. The focus of this research is on the early diagnosis of gastrointestinal diseases using a convolution neural network (CNN) to enhance diagnosis accuracy.

METHODS

Various CNN models (baseline model and using transfer learning (VGG16, InceptionV3, and ResNet50)) were trained on a benchmark image dataset, KVASIR, containing images from inside the GI tract using n-fold cross-validation. The dataset comprises images of three disease states-polyps, ulcerative colitis, and esophagitis-as well as images of the healthy colon. Data augmentation strategies together with statistical measures were used to improve and evaluate the model's performance. Additionally, the test set comprising 1200 images was used to evaluate the model's accuracy and robustness.

RESULTS

The CNN model using the weights of the ResNet50 pre-trained model achieved the highest average accuracy of approximately 99.80% on the training set (100% precision and approximately 99% recall) and accuracies of 99.50% and 99.16% on the validation and additional test set, respectively, while diagnosing GI diseases. When compared to other existing systems, the proposed ResNet50 model outperforms them all.

CONCLUSION

The findings of this study indicate that AI-based prediction models using CNNs, specifically ResNet50, can improve diagnostic accuracy for detecting gastrointestinal polyps, ulcerative colitis, and esophagitis. The prediction model is available at https://github.com/anjus02/GI-disease-classification.git.

Anatomical landmarks localization for capsule endoscopy studies

Wireless Capsule Endoscopy is a medical procedure that uses a small, wireless camera to capture images of the inside of the digestive tract. The identification of the entrance and exit of the small bowel and of the large intestine is one of the first tasks that need to be accomplished to read a video. This paper addresses the design of a clinical decision support tool to detect these anatomical landmarks. We have developed a system based on deep learning that combines images, timestamps, and motion data to achieve state-of-the-art results. Our method does not only classify the images as being inside or outside the studied organs, but it is also able to identify the entrance and exit frames. The experiments performed with three different datasets (one public and two private) show that our system is able to approximate the landmarks while achieving high accuracy on the classification problem (inside/outside of the organ). When comparing the entrance and exit of the studied organs, the distance between predicted and real landmarks is reduced from 1.5 to 10 times with respect to previous state-of-the-art methods.

Capsule Enteroscopy Using the Mirocam® versus OMOM® Systems: A Matched Case-Control Study

Although several devices are available for small bowel capsule endoscopy, few studies have compared their visualization quality and diagnostic yield, despite users reporting subjective differences between them. This study aims to compare two widely used systems (Mirocam® MC1600 and OMOM® HD). Patients who underwent OMOM® HD capsule enteroscopy between August 2022 and February 2023 were prospectively included consecutively (cases). Controls were retrospectively selected from a database of patients who underwent Mirocam® MC1600 enteroscopy between March 2018 and July 2022 in a 1:1 ratio. Controls were matched for potential confounders (age, sex, indication, hospitalization, comorbidities, and opioid prescription). The small bowel cleanliness (global and divided by tertiles), the diagnostic yield (positive findings) and the transit times (TT) were compared. Overall, 214 patients were included (107:107). Global bowel preparation was similar between the OMOM® and Mirocam® groups. However, the average scores for each tertile were significantly higher when the OMOM® HD capsule was used (p < 0.05). Small bowel TT was shorter for OMOM® HD (265 ± 118 versus 307 ± 87 min, p = 0.020), while the diagnostic yield (55.0%) and relative distribution of lesions were similar. This study suggests that capsule characteristics, namely resolution, and illumination, systematically interfere with the perception of preparation quality. However, this did not affect the diagnostic yield.

Computer-Aided Bleeding Detection Algorithms for Capsule Endoscopy: A Systematic Review

Capsule endoscopy (CE) is a widely used medical imaging tool for the diagnosis of gastrointestinal tract abnormalities like bleeding. However, CE captures a huge number of image frames, constituting a time-consuming and tedious task for medical experts to manually inspect. To address this issue, researchers have focused on computer-aided bleeding detection systems to automatically identify bleeding in real time. This paper presents a systematic review of the available state-of-the-art computer-aided bleeding detection algorithms for capsule endoscopy. The review was carried out by searching five different repositories (Scopus, PubMed, IEEE Xplore, ACM Digital Library, and ScienceDirect) for all original publications on computer-aided bleeding detection published between 2001 and 2023. The Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) methodology was used to perform the review, and 147 full texts of scientific papers were reviewed. The contributions of this paper are: (I) a taxonomy for computer-aided bleeding detection algorithms for capsule endoscopy is identified; (II) the available state-of-the-art computer-aided bleeding detection algorithms, including various color spaces (RGB, HSV, etc.), feature extraction techniques, and classifiers, are discussed; and (III) the most effective algorithms for practical use are identified. Finally, the paper is concluded by providing future direction for computer-aided bleeding detection research.

Computer vision-based solutions to overcome the limitations of wireless capsule endoscopy

Endoscopic investigation plays a critical role in the diagnosis of gastrointestinal (GI) diseases. Since 2001, Wireless Capsule Endoscopy (WCE) has been available for small bowel exploration and is in continuous development. Over the last decade, WCE has achieved impressive improvements in areas such as miniaturisation, image quality and battery life. As a result, WCE is currently a very useful alternative to wired enteroscopy in the investigation of various small bowel abnormalities and has the potential to become the leading screening technique for the entire gastrointestinal tract. However, commercial solutions still have several limitations, namely incomplete examination and limited diagnostic capacity. These deficiencies are related to technical issues, such as image quality, motion estimation and power consumption management. Computational methods, based on image processing and analysis, can help to overcome these challenges and reduce both the time required by reviewers and human interpretation errors. Research groups have proposed a series of methods including algorithms for locating the capsule or lesion, assessing intestinal motility and improving image quality.In this work, we provide a critical review of computational vision-based methods for WCE image analysis aimed at overcoming the technological challenges of capsules. This article also reviews several representative public datasets used to evaluate the performance of WCE techniques and methods. Finally, some promising solutions of computational methods based on the analysis of multiple-camera endoscopic images are presented.

Study on image data cleaning method of early esophageal cancer based on VGG_NIN neural network

In order to clean the mislabeled images in the esophageal endoscopy image data set, we designed a new neural network VGG_NIN. Based on the new neural network structure, we developed a method to clean the mislabeled images in the esophageal endoscopy image data set. To verify the effectiveness of the proposed method, we designed two experiments using 3835 esophageal endoscopy images provided by West China Hospital of Sichuan University. The experimental results showed that the proposed method could clean about 93% of the mislabeled images in the data set, which was the first time in the cleaning of esophageal endoscopy image data set. Finally, in order to verify the generalization ability of this method, we cleaned the Kaggle open cat and dog data set, and cleaned out about 167 mislabeled images. Therefore, the proposed method can effectively screen the mislabeled images in the esophageal endoscopy image data set and has good generalization ability, which can provide great help for the development of high-performance gastrointestinal endoscopy image analysis model.

Development and Validation of an Artificial Intelligence Model for Small Bowel Capsule Endoscopy Video Review

IMPORTANCE

Reading small bowel capsule endoscopy (SBCE) videos is a tedious task for clinicians, and a new method should be applied to solve the situation.

OBJECTIVES

To develop and evaluate the performance of a convolutional neural network algorithm for SBCE video review in real-life clinical care.

DESIGN, SETTING, AND PARTICIPANTS

In this multicenter, retrospective diagnostic study, a deep learning neural network (SmartScan) was trained and validated for the SBCE video review. A total of 2927 SBCE examinations from 29 medical centers were used to train SmartScan to detect 17 types of CE structured terminology (CEST) findings from January 1, 2019, to June 30, 2020. SmartScan was later validated with conventional reading (CR) and SmartScan-assisted reading (SSAR) in 2898 SBCE examinations collected from 22 medical centers. Data analysis was performed from January 25 to December 31, 2021.

EXPOSURE

An artificial intelligence-based tool for interpreting clinical images of SBCE.

MAIN OUTCOMES AND MEASURES

The detection rate and efficiency of CEST findings detected by SSAR and CR were compared.

RESULTS

A total of 5825 SBCE examinations were retrospectively collected; 2898 examinations (1765 male participants [60.9%]; mean [SD] age, 49.8 [15.5] years) were included in the validation phase. From a total of 6084 CEST-classified SB findings, SSAR detected 5834 findings (95.9%; 95% CI, 95.4%-96.4%), significantly higher than CR, which detected 4630 findings (76.1%; 95% CI, 75.0%-77.2%). SmartScan-assisted reading achieved a higher per-patient detection rate (79.3% [2298 of 2898]) for CEST findings compared with CR (70.7% [2048 of 2298]; 95% CI, 69.0%-72.3%). With SSAR, the mean (SD) number of images (per SBCE video) requiring review was reduced to 779.2 (337.2) compared with 27 910.8 (12 882.9) with CR, for a mean (SD) reduction rate of 96.1% (4.3%). The mean (SD) reading time with SSAR was shortened to 5.4 (1.5) minutes compared with CR (51.4 [11.6] minutes), for a mean (SD) reduction rate of 89.3% (3.1%).

CONCLUSIONS AND RELEVANCE

This study suggests that a convolutional neural network-based algorithm is associated with an increased detection rate of SBCE findings and reduced SBCE video reading time.

Soft hybrid intrinsically motile robot for wireless small bowel enteroscopy

BACKGROUND

Difficulties in establishing diagnosis of small bowel (SB) disorders, prevented their effective treatment. This problem was largely resolved by wireless capsule endoscopy (WCE), which has since become the first line investigation for suspected SB disorders. Several types of WCE pills are now used in clinical practice, despite their limitations and complications. WCE pills are large, rigid and immotile capsules. When swallowed, they provide SB enteroscopy downloaded to a data logger carried by the patient. Most of the complications of WCEs result from lack of intrinsic locomotion: incomplete examination, capsule retention and impaction within strictures. In addition, the rigid nature and size of current generation of WCE pills is accompanied by 0.1% inability to swallow the pill by patients with normal esophageal motility.

METHODS

The aim of this communication is to describe the initial prototype, P1, which is thinner and slightly longer than the current generation of WCEs. In addition, it exhibits intrinsic active locomotion, produced by vibrating silicon legs. These generate a controlled-skid locomotion on the small bowel mucosal surface, rendered slippery by surface mucus and intraluminal surfactant bile salts. We demonstrate the mechanism responsible for the active locomotion of P1, which we consider translatable into a working prototype, suitable for further R&D for eventual clinical translation.

RESULTS

The shape and attachment of the rubber vibrating legs to vibrating actuators, have been designed specifically to produce a tight clockwise circular motion. When inserted inside a circular tube in vitro of equivalent diameter to human small intestine, the intrinsic circular clockwise motion of P1 translates into a linear locomotion by the constraints imposed by the surrounding circular walls of SB and rest of the gastrointestinal tract. This design ensures device stability during transit, essential for imaging and targeting lesions encountered during the enteroscopy. We preformed two experiments: (i) transit of P1 through a phantom consisting of a segment of PVC tube placed on a horizontal surface and (ii) transit through a transparent slippery nylon sleeve insufflated with air. In the PVC tube, its transit rate averages 15.6 mm/s, which is too fast for endoscopy: whereas inside the very slippery nylon sleeve insufflated with air, the average transit rate of P1 is reduced to 5.9 mm/s, i.e., ideal for inspection endoscopy.

CONCLUSIONS

These in-vitro experiments indicate that the P1 hybrid soft robot prototype has the potential specifically for clinical translation for SB enteroscopy.

Gastrointestinal disorders in children with autism: Could artificial intelligence help?

Autism is one of the pervasive neurodevelopmental disorders usually associated with many medical comorbidities. Gastrointestinal (GI) disorders are pervasive in children, with a 46%-84% prevalence rate. Children with Autism have an increased frequency of diarrhea, nausea and/or vomiting, gastroesophageal reflux and/or disease, abdominal pain, chronic flatulence due to various factors as food allergies, gastrointestinal dysmotility, irritable bowel syndrome (IBS), and inflammatory bowel diseases (IBD). These GI disorders have a significant negative impact on both the child and his/her family. Artificial intelligence (AI) could help diagnose and manage Autism by improving children's communication, social, and emotional skills for a long time. AI is an effective method to enhance early detection of GI disorders, including GI bleeding, gastroesophageal reflux disease, Coeliac disease, food allergies, IBS, IBD, and rectal polyps. AI can also help personalize the diet for children with Autism by microbiome modification. It can help to provide modified gluten without initiating an immune response. However, AI has many obstacles in treating digestive diseases, especially in children with Autism. We need to do more studies and adopt specific algorithms for children with Autism. In this article, we will highlight the role of AI in helping children with gastrointestinal disorders, with particular emphasis on children with Autism.

Automatic Detection of Small Intestinal Hookworms in Capsule Endoscopy Images Based on a Convolutional Neural Network

Ancylostomiasis is a fairly common small bowel parasite disease identified by capsule endoscopy (CE) for which a computer-aided clinical detection method has not been established. We sought to develop an artificial intelligence system with a convolutional neural network (CNN) to automatically detect hookworms in CE images. We trained a deep CNN system based on a YOLO-V4 (You Look Only Once-Version4) detector using 11236 CE images of hookworms. We assessed its performance by calculating the area under the receiver operating characteristic curve and its sensitivity, specificity, and accuracy using an independent test set of 10,529 small-bowel images including 531 images of hookworms. The trained CNN system required 403 seconds to evaluate 10,529 test images. The area under the curve for the detection of hookworms was 0.972 (95% confidence interval (CI), 0.967-0.978). The sensitivity, specificity, and accuracy of the CNN system were 92.2%, 91.1%, and 91.2%, respectively, at a probability score cut-off of 0.485. We developed and validated a CNN-based system for detecting hookworms in CE images. By combining this high-accuracy, high-speed, and oversight-preventing system with other CNN systems, we hope it will become an important supplement for detecting intestinal abnormalities in CE images. This trial is registered with ChiCTR2000034546 (a clinical research of artificial-intelligence-aided diagnosis for hookworms in small intestine by capsule endoscope images).

Deep learning and capsule endoscopy: automatic identification and differentiation of small bowel lesions with distinct haemorrhagic potential using a convolutional neural network

OBJECTIVE

Capsule endoscopy (CE) is pivotal for evaluation of small bowel disease. Obscure gastrointestinal bleeding most often originates from the small bowel. CE frequently identifies a wide range of lesions with different bleeding potentials in these patients. However, reading CE examinations is a time-consuming task. Convolutional neural networks (CNNs) are highly efficient artificial intelligence tools for image analysis. This study aims to develop a CNN-based model for identification and differentiation of multiple small bowel lesions with distinct haemorrhagic potential using CE images.

DESIGN

We developed, trained, and validated a denary CNN based on CE images. Each frame was labelled according to the type of lesion (lymphangiectasia, xanthomas, ulcers, erosions, vascular lesions, protruding lesions, and blood). The haemorrhagic potential was assessed by Saurin's classification. The entire dataset was divided into training and validation sets. The performance of the CNN was measured by the area under the receiving operating characteristic curve, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

A total of 53 555 CE images were included. The model had an overall accuracy of 99%, a sensitivity of 88%, a specificity of 99%, a PPV of 87%, and an NPV of 99% for detection of multiple small bowel abnormalities and respective classification of bleeding potential.

CONCLUSION

We developed and tested a CNN-based model for automatic detection of multiple types of small bowel lesions and classification of the respective bleeding potential. This system may improve the diagnostic yield of CE for these lesions and overall CE efficiency.

Artificial Intelligence in Capsule Endoscopy: A Practical Guide to Its Past and Future Challenges

Artificial intelligence (AI) has revolutionized the medical diagnostic process of various diseases. Since the manual reading of capsule endoscopy videos is a time-intensive, error-prone process, computerized algorithms have been introduced to automate this process. Over the past decade, the evolution of convolutional neural network (CNN) enabled AI to detect multiple lesions simultaneously with increasing accuracy and sensitivity. Difficulty in validating CNN performance and unique characteristics of capsule endoscopy images make computer-aided reading systems in capsule endoscopy still on a preclinical level. Although AI technology can be used as an auxiliary second observer in capsule endoscopy, it is expected that in the near future, it will effectively reduce the reading time and ultimately become an independent, integrated reading system.

Artificial intelligence and colon capsule endoscopy: automatic detection of blood in colon capsule endoscopy using a convolutional neural network

Colon capsule endoscopy (CCE) is a minimally invasive alternative to conventional colonoscopy. Most studies on CCE focus on colorectal neoplasia detection. The development of automated tools may address some of the limitations of this diagnostic tool and widen its indications for different clinical settings. We developed an artificial intelligence model based on a convolutional neural network (CNN) for the automatic detection of blood content in CCE images. Training and validation datasets were constructed for the development and testing of the CNN. The CNN detected blood with a sensitivity, specificity, and positive and negative predictive values of 99.8 %, 93.2 %, 93.8 %, and 99.8 %, respectively. The area under the receiver operating characteristic curve for blood detection was 1.00. We developed a deep learning algorithm capable of accurately detecting blood or hematic residues within the lumen of the colon based on colon CCE images.

Role of Artificial Intelligence in Video Capsule Endoscopy

Capsule endoscopy (CE) has been increasingly utilised in recent years as a minimally invasive tool to investigate the whole gastrointestinal (GI) tract and a range of capsules are currently available for evaluation of upper GI, small bowel, and lower GI pathology. Although CE is undoubtedly an invaluable test for the investigation of small bowel pathology, it presents considerable challenges and limitations, such as long and laborious reading times, risk of missing lesions, lack of bowel cleansing score and lack of locomotion. Artificial intelligence (AI) seems to be a promising tool that may help improve the performance metrics of CE, and consequently translate to better patient care. In the last decade, significant progress has been made to apply AI in the field of endoscopy, including CE. Although it is certain that AI will find soon its place in day-to-day endoscopy clinical practice, there are still some open questions and barriers limiting its widespread application. In this review, we provide some general information about AI, and outline recent advances in AI and CE, issues around implementation of AI in medical practice and potential future applications of AI-aided CE.

Kvasir-Capsule, a video capsule endoscopy dataset

Artificial intelligence (AI) is predicted to have profound effects on the future of video capsule endoscopy (VCE) technology. The potential lies in improving anomaly detection while reducing manual labour. Existing work demonstrates the promising benefits of AI-based computer-assisted diagnosis systems for VCE. They also show great potential for improvements to achieve even better results. Also, medical data is often sparse and unavailable to the research community, and qualified medical personnel rarely have time for the tedious labelling work. We present Kvasir-Capsule, a large VCE dataset collected from examinations at a Norwegian Hospital. Kvasir-Capsule consists of 117 videos which can be used to extract a total of 4,741,504 image frames. We have labelled and medically verified 47,238 frames with a bounding box around findings from 14 different classes. In addition to these labelled images, there are 4,694,266 unlabelled frames included in the dataset. The Kvasir-Capsule dataset can play a valuable role in developing better algorithms in order to reach true potential of VCE technology.

Automated bleeding detection in wireless capsule endoscopy images based on color feature extraction from Gaussian mixture model superpixels

Wireless capsule endoscopy is the commonly employed modality in the treatment of gastrointestinal tract pathologies. However, the time taken for interpretation of these images is very high due to the large volume of images generated. Automated detection of disorders with these images can facilitate faster clinical interventions. In this paper, we propose an automated system based on Gaussian mixture model superpixels for bleeding detection and segmentation of candidate regions. The proposed system is realized with a classic binary support vector machine classifier trained with seven features including color and texture attributes extracted from the Gaussian mixture model superpixels of the WCE images. On detection of bleeding images, bleeding regions are segmented from them, by incrementally grouping the superpixels based on deltaE color differences. Tested with standard datasets, this system exhibits best performance compared to the state-of-the-art approaches with respect to classification accuracy, feature selection, computational time, and segmentation accuracy. The proposed system achieves 99.88% accuracy, 99.83% sensitivity, and 100% specificity signifying the effectiveness of the proposed system in bleeding detection with very few classification errors.

Deep Transfer Learning for Automated Intestinal Bleeding Detection in Capsule Endoscopy Imaging

PURPOSE

The objective of this paper was to develop a computer-aided diagnostic (CAD) tools for automated analysis of capsule endoscopic (CE) images, more precisely, detect small intestinal abnormalities like bleeding.

METHODS

In particular, we explore a convolutional neural network (CNN)-based deep learning framework to identify bleeding and non-bleeding CE images, where a pre-trained AlexNet neural network is used to train a transfer learning CNN that carries out the identification. Moreover, bleeding zones in a bleeding-identified image are also delineated using deep learning-based semantic segmentation that leverages a SegNet deep neural network.

RESULTS

To evaluate the performance of the proposed framework, we carry out experiments on two publicly available clinical datasets and achieve a 98.49% and 88.39% F1 score, respectively, on the capsule endoscopy.org and KID datasets. For bleeding zone identification, 94.42% global accuracy and 90.69% weighted intersection over union (IoU) are achieved.

CONCLUSION

Finally, our performance results are compared to other recently developed state-of-the-art methods, and consistent performance advances are demonstrated in terms of performance measures for bleeding image and bleeding zone detection. Relative to the present and established practice of manual inspection and annotation of CE images by a physician, our framework enables considerable annotation time and human labor savings in bleeding detection in CE images, while providing the additional benefits of bleeding zone delineation and increased detection accuracy. Moreover, the overall cost of CE enabled by our framework will also be much lower due to the reduction of manual labor, which can make CE affordable for a larger population.

Artificial Intelligence Research and Development for Application in Video Capsule Endoscopy

Artificial intelligence (AI) research for medical applications has expanded quickly. Advancements in computer processing now allow for the development of complex neural network architectures (eg, convolutional neural networks) that are capable of extracting and learning complex features from massive data sets, including large image databases. Gastroenterology and endoscopy are well suited for AI research. Video capsule endoscopy is an ideal platform for AI model research given the large amount of data produced by each capsule examination and the annotated databases that are already available. Studies have demonstrated high performance for applications of capsule-based AI models developed for various pathologic conditions.

Capsule endoscopy transit-related indicators in choosing the insertion route for double-balloon enteroscopy: a systematic review

Background and study aims  When capsule endoscopy (CE) detects a small bowel (SB) target lesion that may be manageable with enteroscopy, the selection of the insertion route is critical. Time- and progression-based CE indices have been proposed for localization of SB lesions. This systematic review analysed the role of CE transit indicators in choosing the insertion route for double-balloon enteroscopy (DBE). Methods  A comprehensive literature search identified papers assessing the role of CE on the choice of the route selection for DBE. Data on CE, criteria for route selection, and DBE success parameters were retrieved and analyzed according to the PRISMA statement. Risk of bias was assessed through the STROBE assessment. The primary outcome evaluated was DBE success rate in reaching a SB lesion, measured as the ratio of positive initial DBE to the number of total DBE. Results  Seven studies including 262 CEs requiring subsequent DBE were selected. Six studies used time-based indices and one used the PillCam Progress indicator. SB lesions were identified and insertion route was selected according to a specific cut-off, using fixed landmarks for defining SB transit except for one study in which the mouth-cecum transit was considered. DBE success rate was high in all studies, ranging from 78.3 % to 100 %. Six of seven studies were high quality. Conclusions  The precise localization of SB lesions remains an open issue, and larger studies are required to determine the most accurate index for selecting the DBE insertion route. In the future, 3 D localization technologies and tracking systems will be essential to accomplish this tricky task.

Video capsule endoscopy: pushing the boundaries with software technology

Video capsule endoscopy (VCE) has transformed imaging of the small bowel as it is a non-invasive and well tolerated modality with excellent diagnostic capabilities. The way we read VCE has not changed much since its introduction nearly two decades ago. Reading is still very time intensive and prone to reader error. This review outlines the evidence regarding software enhancements which aim to address these challenges. These include the suspected blood indicator (SBI), automated fast viewing modes including QuickView, lesion characterization tools such Fuji Intelligent Color Enhancement, and three-dimensional (3D) representation tools. We also outline the exciting new evidence of artificial intelligence (AI) and deep learning (DL), which promises to revolutionize capsule reading. DL algorithms have been developed for identifying organs of origin, intestinal motility events, active bleeding, coeliac disease, polyp detection, hookworms and angioectasias, all with impressively high sensitivity and accuracy. More recently, an algorithm has been created to detect multiple abnormalities with a sensitivity of 99.9% and reading time of only 5.9 minutes. These algorithms will need to be validated robustly. However, it will not be long before we see this in clinical practice, aiding the clinician in rapid and accurate diagnosis.

Artificial intelligence and deep learning for small bowel capsule endoscopy

Capsule endoscopy is ideally suited to artificial intelligence-based interpretation given its reliance on pattern recognition in still images. Time saving viewing modes and lesion detection features currently available rely on machine learning algorithms, a form of artificial intelligence. Current software necessitates close human supervision given poor sensitivity relative to an expert reader. However, with the advent of deep learning, artificial intelligence is becoming increasingly reliable and will be increasingly relied upon. We review the major advances in artificial intelligence for capsule endoscopy in recent publications and briefly review artificial intelligence development for historical understanding. Importantly, recent advancements in artificial intelligence have not yet been incorporated into practice and it is immature to judge the potential of this technology based on current platforms. Remaining regulatory and standardization hurdles are being overcome and artificial intelligence-based clinical applications are likely to proliferate rapidly.

Artificial intelligence in small bowel capsule endoscopy - current status, challenges and future promise

Neural network-based solutions are under development to alleviate physicians from the tedious task of small-bowel capsule endoscopy reviewing. Computer-assisted detection is a critical step, aiming to reduce reading times while maintaining accuracy. Weakly supervised solutions have shown promising results; however, video-level evaluations are scarce, and no prospective studies have been conducted yet. Automated characterization (in terms of diagnosis and pertinence) by supervised machine learning solutions is the next step. It relies on large, thoroughly labeled databases, for which preliminary "ground truth" definitions by experts are of tremendous importance. Other developments are under ways, to assist physicians in localizing anatomical landmarks and findings in the small bowel, in measuring lesions, and in rating bowel cleanliness. It is still questioned whether artificial intelligence will enter the market with proprietary, built-in or plug-in software, or with a universal cloud-based service, and how it will be accepted by physicians and patients.

Progress in Nanorobotics for Advancing Biomedicine

Nanorobotics, which has long been a fantasy in the realm of science fiction, is now a reality due to the considerable developments in diverse fields including chemistry, materials, physics, information and nanotechnology in the past decades. Not only different prototypes of nanorobots whose sizes are nanoscale are invented for various biomedical applications, but also robotic nanomanipulators which are able to handle nano-objects obtain substantial achievements for applications in biomedicine. The outstanding achievements in nanorobotics have significantly expanded the field of medical robotics and yielded novel insights into the underlying mechanisms guiding life activities, remarkably showing an emerging and promising way for advancing the diagnosis & treatment level in the coming era of personalized precision medicine. In this review, the recent advances in nanorobotics (nanorobots, nanorobotic manipulations) for biomedical applications are summarized from several facets (including molecular machines, nanomotors, DNA nanorobotics, and robotic nanomanipulators), and the future perspectives are also presented.

Artificial intelligence in gastrointestinal endoscopy

BACKGROUND AND AIMS

Artificial intelligence (AI)-based applications have transformed several industries and are widely used in various consumer products and services. In medicine, AI is primarily being used for image classification and natural language processing and has great potential to affect image-based specialties such as radiology, pathology, and gastroenterology (GE). This document reviews the reported applications of AI in GE, focusing on endoscopic image analysis.

METHODS

The MEDLINE database was searched through May 2020 for relevant articles by using key words such as machine learning, deep learning, artificial intelligence, computer-aided diagnosis, convolutional neural networks, GI endoscopy, and endoscopic image analysis. References and citations of the retrieved articles were also evaluated to identify pertinent studies. The manuscript was drafted by 2 authors and reviewed in person by members of the American Society for Gastrointestinal Endoscopy Technology Committee and subsequently by the American Society for Gastrointestinal Endoscopy Governing Board.

RESULTS

Deep learning techniques such as convolutional neural networks have been used in several areas of GI endoscopy, including colorectal polyp detection and classification, analysis of endoscopic images for diagnosis of Helicobacter pylori infection, detection and depth assessment of early gastric cancer, dysplasia in Barrett's esophagus, and detection of various abnormalities in wireless capsule endoscopy images.

CONCLUSIONS

The implementation of AI technologies across multiple GI endoscopic applications has the potential to transform clinical practice favorably and improve the efficiency and accuracy of current diagnostic methods.

Zoom in Lesions for Better Diagnosis: Attention Guided Deformation Network for WCE Image Classification

Wireless capsule endoscopy (WCE) is a novel imaging tool that allows noninvasive visualization of the entire gastrointestinal (GI) tract without causing discomfort to patients. Convolutional neural networks (CNNs), though perform favorably against traditional machine learning methods, show limited capacity in WCE image classification due to the small lesions and background interference. To overcome these limits, we propose a two-branch Attention Guided Deformation Network (AGDN) for WCE image classification. Specifically, the attention maps of branch1 are utilized to guide the amplification of lesion regions on the input images of branch2, thus leading to better representation and inspection of the small lesions. What's more, we devise and insert Third-order Long-range Feature Aggregation (TLFA) modules into the network. By capturing long-range dependencies and aggregating contextual features, TLFAs endow the network with a global contextual view and stronger feature representation and discrimination capability. Furthermore, we propose a novel Deformation based Attention Consistency (DAC) loss to refine the attention maps and achieve the mutual promotion of the two branches. Finally, the global feature embeddings from the two branches are fused to make image label predictions. Extensive experiments show that the proposed AGDN outperforms state-of-the-art methods with an overall classification accuracy of 91.29% on two public WCE datasets. The source code is available at https://github.com/hathawayxxh/WCE-AGDN.

Clinical Performance of New Software to Automatically Detect Angioectasias in Small Bowel Capsule Endoscopy

Background

Video capsule endoscopy (VCE) revolutionized the diagnosis and management of obscure gastrointestinal bleeding, though the rate of detection of small bowel lesions by the physician is still disappointing. Our group developed a novel algorithm (CMEMS-Uminho) to automatically detect angioectasias which display greater accuracy in VCE static frames than other methods previously published. We aimed to evaluate the algorithm overall performance and assess its diagnostic yield and usability in clinical practice.

Methods

Algorithm overall performance was determined using 54 full-length VCE recordings. To assess its diagnostic yield and usability in clinical practice, 38 VCE examinations with the clinical diagnosis of angioectasias consecutively performed (2017-2018) were evaluated by three physicians with different experiences. The CMEMS-Uminho algorithm was also applied. The performance of the CMEMS-Uminho algorithm was defined by a positive concordance between a frame automatically selected by the software and a study independent capsule endoscopist.

Results

Overall performance in complete VCE recordings was 77.7%, and diagnostic yield was 94.7%. There were significant differences between physicians in regard to global detection rate (p < 0.001), detection rate per capsule (p < 0.001), diagnostic yield (p = 0.007), true positive rate (p < 0.001), time (p < 0.001), and speed viewing (p < 0.001). The application of CMEMS-Uminho algorithm significantly enhanced all readers' global detection rate (p < 0.001) and the differences between them were no longer observed.

Conclusion

The CMEMS-Uminho algorithm detained a good overall performance and was able to enhance physicians' performance, suggesting a potential usability of this tool in clinical practice.

Low Complexity CNN Structure for Automatic Bleeding Zone Detection in Wireless Capsule Endoscopy Imaging

Wireless capsule endoscopy (WCE) is a swallowable device used for screening different parts of the human digestive system. Automatic WCE image analysis methods reduce the duration of the screening procedure and alleviate the burden of manual screening by medical experts. Recent studies widely employ convolutional neural networks (CNNs) for automatic analysis of WCE images; however, these studies do not consider CNN's structural and computational complexities. In this paper, we address the problem of simplifying the CNN's structure. A low complexity CNN structure for bleeding zone detection is proposed which takes a single patch as input and then outputs a segmented patch of the same size. The proposed network is inspired by the FCN paradigm with a simplified structure. Since it is based on image patches, the resulting network benefits from moderate-sized intermediate feature maps. Moreover, the problem of redundant computations in patch-based methods is circumvented by non-overlapping patch processing. The proposed method is evaluated using the publicly available KID dataset for WCE image analysis. Experimental results show that the proposed network has better accuracy and AUC than previous structures while requiring less computational operations.

WCE polyp detection based on novel feature descriptor with normalized variance locality-constrained linear coding

PURPOSE

Wireless capsule endoscopy (WCE) has become an effective facility to detect digestive tract diseases. To further improve the accuracy and efficiency of computer-aided diagnosis system in the detection of intestine polyps, a novel algorithm is proposed for WCE polyp detection in this paper.

METHODS

First, by considering the rich color information of endoscopic images, a novel local color texture feature called histogram of local color difference (LCDH) is proposed for describing endoscopic images. A codebook acquisition method which is based upon positive samples is also proposed, generating more balanced visual words with the LCDH features. Furthermore, based on locality-constrained linear coding (LLC) algorithm, a normalized variance regular term is introduced as NVLLC algorithm, which considers the dispersion degree between k nearest visual words and features in the approximate coding phase. The final image representations are obtained from using the spatial matching pyramid model. Finally, the support vector machine is employed to classify the polyp images.

RESULTS

The WCE dataset including 500 polyp and 500 normal images is adopted for evaluating the proposed method. Experimental results indicate that the classification accuracy, sensitivity and specificity have reached 96.00%, 95.80% and 96.20%, which performances better than traditional ways.

CONCLUSION

A novel method for WCE polyp detection is developed using LCDH feature descriptor and NVLLC coding scheme, which achieves a promising performance and can be implemented in clinical-assisted diagnosis of intestinal diseases.

A Fluorescence-Based Wireless Capsule Endoscopy System for Detecting Colorectal Cancer

Wireless capsule endoscopy (WCE) has been widely used in gastrointestinal (GI) diagnosis that allows the physicians to examine the interior wall of the human GI tract through a pain-free procedure. However, there are still several limitations of the technology, which limits its functionality, ultimately limiting its wide acceptance. Its counterpart, the wired endoscopic system is a painful procedure that demotivates patients from going through the procedure, and adversely affects early diagnosis. Furthermore, the current generation of capsules is unable to automate the detection of abnormality. As a result, physicians are required to spend longer hours to examine each image from the endoscopic capsule for abnormalities, which makes this technology tiresome and error-prone. Early detection of cancer is important to improve the survival rate in patients with colorectal cancer. Hence, a fluorescence-imaging-based endoscopic capsule that automates the detection process of colorectal cancer was designed and developed in our lab. The proof of concept of this endoscopic capsule was tested on porcine intestine and liquid phantom. The proposed WCE system offers great possibilities for future applicability in selective and specific detection of other fluorescently labelled cancers.

CAD-CAP: a 25,000-image database serving the development of artificial intelligence for capsule endoscopy

Background and study aims  Capsule endoscopy (CE) is the preferred method for small bowel (SB) exploration. With a mean number of 50,000 SB frames per video, SBCE reading is time-consuming and tedious (30 to 60 minutes per video). We describe a large, multicenter database named CAD-CAP (Computer-Assisted Diagnosis for CAPsule Endoscopy, CAD-CAP). This database aims to serve the development of CAD tools for CE reading. Materials and methods  Twelve French endoscopy centers were involved. All available third-generation SB-CE videos (Pillcam, Medtronic) were retrospectively selected from these centers and deidentified. Any pathological frame was extracted and included in the database. Manual segmentation of findings within these frames was performed by two pre-med students trained and supervised by an expert reader. All frames were then classified by type and clinical relevance by a panel of three expert readers. An automated extraction process was also developed to create a dataset of normal, proofread, control images from normal, complete, SB-CE videos. Results  Four-thousand-one-hundred-and-seventy-four SB-CE were included. Of them, 1,480 videos (35 %) containing at least one pathological finding were selected. Findings from 5,184 frames (with their short video sequences) were extracted and delimited: 718 frames with fresh blood, 3,097 frames with vascular lesions, and 1,369 frames with inflammatory and ulcerative lesions. Twenty-thousand normal frames were extracted from 206 SB-CE normal videos. CAD-CAP has already been used for development of automated tools for angiectasia detection and also for two international challenges on medical computerized analysis.

Application of Artificial Intelligence to Gastroenterology and Hepatology

Since 2010, substantial progress has been made in artificial intelligence (AI) and its application to medicine. AI is explored in gastroenterology for endoscopic analysis of lesions, in detection of cancer, and to facilitate the analysis of inflammatory lesions or gastrointestinal bleeding during wireless capsule endoscopy. AI is also tested to assess liver fibrosis and to differentiate patients with pancreatic cancer from those with pancreatitis. AI might also be used to establish prognoses of patients or predict their response to treatments, based on multiple factors. We review the ways in which AI may help physicians make a diagnosis or establish a prognosis and discuss its limitations, knowing that further randomized controlled studies will be required before the approval of AI techniques by the health authorities.

The future of capsule endoscopy in clinical practice: from diagnostic to therapeutic experimental prototype capsules

Capsule endoscopy (CE) is indicated as a first-line clinical examination for the detection of small-bowel pathology, and there is an ever-growing drive for it to become a method for the screening of the entire gastrointestinal tract (GI). Although CE's main function is diagnosis, the research for therapeutic capabilities has intensified to make therapeutic capsule endoscopy (TCE) a target within reach. This manuscript presents the research evolution of CE and TCE through the last 5 years and describes notable problems, as well as clinical and technological challenges to overcome. This review also reports the state-of-the-art of capsule devices with a focus on CE research prototypes promising an enhanced diagnostic yield (DY) and treatment. Lastly, this article provides an overview of the research progress made in software for enhancing DY by increasing the accuracy of abnormality detection and lesion localisation.

Recent Development of Computer Vision Technology to Improve Capsule Endoscopy

Capsule endoscopy (CE) is a preferred diagnostic method for analyzing small bowel diseases. However, capsule endoscopes capture a sparse number of images because of their mechanical limitations. Post-procedural management using computational methods can enhance image quality. Additional information, including depth, can be obtained by using recently developed computer vision techniques. It is possible to measure the size of lesions and track the trajectory of capsule endoscopes using the computer vision technology, without requiring additional equipment. Moreover, the computational analysis of CE images can help detect lesions more accurately within a shorter time. Newly introduced deep leaning-based methods have shown more remarkable results over traditional computerized approaches. A large-scale standard dataset should be prepared to develop an optimal algorithms for improving the diagnostic yield of CE. The close collaboration between information technology and medical professionals is needed.

Follow-up on: optimizing lesion detection in small bowel capsule endoscopy and beyond: from present problems to future solutions

This review presents noteworthy advances in clinical and experimental Capsule Endoscopy (CE), focusing on the progress that has been reported over the last 5 years since our previous review on the subject. Areas covered: This study presents the commercially available CE platforms, as well as the advances made in optimizing the diagnostic capabilities of CE. The latter includes recent concept and prototype capsule endoscopes, medical approaches to improve diagnostic yield, and progress in software for enhancing visualization, abnormality detection, and lesion localization. Expert commentary: Currently, moving through the second decade of CE evolution, there are still several open issues and remarkable challenges to overcome.