Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images

Endoscopic diagnosis and treatment of gastric dysplasia and early cancer: Current evidence and what the future may hold

Gastric cancer accounts for a significant proportion of worldwide cancer-related morbidity and mortality. The well documented precancerous cascade provides an opportunity for clinicians to detect and treat gastric cancers at an endoscopically curable stage. In high prevalence regions such as Japan and Korea, this has led to the implementation of population screening programs. However, guidelines remain ambiguous in lower prevalence regions. In recent years, there have been many advances in the endoscopic diagnosis and treatment of early gastric cancer and precancerous lesions. More advanced endoscopic imaging has led to improved detection and characterization of gastric lesions as well as superior accuracy for delineation of margins prior to resection. In addition, promising early data on artificial intelligence in gastroscopy suggests a future role for this technology in maximizing the yield of advanced endoscopic imaging. Data on endoscopic resection (ER) are particularly robust in Japan and Korea, with high rates of curative ER and markedly reduced procedural morbidity. However, there is a shortage of data in other regions to support the applicability of protocols from these high prevalence countries. Future advances in endoscopic therapeutics will likely lead to further expansion of the current indications for ER, as both technology and proceduralist expertise continue to grow.

Helicobacter pylori eradication: Exploring its impacts on the gastric mucosa

Helicobacter pylori (H. pylori) infects approximately 50% of all humans globally. Persistent H. pylori infection causes multiple gastric and extragastric diseases, indicating the importance of early diagnosis and timely treatment. H. pylori eradication produces dramatic changes in the gastric mucosa, resulting in restored function. Consequently, to better understand the importance of H. pylori eradication and clarify the subsequent recovery of gastric mucosal functions after eradication, we summarize histological, endoscopic, and gastric microbiota changes to assess the therapeutic effects on the gastric mucosa.

Use of a convolutional neural network for classifying microvessels of superficial esophageal squamous cell carcinomas

BACKGROUND AND AIM

The morphological diagnosis of microvessels on the surface of superficial esophageal squamous cell carcinomas using magnifying endoscopy with narrow-band imaging is widely used in clinical practice. Nevertheless, inconsistency, even among experts, remains a problem. We constructed a convolutional neural network-based computer-aided diagnosis system to classify the microvessels of superficial esophageal squamous cell carcinomas and evaluated its diagnostic performance.

METHODS

In this retrospective study, a cropped magnifying endoscopy with narrow-band images from superficial esophageal squamous cell carcinoma lesions was used as the dataset. All images were assessed by three experts, and classified into three classes, Type B1, B2, and B3, based on the Japan Esophagus Society classification. The dataset was divided into training and validation datasets. A convolutional neural network model (ResNeXt-101) was trained and tuned with the training dataset. To evaluate diagnostic accuracy, the validation dataset was assessed by the computer-aided diagnosis system and eight endoscopists.

RESULTS

In total, 1777 and 747 cropped images (total, 393 lesions) were included in the training and validation datasets, respectively. The diagnosis system took 20.3 s to evaluate the 747 images in the validation dataset. The microvessel classification accuracy of the computer-aided diagnosis system was 84.2%, which was higher than the average of the eight endoscopists (77.8%, P < 0.001). The area under the receiver operating characteristic curves for diagnosing Type B1, B2, and B3 vessels were 0.969, 0.948, and 0.973, respectively.

CONCLUSIONS

The computer-aided diagnosis system showed remarkable performance in the classification of microvessels on superficial esophageal squamous cell carcinomas.

Artificial Intelligence in Upper Gastrointestinal Endoscopy

BACKGROUND

Over the past decade, several artificial intelligence (AI) systems are developed to assist in endoscopic assessment of (pre-)cancerous lesions of the gastrointestinal (GI) tract. In this review, we aimed to provide an overview of the possible indications of AI technology in upper GI endoscopy and hypothesize about potential challenges for its use in clinical practice.

SUMMARY

Application of AI in upper GI endoscopy has been investigated for several indications: (1) detection, characterization, and delineation of esophageal and gastric cancer (GC) and their premalignant conditions; (2) prediction of tumor invasion; and (3) detection of Helicobacter pylori. AI systems show promising results with an accuracy of up to 99% for the detection of superficial and advanced upper GI cancers. AI outperformed trainee and experienced endoscopists for the detection of esophageal lesions and atrophic gastritis. For GC, AI outperformed mid-level and trainee endoscopists but not expert endoscopists.

KEY MESSAGES

Application of artificial intelligence (AI) in upper gastrointestinal endoscopy may improve early diagnosis of esophageal and gastric cancer and may enable endoscopists to better identify patients eligible for endoscopic resection. The benefit of AI on the quality of upper endoscopy still needs to be demonstrated, while prospective trials are needed to confirm accuracy and feasibility during real-time daily endoscopy.

Role of artificial intelligence in multidisciplinary imaging diagnosis of gastrointestinal diseases

The use of artificial intelligence-based tools is regarded as a promising approach to increase clinical efficiency in diagnostic imaging, improve the interpretability of results, and support decision-making for the detection and prevention of diseases. Radiology, endoscopy and pathology images are suitable for deep-learning analysis, potentially changing the way care is delivered in gastroenterology. The aim of this review is to examine the key aspects of different neural network architectures used for the evaluation of gastrointestinal conditions, by discussing how different models behave in critical tasks, such as lesion detection or characterization (i.e. the distinction between benign and malignant lesions of the esophagus, the stomach and the colon). To this end, we provide an overview on recent achievements and future prospects in deep learning methods applied to the analysis of radiology, endoscopy and histologic whole-slide images of the gastrointestinal tract.

Deep neural network approaches for detecting gastric polyps in endoscopic images

Gastrointestinal endoscopy is the primary method used for the diagnosis and treatment of gastric polyps. The early detection and removal of polyps is vitally important in preventing cancer development. Many studies indicate that a high workload can contribute to misdiagnosing gastric polyps, even for experienced physicians. In this study, we aimed to establish a deep learning-based computer-aided diagnosis system for automatic gastric polyp detection. A private gastric polyp dataset was generated for this purpose consisting of 2195 endoscopic images and 3031 polyp labels. Retrospective gastrointestinal endoscopy data from the Karadeniz Technical University, Farabi Hospital, were used in the study. YOLOv4, CenterNet, EfficientNet, Cross Stage ResNext50-SPP, YOLOv3, YOLOv3-SPP, Single Shot Detection, and Faster Regional CNN deep learning models were implemented and assessed to determine the most efficient model for precancerous gastric polyp detection. The dataset was split 70% and 30% for training and testing all the implemented models. YOLOv4 was determined to be the most accurate model, with an 87.95% mean average precision. We also evaluated all the deep learning models using a public gastric polyp dataset as the test data. The results show that YOLOv4 has significant potential applicability in detecting gastric polyps and can be used effectively in gastrointestinal CAD systems. Gastric Polyp Detection Process using Deep Learning with Private Dataset.

Performance of a deep learning-based identification system for esophageal cancer from CT images

BACKGROUND

Because cancers of hollow organs such as the esophagus are hard to detect even by the expert physician, it is important to establish diagnostic systems to support physicians and increase the accuracy of diagnosis. In recent years, deep learning-based artificial intelligence (AI) technology has been employed for medical image recognition. However, no optimal CT diagnostic system employing deep learning technology has been attempted and established for esophageal cancer so far.

PURPOSE

To establish an AI-based diagnostic system for esophageal cancer from CT images.

MATERIALS AND METHODS

In this single-center, retrospective cohort study, 457 patients with primary esophageal cancer referred to our division between 2005 and 2018 were enrolled. We fine-tuned VGG16, an image recognition model of deep learning convolutional neural network (CNN), for the detection of esophageal cancer. We evaluated the diagnostic accuracy of the CNN using a test data set including 46 cancerous CT images and 100 non-cancerous images and compared it to that of two radiologists.

RESULTS

Pre-treatment esophageal cancer stages of the patients included in the test data set were clinical T1 (12 patients), clinical T2 (9 patients), clinical T3 (20 patients), and clinical T4 (5 patients). The CNN-based system showed a diagnostic accuracy of 84.2%, F value of 0.742, sensitivity of 71.7%, and specificity of 90.0%.

CONCLUSIONS

Our AI-based diagnostic system succeeded in detecting esophageal cancer with high accuracy. More training with vast datasets collected from multiples centers would lead to even higher diagnostic accuracy and aid better decision making.

Diagnosis of gastric lesions through a deep convolutional neural network

BACKGROUND AND AIMS

A deep convolutional neural network (CNN) was used to achieve fast and accurate artificial intelligence (AI)-assisted diagnosis of early gastric cancer (GC) and other gastric lesions based on endoscopic images.

METHODS

A CNN-based diagnostic system based on a ResNet34 residual network structure and a DeepLabv3 structure was constructed and trained using 21,217 gastroendoscopic images of five gastric conditions, peptic ulcer (PU), early gastric cancer (EGC) and high-grade intraepithelial neoplasia (HGIN), advanced gastric cancer (AGC), gastric submucosal tumors (SMTs), and normal gastric mucosa without lesions. The trained CNN was evaluated using a test dataset of 1091 images. The accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the CNN were calculated. The CNN diagnosis was compared with those of 10 endoscopists with over 8 years of experience in endoscopic diagnosis.

RESULTS

The diagnostic specificity and PPV of the CNN were higher than that of the endoscopists for the EGC and HGIN images (specificity: 91.2% vs. 86.7%, by 4.5%, 95% CI 2.8-7.2%; PPV: 55.4% vs. 41.7%, by 13.7%, 95% CI 11.2-16.8%) and the diagnostic accuracy of the CNN was close to those of the endoscopists for the lesion-free, EGC and HGIN, PU, AGC, and SMTs images. The CNN had image recognition time of 42 s for all the test set images.

CONCLUSION

The constructed CNN system could be used as a rapid auxiliary diagnostic instrument to detect EGC and HGIN, as well as other gastric lesions, to reduce the workload of endoscopists.

Endoscopic Advances for Gastric Neoplasia Detection

This article explores advances in endoscopic neoplasia detection with supporting clinical evidence and future aims. The ability to detect early gastric neoplastic lesions amenable to curative endoscopic submucosal dissection provides the opportunity to decrease gastric cancer mortality rates. Newer imaging techniques offer enhanced views of mucosal and microvascular structures and show promise in differentiating benign from malignant lesions and improving targeted biopsies. Conventional chromoendoscopy is well studied and validated. Narrow band imaging demonstrates superiority over magnified white light. Autofluorescence imaging, i-scan, flexible spectral imaging color enhancement, and bright image enhanced endoscopy show promise but insufficient evidence to change current clinical practice.

Usefulness of artificial intelligence in gastric neoplasms

Recently, studies in many medical fields have reported that image analysis based on artificial intelligence (AI) can be used to analyze structures or features that are difficult to identify with human eyes. To diagnose early gastric cancer, related efforts such as narrow-band imaging technology are on-going. However, diagnosis is often difficult. Therefore, a diagnostic method based on AI for endoscopic imaging was developed and its effectiveness was confirmed in many studies. The gastric cancer diagnostic program based on AI showed relatively high diagnostic accuracy and could differentially diagnose non-neoplastic lesions including benign gastric ulcers and dysplasia. An AI system has also been developed that helps to predict the invasion depth of gastric cancer through endoscopic images and observe the stomach during endoscopy without blind spots. Therefore, if AI is used in the field of endoscopy, it is expected to aid in the diagnosis of gastric neoplasms and determine the application of endoscopic therapy by predicting the invasion depth.

Application of convolutional neural network in detecting and classifying gastric cancer

Gastric cancer (GC) is the fifth most common cancer in the world, and at present, esophagogastroduodenoscopy is recognized as an acceptable method for the screening and monitoring of GC. Convolutional neural networks (CNNs) are a type of deep learning model and have been widely used for image analysis. This paper reviews the application and prospects of CNNs in detecting and classifying GC, aiming to introduce a computer-aided diagnosis system and to provide evidence for subsequent studies.

Artificial intelligence in gastrointestinal diseases

Artificial intelligence (AI) applications are growing in medicine. It is important to understand the current state of the AI applications prior to utilizing in disease research and treatment. In this review, AI application in the diagnosis and treatment of gastrointestinal diseases are studied and summarized. In most cases, AI studies had large amounts of data, including images, to learn to distinguish disease characteristics according to a human’s perspectives. The detailed pros and cons of utilizing AI approaches should be investigated in advance to ensure the safe application of AI in medicine. Evidence suggests that the collaborative usage of AI in both diagnosis and treatment of diseases will increase the precision and effectiveness of medicine. Recent progress in genome technology such as genome editing provides a specific example where AI has revealed the diagnostic and therapeutic possibilities of RNA detection and targeting.

Object and anatomical feature recognition in surgical video images based on a convolutional neural network

Purpose

Artificial intelligence-enabled techniques can process large amounts of surgical data and may be utilized for clinical decision support to recognize or forecast adverse events in an actual intraoperative scenario. To develop an image-guided navigation technology that will help in surgical education, we explored the performance of a convolutional neural network (CNN)-based computer vision system in detecting intraoperative objects.

Methods

The surgical videos used for annotation were recorded during surgeries conducted in the Department of Surgery of Tokyo Women’s Medical University from 2019 to 2020. Abdominal endoscopic images were cut out from manually captured surgical videos. An open-source programming framework for CNN was used to design a model that could recognize and segment objects in real time through IBM Visual Insights. The model was used to detect the GI tract, blood, vessels, uterus, forceps, ports, gauze and clips in the surgical images.

Results

The accuracy, precision and recall of the model were 83%, 80% and 92%, respectively. The mean average precision (mAP), the calculated mean of the precision for each object, was 91%. Among surgical tools, the highest recall and precision of 96.3% and 97.9%, respectively, were achieved for forceps. Among the anatomical structures, the highest recall and precision of 92.9% and 91.3%, respectively, were achieved for the GI tract.

Conclusion

The proposed model could detect objects in operative images with high accuracy, highlighting the possibility of using AI-based object recognition techniques for intraoperative navigation. Real-time object recognition will play a major role in navigation surgery and surgical education.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-021-02434-w.

Application of artificial intelligence-driven endoscopic screening and diagnosis of gastric cancer

The landscape of gastrointestinal endoscopy continues to evolve as new technologies and techniques become available. The advent of image-enhanced and magnifying endoscopies has highlighted the step toward perfecting endoscopic screening and diagnosis of gastric lesions. Simultaneously, with the development of convolutional neural network, artificial intelligence (AI) has made unprecedented breakthroughs in medical imaging, including the ongoing trials of computer-aided detection of colorectal polyps and gastrointestinal bleeding. In the past demi-decade, applications of AI systems in gastric cancer have also emerged. With AI’s efficient computational power and learning capacities, endoscopists can improve their diagnostic accuracies and avoid the missing or mischaracterization of gastric neoplastic changes. So far, several AI systems that incorporated both traditional and novel endoscopy technologies have been developed for various purposes, with most systems achieving an accuracy of more than 80%. However, their feasibility, effectiveness, and safety in clinical practice remain to be seen as there have been no clinical trials yet. Nonetheless, AI-assisted endoscopies shed light on more accurate and sensitive ways for early detection, treatment guidance and prognosis prediction of gastric lesions. This review summarizes the current status of various AI applications in gastric cancer and pinpoints directions for future research and clinical practice implementation from a clinical perspective.

[Deep Learning in Upper Gastrointestinal Disorders: Status and Future Perspectives]

Artificial intelligence using deep learning has been applied to gastrointestinal disorders for the detection, classification, and delineation of various lesion images. With the accumulation of enormous medical records, the evolution of computation power with graphic processing units, and the widespread use of open-source libraries in large-scale machine learning processes, medical artificial intelligence is overcoming its traditional limitations. This paper explains the basic concepts of deep learning model establishment and summarizes previous studies on upper gastrointestinal disorders. The limitations and perspectives on future development are also discussed.

Status quo and future prospects of artificial neural network from the perspective of gastroenterologists

Artificial neural networks (ANNs) are one of the primary types of artificial intelligence and have been rapidly developed and used in many fields. In recent years, there has been a sharp increase in research concerning ANNs in gastrointestinal (GI) diseases. This state-of-the-art technique exhibits excellent performance in diagnosis, prognostic prediction, and treatment. Competitions between ANNs and GI experts suggest that efficiency and accuracy might be compatible in virtue of technique advancements. However, the shortcomings of ANNs are not negligible and may induce alterations in many aspects of medical practice. In this review, we introduce basic knowledge about ANNs and summarize the current achievements of ANNs in GI diseases from the perspective of gastroenterologists. Existing limitations and future directions are also proposed to optimize ANN’s clinical potential. In consideration of barriers to interdisciplinary knowledge, sophisticated concepts are discussed using plain words and metaphors to make this review more easily understood by medical practitioners and the general public.

The role of computer-assisted systems for upper-endoscopy quality monitoring and assessment of gastric lesions

This article analyses the literature regarding the value of computer-assisted systems in esogastroduodenoscopy-quality monitoring and the assessment of gastric lesions. Current data show promising results in upper-endoscopy quality control and a satisfactory detection accuracy of gastric premalignant and malignant lesions, similar or even exceeding that of experienced endoscopists. Moreover, artificial systems enable the decision for the best treatment strategies in gastric-cancer patient care, namely endoscopic vs surgical resection according to tumor depth. In so doing, unnecessary surgical interventions would be avoided whilst providing a better quality of life and prognosis for these patients. All these performance data have been revealed by numerous studies using different artificial intelligence (AI) algorithms in addition to white-light endoscopy or novel endoscopic techniques that are available in expert endoscopy centers. It is expected that ongoing clinical trials involving AI and the embedding of computer-assisted diagnosis systems into endoscopic devices will enable real-life implementation of AI endoscopic systems in the near future and at the same time will help to overcome the current limits of the computer-assisted systems leading to an improvement in performance. These benefits should lead to better diagnostic and treatment strategies for gastric-cancer patients. Furthermore, the incorporation of AI algorithms in endoscopic tools along with the development of large electronic databases containing endoscopic images might help in upper-endoscopy assistance and could be used for telemedicine purposes and second opinion for difficult cases.

Differential diagnosis for esophageal protruded lesions using a deep convolution neural network in endoscopic images

BACKGROUND AND AIMS

Recent advances in deep convolutional neural networks (CNNs) have led to remarkable results in digestive endoscopy. In this study, we aimed to develop CNN-based models for the differential diagnosis of benign esophageal protruded lesions using endoscopic images acquired during real clinical settings.

METHODS

We retrospectively reviewed the images from 1217 patients who underwent white-light endoscopy (WLE) and EUS between January 2015 and April 2020. Three deep CNN models were developed to accomplish the following tasks: (1) identification of esophageal benign lesions from healthy controls using WLE images; (2) differentiation of 3 subtypes of esophageal protruded lesions (including esophageal leiomyoma [EL], esophageal cyst (EC], and esophageal papilloma [EP]) using WLE images; and (3) discrimination between EL and EC using EUS images. Six endoscopists blinded to the patients' clinical status were enrolled to interpret all images independently. Their diagnostic performances were evaluated and compared with the CNN models using the area under the receiver operating characteristic curve (AUC).

RESULTS

For task 1, the CNN model achieved an AUC of 0.751 (95% confidence interval [CI], 0.652-0.850) in identifying benign esophageal lesions. For task 2, the proposed model using WLE images for differentiation of esophageal protruded lesions achieved an AUC of 0.907 (95% CI, 0.835-0.979), 0.897 (95% CI, 0.841-0.953), and 0.868 (95% CI, 0.769-0.968) for EP, EL, and EC, respectively. The CNN model achieved equivalent or higher identification accuracy for EL and EC compared with skilled endoscopists. In the task of discriminating EL from EC (task 3), the proposed CNN model had AUC values of 0.739 (EL, 95% CI, 0.600-0.878) and 0.724 (EC, 95% CI, 0.567-0.881), which outperformed seniors and novices. Attempts to combine the CNN and endoscopist predictions led to significantly improved diagnostic accuracy compared with endoscopists interpretations alone.

CONCLUSIONS

Our team established CNN-based methodologies to recognize benign esophageal protruded lesions using routinely obtained WLE and EUS images. Preliminary results combining the results from the models and the endoscopists underscored the potential of ensemble models for improved differentiation of lesions in real endoscopic settings.

Detection and Classification of Gastrointestinal Diseases using Machine Learning

BACKGROUND

Traditional endoscopy is an invasive and painful method of examining the gastrointestinal tract (GIT) not supported by physicians and patients. To handle this issue, video endoscopy (VE) or wireless capsule endoscopy (WCE) is recommended and utilized for GIT examination. Furthermore, manual assessment of captured images is not possible for an expert physician because it's a time taking task to analyze thousands of images thoroughly. Hence, there comes the need for a Computer-Aided-Diagnosis (CAD) method to help doctors analyze images. Many researchers have proposed techniques for automated recognition and classification of abnormality in captured images.

METHODS

In this article, existing methods for automated classification, segmentation and detection of several GI diseases are discussed. Paper gives a comprehensive detail about these state-of-theart methods. Furthermore, literature is divided into several subsections based on preprocessing techniques, segmentation techniques, handcrafted features based techniques and deep learning based techniques. Finally, issues, challenges and limitations are also undertaken.

RESULTS

A comparative analysis of different approaches for the detection and classification of GI infections.

CONCLUSION

This comprehensive review article combines information related to a number of GI diseases diagnosis methods at one place. This article will facilitate the researchers to develop new algorithms and approaches for early detection of GI diseases detection with more promising results as compared to the existing ones of literature.

Application of Convolutional Neural Networks for Detection of Superficial Nonampullary Duodenal Epithelial Tumors in Esophagogastroduodenoscopic Images

A superficial nonampullary duodenal epithelial tumor (SNADET) is defined as a mucosal or submucosal sporadic tumor of the duodenum that does not arise from the papilla of Vater. SNADETs rarely metastasize to the lymph nodes, and most can be treated endoscopically. However, SNADETs are sometimes missed during esophagogastroduodenoscopic examination. In this study, we constructed a convolutional neural network (CNN) and evaluated its ability to detect SNADETs.

Differentiation Between Malignant and Benign Endoscopic Images of Gastric Ulcers Using Deep Learning

Background and Aim

Endoscopic differentiation between malignant and benign gastric ulcers (GU) affects further evaluation and prognosis. The aim of our study was to evaluate a deep learning algorithm for discrimination between benign and malignant GU in a database of endoscopic ulcer images.

Methods

We retrospectively collected consecutive upper gastrointestinal endoscopy images of GU performed between 2011 and 2019 at the Sheba Medical Center. All ulcers had a corresponding histopathology result of either benign peptic ulcer or gastric adenocarcinoma. A convolutional neural network (CNN) was trained to classify the images into either benign or malignant. Endoscopies from 2011 to 2017 were used for training (2011-2015) and validation (2016-2017). Hyper-parameters, image augmentation and pre-training on Google images obtained images were evaluated on the validation data. Held-out data from 2018 to 2019 was used for testing the final model.

Results

Overall, the Sheba dataset included 1978 GU images; 1894 images from benign GU and 84 images of malignant ulcers. The final CNN model showed an AUC 0.91 (95% CI 0.85-0.96) for detecting malignant ulcers. For cut-off probability 0.5, the network showed a sensitivity of 92% and specificity of 75% for malignant ulcers.

Conclusion

Our study displays the applicability of a CNN model for automated evaluation of gastric ulcers images for malignant potential. Following further research, the algorithm may improve accuracy of differentiating benign from malignant ulcers during endoscopies and assist in patients' stratification, allowing accelerated patients management and individualized approach towards surveillance endoscopy.

Accurate measurement of colorectal polyps using computer-aided analysis

INTRODUCTION

As we know, the majority of colorectal cancers are thought to evolve from colorectal adenomas. In this study, we explored the use of Computer-aided diagnosis (CAD) in the detection of colorectal polyps and the estimation of their sizes, which is important for the diagnosis and management of colorectal cancer.

MATERIALS AND METHODS

As the distance between colonoscopy and lesion increases, magnification tends to decrease. Therefore, the size of colorectal polyps can be calculated by taking into account the captured image and the shooting distance. In this study, the fitting curve of the magnification of electronic colonoscopy was obtained by simulating intestinal tract and polyps in vitro. Then, the distance was artificially controlled in the endoscopic operation, and the image was taken at a preset distance. The CAD system was then trained on the overall shape of colorectal polyps. Image segmentation was employed to accurately identify colorectal polyps. Finally, on the basis of the magnification factor, the real value of polyps was predicted from the shooting distance and the segmentation image size.

RESULTS

The CAD system can automatically calculate the range of colorectal polyps and calculate the true size of the colorectal polyps according to the magnification of the corresponding distance.

CONCLUSIONS

In this study, we developed a method of accurately estimating the size of colorectal polyps. This approach is compatible with many devices, which would expand its range of applications. This method has the potential for application in other areas of clinical diagnosis.

Improved classification and localization approach to small bowel capsule endoscopy using convolutional neural network

BACKGROUND

Although great advances in artificial intelligence for interpreting small bowel capsule endoscopy (SBCE) images have been made in recent years, its practical use is still limited. The aim of this study was to develop a more practical convolutional neural network (CNN) algorithm for the automatic detection of various small bowel lesions.

METHODS

A total of 7556 images were collected for the training dataset from 526 SBCE videos. Abnormal images were classified into two categories: hemorrhagic lesions (red spot/angioectasia/active bleeding) and ulcerative lesions (erosion/ulcer/stricture). A CNN algorithm based on VGGNet was trained in two different ways: the combined model (hemorrhagic and ulcerative lesions trained separately) and the binary model (all abnormal images trained without discrimination). The detected lesions were visualized using a gradient class activation map (Grad-CAM). The two models were validated using 5,760 independent images taken at two other academic hospitals.

RESULTS

Both the combined and binary models acquired high accuracy for lesion detection, and the difference between the two models was not significant (96.83% vs 96.62%, P = 0.122). However, the combined model showed higher sensitivity (97.61% vs 95.07%, P < 0.001) and higher accuracy for individual lesions from the hemorrhagic and ulcerative categories than the binary model. The combined model also revealed more accurate localization of the culprit area on images evaluated by the Grad-CAM.

CONCLUSIONS

Diagnostic sensitivity and classification of small bowel lesions using a convolutional neural network are improved by the independent training for hemorrhagic and ulcerative lesions. Grad-CAM is highly effective in localizing the lesions.

Diagnosis of pharyngeal cancer on endoscopic video images by Mask region-based convolutional neural network

OBJECTIVES

We aimed to develop an artificial intelligence (AI) system for the real-time diagnosis of pharyngeal cancers.

METHODS

Endoscopic video images and still images of pharyngeal cancer treated in our facility were collected. A total of 4559 images of pathologically proven pharyngeal cancer (1243 using white light imaging and 3316 using narrow-band imaging/blue laser imaging) from 276 patients were used as a training dataset. The AI system used a convolutional neural network (CNN) model typical of the type used to analyze visual imagery. Supervised learning was used to train the CNN. The AI system was evaluated using an independent validation dataset of 25 video images of pharyngeal cancer and 36 video images of normal pharynx taken at our hospital.

RESULTS

The AI system diagnosed 23/25 (92%) pharyngeal cancers as cancers and 17/36 (47%) non-cancers as non-cancers. The transaction speed of the AI system was 0.03 s per image, which meets the required speed for real-time diagnosis. The sensitivity, specificity, and accuracy for the detection of cancer were 92%, 47%, and 66% respectively.

CONCLUSIONS

Our single-institution study showed that our AI system for diagnosing cancers of the pharyngeal region had promising performance with high sensitivity and acceptable specificity. Further training and improvement of the system are required with a larger dataset including multiple centers.

Application of deep learning in image recognition and diagnosis of gastric cancer

In recent years, artificial intelligence has been extensively applied in the diagnosis of gastric cancer based on medical imaging. In particular, using deep learning as one of the mainstream approaches in image processing has made remarkable progress. In this paper, we also provide a comprehensive literature survey using four electronic databases, PubMed, EMBASE, Web of Science, and Cochrane. The literature search is performed until November 2020. This article provides a summary of the existing algorithm of image recognition, reviews the available datasets used in gastric cancer diagnosis and the current trends in applications of deep learning theory in image recognition of gastric cancer. covers the theory of deep learning on endoscopic image recognition. We further evaluate the advantages and disadvantages of the current algorithms and summarize the characteristics of the existing image datasets, then combined with the latest progress in deep learning theory, and propose suggestions on the applications of optimization algorithms. Based on the existing research and application, the label, quantity, size, resolutions, and other aspects of the image dataset are also discussed. The future developments of this field are analyzed from two perspectives including algorithm optimization and data support, aiming to improve the diagnosis accuracy and reduce the risk of misdiagnosis.

Artificial intelligence in gastroenterology and hepatology: Status and challenges

Originally proposed by John McCarthy in 1955, artificial intelligence (AI) has achieved a breakthrough and revolutionized the processing methods of clinical medicine with the increasing workloads of medical records and digital images. Doctors are paying attention to AI technologies for various diseases in the fields of gastroenterology and hepatology. This review will illustrate AI technology procedures for medical image analysis, including data processing, model establishment, and model validation. Furthermore, we will summarize AI applications in endoscopy, radiology, and pathology, such as detecting and evaluating lesions, facilitating treatment, and predicting treatment response and prognosis with excellent model performance. The current challenges for AI in clinical application include potential inherent bias in retrospective studies that requires larger samples for validation, ethics and legal concerns, and the incomprehensibility of the output results. Therefore, doctors and researchers should cooperate to address the current challenges and carry out further investigations to develop more accurate AI tools for improved clinical applications.

A Novel Model Based on Deep Convolutional Neural Network Improves Diagnostic Accuracy of Intramucosal Gastric Cancer (With Video)

Background and Aims

Prediction of intramucosal gastric cancer (GC) is a big challenge. It is not clear whether artificial intelligence could assist endoscopists in the diagnosis.

Methods

A deep convolutional neural networks (DCNN) model was developed via retrospectively collected 3407 endoscopic images from 666 gastric cancer patients from two Endoscopy Centers (training dataset). The DCNN model’s performance was tested with 228 images from 62 independent patients (testing dataset). The endoscopists evaluated the image and video testing dataset with or without the DCNN model’s assistance, respectively. Endoscopists’ diagnostic performance was compared with or without the DCNN model’s assistance and investigated the effects of assistance using correlations and linear regression analyses.

Results

The DCNN model discriminated intramucosal GC from advanced GC with an AUC of 0.942 (95% CI, 0.915–0.970), a sensitivity of 90.5% (95% CI, 84.1%–95.4%), and a specificity of 85.3% (95% CI, 77.1%–90.9%) in the testing dataset. The diagnostic performance of novice endoscopists was comparable to those of expert endoscopists with the DCNN model’s assistance (accuracy: 84.6% vs. 85.5%, sensitivity: 85.7% vs. 87.4%, specificity: 83.3% vs. 83.0%). The mean pairwise kappa value of endoscopists was increased significantly with the DCNN model’s assistance (0.430–0.629 vs. 0.660–0.861). The diagnostic duration reduced considerably with the assistance of the DCNN model from 4.35s to 3.01s. The correlation between the perseverance of effort and diagnostic accuracy of endoscopists was diminished using the DCNN model (r: 0.470 vs. 0.076).

Conclusions

An AI-assisted system was established and found useful for novice endoscopists to achieve comparable diagnostic performance with experts.

A review on recent advancements in diagnosis and classification of cancers using artificial intelligence

Artificial intelligence has illustrated drastic changes in radiology and medical imaging techniques which in turn led to tremendous changes in screening patterns. In particular, advancements in these techniques led to the development of computer aided detection (CAD) strategy. These approaches provided highly accurate diagnostic reports which served as a "second-opinion" to the radiologists. However, with significant advancements in artificial intelligence strategy, the diagnostic and classifying capabilities of CAD system are meeting the levels of radiologists and clinicians. Thus, it shifts the CAD system from second opinion approach to a high utility tool. This article reviews the strategies and algorithms developed using artificial intelligence for the foremost cancer diagnosis and classification which overcomes the challenges in the traditional method. In addition, the possible direction of AI in medical aspects is also discussed in this study.

Artificial intelligence-assisted analysis of endoscopic retrograde cholangiopancreatography image for identifying ampulla and difficulty of selective cannulation

The advancement of artificial intelligence (AI) has facilitated its application in medical fields. However, there has been little research for AI-assisted endoscopy, despite the clinical significance of the efficiency and safety of cannulation in the endoscopic retrograde cholangiopancreatography (ERCP). In this study, we aim to assist endoscopists performing ERCP through automatic detection of the ampulla and the identification of cannulation difficulty. We developed a novel AI-assisted system based on convolutional neural networks that predict the location of the ampulla and the difficulty of cannulation to the ampulla. ERCP data of 531 and 451 patients were utilized in the evaluation of our model for each task. Our model detected the ampulla with mean intersection-over-union 64.1%, precision 76.2%, recall 78.4%, and centroid distance 0.021. In classifying the cannulation difficulty, it achieved the recall of 71.9% for the class of easy cases and that of 61.1% for that of difficult cases. Remarkably, our model accurately detected AOV with varying morphological shape, size, and texture on par with the level of a human expert and showed promising results for recognizing cannulation difficulty. It demonstrated its potential to improve the quality of ERCP by assisting endoscopists.

The Role and Impact of Deep Learning Methods in Computer-Aided Diagnosis Using Gastrointestinal Endoscopy

At present, the application of artificial intelligence (AI) based on deep learning in the medical field has become more extensive and suitable for clinical practice compared with traditional machine learning. The application of traditional machine learning approaches to clinical practice is very challenging because medical data are usually uncharacteristic. However, deep learning methods with self-learning abilities can effectively make use of excellent computing abilities to learn intricate and abstract features. Thus, they are promising for the classification and detection of lesions through gastrointestinal endoscopy using a computer-aided diagnosis (CAD) system based on deep learning. This study aimed to address the research development of a CAD system based on deep learning in order to assist doctors in classifying and detecting lesions in the stomach, intestines, and esophagus. It also summarized the limitations of the current methods and finally presented a prospect for future research.

State of the Art: The Impact of Artificial Intelligence in Endoscopy 2020

PURPOSE OF REVIEW

Recently numerous researchers have shown remarkable progress using convolutional neural network-based artificial intelligence (AI) for endoscopy. In this manuscript we aim to summarize recent AI impact on endoscopy.

RECENT FINDINGS

AI for detecting colon polyps has been the most promising area for application of AI in endoscopy. Recent prospective randomized studies showed that AI assisted colonoscopy increased adenoma detection rate and the mean number of adenomas per patient compared to standard colonoscopy alone. AI for optical biopsy of colon polyp showed a negative predictive value of ≥90%. For capsule endoscopy, applying AI to pre-read the video images decreased physician reading time significantly. Recently, researchers are broadening the area of AI to quality assessment of endoscopy such as bowel preparation and automated report generation. AI systems have shown great potential to increase physician performance by enhancing detection, reducing procedure time, and providing real-time feedback of endoscopy quality. To build a generally applicable AI, we need further investigations in real world settings and also integration of AI tools into pragmatic platforms.

Ability of artificial intelligence to detect T1 esophageal squamous cell carcinoma from endoscopic videos and the effects of real-time assistance

Diagnosis using artificial intelligence (AI) with deep learning could be useful in endoscopic examinations. We investigated the ability of AI to detect superficial esophageal squamous cell carcinoma (ESCC) from esophagogastroduodenoscopy (EGD) videos. We retrospectively collected 8428 EGD images of esophageal cancer to develop a convolutional neural network through deep learning. We evaluated the detection accuracy of the AI diagnosing system compared with that of 18 endoscopists. We used 144 EGD videos for the two validation sets. First, we used 64 EGD observation videos of ESCCs using both white light imaging (WLI) and narrow-band imaging (NBI). We then evaluated the system using 80 EGD videos from 40 patients (20 with superficial ESCC and 20 with non-ESCC). In the first set, the AI system correctly diagnosed 100% ESCCs. In the second set, it correctly detected 85% (17/20) ESCCs. Of these, 75% (15/20) and 55% (11/22) were detected by WLI and NBI, respectively, and the positive predictive value was 36.7%. The endoscopists correctly detected 45% (25–70%) ESCCs. With AI real-time assistance, the sensitivities of the endoscopists were significantly improved without AI assistance (p < 0.05). AI can detect superficial ESCCs from EGD videos with high sensitivity and the sensitivity of the endoscopist was improved with AI real-time support.

Application of artificial intelligence using a convolutional neural network for detecting cholesteatoma in endoscopic enhanced images

OBJECTIVE

We examined whether artificial intelligence (AI) used with the novel digital image enhancement system modalities (CLARA+CHROMA, SPECTRA A, and SPECTRA B) could distinguish the cholesteatoma matrix, cholesteatoma debris, and normal middle ear mucosa, and observe the middle ear cavity during middle ear cholesteatoma surgery.

METHODS

A convolutional neural network (CNN) was trained with a set of images chosen by an otologist. To evaluate the diagnostic accuracy of the constructed CNN, an independent test data set of middle ear images was collected from 14 consecutive patients with 26 cholesteatoma matrix lesions, who underwent transcanal endoscopic ear surgery at a single hospital from August 2018 to September 2019. The final test data set included 58 total images, with 1‒5 images from each modality for each case.

RESULTS

The CNN required only 10 s to analyze more than 58 test images. Using SPECTRA A and SPECTRA B, the CNN correctly diagnosed 15 and 15 of 26 cholesteatoma matrix lesions, with a sensitivity of 34.6% and 42.3%, and with a specificity of 81.3% and 87.5%, respectively.

CONCLUSION

Our preliminary study revealed that AI and novel imaging modalities are potentially useful tools for identifying and visualizing the cholesteatoma matrix during endoscopic ear surgery. The diagnostic ability of the CNN is not yet appropriate for implementation in daily clinical practice, based on our study findings. However, in the future, these techniques and AI tools could help to reduce the burden on surgeons and will facilitate telemedicine in remote and rural areas, as well as in developing countries where the number of surgeons is limited.

Automated system for diagnosing endometrial cancer by adopting deep-learning technology in hysteroscopy

Endometrial cancer is a ubiquitous gynecological disease with increasing global incidence. Therefore, despite the lack of an established screening technique to date, early diagnosis of endometrial cancer assumes critical importance. This paper presents an artificial-intelligence-based system to detect the regions affected by endometrial cancer automatically from hysteroscopic images. In this study, 177 patients (60 with normal endometrium, 21 with uterine myoma, 60 with endometrial polyp, 15 with atypical endometrial hyperplasia, and 21 with endometrial cancer) with a history of hysteroscopy were recruited. Machine-learning techniques based on three popular deep neural network models were employed, and a continuity-analysis method was developed to enhance the accuracy of cancer diagnosis. Finally, we investigated if the accuracy could be improved by combining all the trained models. The results reveal that the diagnosis accuracy was approximately 80% (78.91–80.93%) when using the standard method, and it increased to 89% (83.94–89.13%) and exceeded 90% (i.e., 90.29%) when employing the proposed continuity analysis and combining the three neural networks, respectively. The corresponding sensitivity and specificity equaled 91.66% and 89.36%, respectively. These findings demonstrate the proposed method to be sufficient to facilitate timely diagnosis of endometrial cancer in the near future.

Development and Application of Artificial Intelligence in Auxiliary TCM Diagnosis

As an emerging comprehensive discipline, artificial intelligence (AI) has been widely applied in various fields, including traditional Chinese medicine (TCM), a treasure of the Chinese nation. Realizing the organic combination of AI and TCM can promote the inheritance and development of TCM. The paper summarizes the development and application of AI in auxiliary TCM diagnosis, analyzes the bottleneck of artificial intelligence in the field of auxiliary TCM diagnosis at present, and proposes a possible future direction of its development.

Detection of sitting posture using hierarchical image composition and deep learning

Human posture detection allows the capture of the kinematic parameters of the human body, which is important for many applications, such as assisted living, healthcare, physical exercising and rehabilitation. This task can greatly benefit from recent development in deep learning and computer vision. In this paper, we propose a novel deep recurrent hierarchical network (DRHN) model based on MobileNetV2 that allows for greater flexibility by reducing or eliminating posture detection problems related to a limited visibility human torso in the frame, i.e., the occlusion problem. The DRHN network accepts the RGB-Depth frame sequences and produces a representation of semantically related posture states. We achieved 91.47% accuracy at 10 fps rate for sitting posture recognition.

Artificial intelligence (AI) in medicine, current applications and future role with special emphasis on its potential and promise in pathology: present and future impact, obstacles including costs and acceptance among pathologists, practical and philosophical considerations. A comprehensive review

BACKGROUND

The role of Artificial intelligence (AI) which is defined as the ability of computers to perform tasks that normally require human intelligence is constantly expanding. Medicine was slow to embrace AI. However, the role of AI in medicine is rapidly expanding and promises to revolutionize patient care in the coming years. In addition, it has the ability to democratize high level medical care and make it accessible to all parts of the world.

MAIN TEXT

Among specialties of medicine, some like radiology were relatively quick to adopt AI whereas others especially pathology (and surgical pathology in particular) are only just beginning to utilize AI. AI promises to play a major role in accurate diagnosis, prognosis and treatment of cancers. In this paper, the general principles of AI are defined first followed by a detailed discussion of its current role in medicine. In the second half of this comprehensive review, the current and future role of AI in surgical pathology is discussed in detail including an account of the practical difficulties involved and the fear of pathologists of being replaced by computer algorithms. A number of recent studies which demonstrate the usefulness of AI in the practice of surgical pathology are highlighted.

CONCLUSION

AI has the potential to transform the practice of surgical pathology by ensuring rapid and accurate results and enabling pathologists to focus on higher level diagnostic and consultative tasks such as integrating molecular, morphologic and clinical information to make accurate diagnosis in difficult cases, determine prognosis objectively and in this way contribute to personalized care.

Current Evidence and Future Perspective of Accuracy of Artificial Intelligence Application for Early Gastric Cancer Diagnosis With Endoscopy: A Systematic and Meta-Analysis

Background & Aims: Gastric cancer is the common malignancies from cancer worldwide. Endoscopy is currently the most effective method to detect early gastric cancer (EGC). However, endoscopy is not infallible and EGC can be missed during endoscopy. Artificial intelligence (AI)-assisted endoscopic diagnosis is a recent hot spot of research. We aimed to quantify the diagnostic value of AI-assisted endoscopy in diagnosing EGC.

Method: The PubMed, MEDLINE, Embase and the Cochrane Library Databases were searched for articles on AI-assisted endoscopy application in EGC diagnosis. The pooled sensitivity, specificity, and area under the curve (AUC) were calculated, and the endoscopists' diagnostic value was evaluated for comparison. The subgroup was set according to endoscopy modality, and number of training images. A funnel plot was delineated to estimate the publication bias.

Result: 16 studies were included in this study. We indicated that the application of AI in endoscopic detection of EGC achieved an AUC of 0.96 (95% CI, 0.94–0.97), a sensitivity of 86% (95% CI, 77–92%), and a specificity of 93% (95% CI, 89–96%). In AI-assisted EGC depth diagnosis, the AUC was 0.82(95% CI, 0.78–0.85), and the pooled sensitivity and specificity was 0.72(95% CI, 0.58–0.82) and 0.79(95% CI, 0.56–0.92). The funnel plot showed no publication bias.

Conclusion: The AI applications for EGC diagnosis seemed to be more accurate than the endoscopists. AI assisted EGC diagnosis was more accurate than experts. More prospective studies are needed to make AI-aided EGC diagnosis universal in clinical practice.

Artificial Convolutional Neural Network in Object Detection and Semantic Segmentation for Medical Imaging Analysis

In the era of digital medicine, a vast number of medical images are produced every day. There is a great demand for intelligent equipment for adjuvant diagnosis to assist medical doctors with different disciplines. With the development of artificial intelligence, the algorithms of convolutional neural network (CNN) progressed rapidly. CNN and its extension algorithms play important roles on medical imaging classification, object detection, and semantic segmentation. While medical imaging classification has been widely reported, the object detection and semantic segmentation of imaging are rarely described. In this review article, we introduce the progression of object detection and semantic segmentation in medical imaging study. We also discuss how to accurately define the location and boundary of diseases.

Designing deep learning studies in cancer diagnostics

The number of publications on deep learning for cancer diagnostics is rapidly increasing, and systems are frequently claimed to perform comparable with or better than clinicians. However, few systems have yet demonstrated real-world medical utility. In this Perspective, we discuss reasons for the moderate progress and describe remedies designed to facilitate transition to the clinic. Recent, presumably influential, deep learning studies in cancer diagnostics, of which the vast majority used images as input to the system, are evaluated to reveal the status of the field. By manipulating real data, we then exemplify that much and varied training data facilitate the generalizability of neural networks and thus the ability to use them clinically. To reduce the risk of biased performance estimation of deep learning systems, we advocate evaluation in external cohorts and strongly advise that the planned analyses, including a predefined primary analysis, are described in a protocol preferentially stored in an online repository. Recommended protocol items should be established for the field, and we present our suggestions.

Artificial intelligence (AI) in medicine as a strategic valuable tool

Humans' creativity led to machines that outperform human capabilities in terms of workload, effectiveness, precision, endurance, strength, and repetitiveness. It has always been a vision and a way to transcend the existence and to give more sense to life, which is precious. The common denominator of all these creations was that they were meant to replace, enhance or go beyond the mechanical capabilities of the human body. The story takes another bifurcation when Alan Turing introduced the concept of a machine that could think, in 1950. Artificial intelligence, presented as a term in 1956, describes the use of computers to imitate intelligence and critical thinking comparable to humans. However, the revolution began in 1943, when artificial neural networks was an attempt to exploit the architecture of the human brain to perform tasks that conventional algorithms had little success with. Artificial intelligence is becoming a research focus and a tool of strategic value. The same observations apply in the field of healthcare, too. In this manuscript, we try to address key questions regarding artificial intelligence in medicine, such as what artificial intelligence is and how it works, what is its value in terms of application in medicine, and what are the prospects?

Overcoming barriers to implementation of artificial intelligence in gastroenterology

Artificial intelligence is poised to revolutionize the field of medicine, however significant questions must be answered prior to its implementation on a regular basis. Many artificial intelligence algorithms remain limited by isolated datasets which may cause selection bias and truncated learning for the program. While a central database may solve this issue, several barriers such as security, patient consent, and management structure prevent this from being implemented. An additional barrier to daily use is device approval by the Food and Drug Administration. In order for this to occur, clinical studies must address new endpoints, including and beyond the traditional bio- and medical statistics. These must showcase artificial intelligence's benefit and answer key questions, including challenges posed in the field of medical ethics.

Wavelet Transform and Deep Convolutional Neural Network-Based Smart Healthcare System for Gastrointestinal Disease Detection

This work presents a smart healthcare system for the detection of various abnormalities present in the gastrointestinal (GI) region with the help of time-frequency analysis and convolutional neural network. In this regard, the KVASIR V2 dataset comprising of eight classes of GI-tract images such as Normal cecum, Normal pylorus, Normal Z-line, Esophagitis, Polyps, Ulcerative Colitis, Dyed and lifted polyp, and Dyed resection margins are used for training and validation. The initial phase of the work involves an image pre-processing step, followed by the extraction of approximate discrete wavelet transform coefficients. Each class of decomposed images is later given as input to a couple of considered convolutional neural network (CNN) models for training and testing in two different classification levels to recognize its predicted value. Afterward, the classification performance is measured through the following measuring indices: accuracy, precision, recall, specificity, and F1 score. The experimental result shows 97.25% and 93.75% of accuracy in the first level and second level of classification, respectively. Lastly, a comparative performance analysis is carried out with several other previously published works on a similar dataset where the proposed approach performs better than its contemporary methods.