Convolutional neural network-based object detection model to identify gastrointestinal stromal tumors in endoscopic ultrasound images

New vision of HookEfficientNet deep neural network: Intelligent histopathological recognition system of non-small cell lung cancer

BACKGROUND

Efficient and precise diagnosis of non-small cell lung cancer (NSCLC) is quite critical for subsequent targeted therapy and immunotherapy. Since the advent of whole slide images (WSIs), the transition from traditional histopathology to digital pathology has aroused the application of convolutional neural networks (CNNs) in histopathological recognition and diagnosis. HookNet can make full use of macroscopic and microscopic information for pathological diagnosis, but it cannot integrate other excellent CNN structures. The new version of HookEfficientNet is based on a combination of HookNet structure and EfficientNet that performs well in the recognition of general objects. Here, a high-precision artificial intelligence-guided histopathological recognition system was established by HookEfficientNet to provide a basis for the intelligent differential diagnosis of NSCLC.

METHODS

A total of 216 WSIs of lung adenocarcinoma (LUAD) and 192 WSIs of lung squamous cell carcinoma (LUSC) were recruited from the First Affiliated Hospital of Zhengzhou University. Deep learning methods based on HookEfficientNet, HookNet and EfficientNet B4-B6 were developed and compared with each other using area under the curve (AUC) and the Youden index. Temperature scaling was used to calibrate the heatmap and highlight the cancer region of interest. Four pathologists of different levels blindly reviewed 108 WSIs of LUAD and LUSC, and the diagnostic results were compared with the various deep learning models.

RESULTS

The HookEfficientNet model outperformed HookNet and EfficientNet B4-B6. After temperature scaling, the HookEfficientNet model achieved AUCs of 0.973, 0.980, and 0.989 and Youden index values of 0.863, 0.899, and 0.922 for LUAD, LUSC and normal lung tissue, respectively, in the testing set. The accuracy of the model was better than the average accuracy from experienced pathologists, and the model was superior to pathologists in the diagnosis of LUSC.

CONCLUSIONS

HookEfficientNet can effectively recognize LUAD and LUSC with performance superior to that of senior pathologists, especially for LUSC. The model has great potential to facilitate the application of deep learning-assisted histopathological diagnosis for LUAD and LUSC in the future.

Efficacy of real-time artificial intelligence-aid endoscopic ultrasonography diagnostic system in discriminating gastrointestinal stromal tumors and leiomyomas: a multicenter diagnostic study

Background

Gastrointestinal stromal tumors (GISTs) represent the most prevalent type of subepithelial lesions (SELs) with malignant potential. Current imaging tools struggle to differentiate GISTs from leiomyomas. This study aimed to create and assess a real-time artificial intelligence (AI) system using endoscopic ultrasonography (EUS) images to differentiate between GISTs and leiomyomas.

Methods

The AI system underwent development and evaluation using EUS images from 5 endoscopic centers in China between January 2020 and August 2023. EUS images of 1101 participants with SELs were retrospectively collected for AI system development. A cohort of 241 participants with SELs was recruited for external AI system evaluation. Another cohort of 59 participants with SELs was prospectively enrolled to assess the real-time clinical application of the AI system. The AI system's performance was compared to that of endoscopists. This study is registered with Chictr.org.cn, Number ChiCT2000035787.

Findings

The AI system displayed an area under the curve (AUC) of 0.948 (95% CI: 0.921-0.969) for discriminating GISTs and leiomyomas. The AI system's accuracy (ACC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) reached 91.7% (95% CI 87.5%-94.6%), 90.3% (95% CI 83.4%-94.5%), 93.0% (95% CI 87.2%-96.3%), 91.9% (95% CI 85.3%-95.7%), and 91.5% (95% CI 85.5%-95.2%), respectively. Moreover, the AI system exhibited excellent performance in diagnosing ≤20 mm SELs (ACC 93.5%, 95% CI 0.900-0.969). In a prospective real-time clinical application trial, the AI system achieved an AUC of 0.865 (95% CI 0.764-0.966) and 0.864 (95% CI 0.762-0.966) for GISTs and leiomyomas diagnosis, respectively, markedly surpassing endoscopists [AUC 0.698 (95% CI 0.562-0.834) for GISTs and AUC 0.695 (95% CI 0.546-0.825) for leiomyomas].

Interpretation

We successfully developed a real-time AI-assisted EUS diagnostic system. The incorporation of the real-time AI system during EUS examinations can assist endoscopists in rapidly and accurately differentiating various types of SELs in clinical practice, facilitating improved diagnostic and therapeutic decision-making.

Funding

Science and Technology Commission Foundation of Shanghai Municipality, Science and Technology Commission Foundation of the Xuhui District, the Interdisciplinary Program of Shanghai Jiao Tong University and the Research Funds of Shanghai Sixth people's Hospital.

Deep learning model for differentiating nasal cavity masses based on nasal endoscopy images

BACKGROUND

Nasal polyps and inverted papillomas often look similar. Clinically, it is difficult to distinguish the masses by endoscopic examination. Therefore, in this study, we aimed to develop a deep learning algorithm for computer-aided diagnosis of nasal endoscopic images, which may provide a more accurate clinical diagnosis before pathologic confirmation of the nasal masses.

METHODS

By performing deep learning of nasal endoscope images, we evaluated our computer-aided diagnosis system's assessment ability for nasal polyps and inverted papilloma and the feasibility of their clinical application. We used curriculum learning pre-trained with patches of nasal endoscopic images and full-sized images. The proposed model's performance for classifying nasal polyps, inverted papilloma, and normal tissue was analyzed using five-fold cross-validation.

RESULTS

The normal scores for our best-performing network were 0.9520 for recall, 0.7900 for precision, 0.8648 for F1-score, 0.97 for the area under the curve, and 0.8273 for accuracy. For nasal polyps, the best performance was 0.8162, 0.8496, 0.8409, 0.89, and 0.8273, respectively, for recall, precision, F1-score, area under the curve, and accuracy. Finally, for inverted papilloma, the best performance was obtained for recall, precision, F1-score, area under the curve, and accuracy values of 0.5172, 0.8125, 0.6122, 0.83, and 0.8273, respectively.

CONCLUSION

Although there were some misclassifications, the results of gradient-weighted class activation mapping were generally consistent with the areas under the curve determined by otolaryngologists. These results suggest that the convolutional neural network is highly reliable in resolving lesion locations in nasal endoscopic images.

Deep learning-based computer-aided diagnosis system for the automatic detection and classification of lateral cervical lymph nodes on original ultrasound images of papillary thyroid carcinoma: a prospective diagnostic study

PURPOSE

This study aims to develop a deep learning-based computer-aided diagnosis (CAD) system for the automatic detection and classification of lateral cervical lymph nodes (LNs) on original ultrasound images of papillary thyroid carcinoma (PTC) patients.

METHODS

A retrospective data set of 1801 cervical LN ultrasound images from 1675 patients with PTC and a prospective test set including 185 images from 160 patients were collected. Four different deep leaning models were trained and validated in the retrospective data set. The best model was selected for CAD system development and compared with three sonographers in the retrospective and prospective test sets.

RESULTS

The Deformable Detection Transformer (DETR) model showed the highest diagnostic efficacy, with a mean average precision score of 86.3% in the retrospective test set, and was therefore used in constructing the CAD system. The detection performance of the CAD system was superior to the junior sonographer and intermediate sonographer with accuracies of 86.3% and 92.4% in the retrospective and prospective test sets, respectively. The classification performance of the CAD system was better than all sonographers with the areas under the curve (AUCs) of 94.4% and 95.2% in the retrospective and prospective test sets, respectively.

CONCLUSIONS

This study developed a Deformable DETR model-based CAD system for automatically detecting and classifying lateral cervical LNs on original ultrasound images, which showed excellent diagnostic efficacy and clinical utility. It can be an important tool for assisting sonographers in the diagnosis process.

Current status of artificial intelligence analysis for the treatment of pancreaticobiliary diseases using endoscopic ultrasonography and endoscopic retrograde cholangiopancreatography

Pancreatic and biliary diseases encompass a range of conditions requiring accurate diagnosis for appropriate treatment strategies. This diagnosis relies heavily on imaging techniques like endoscopic ultrasonography and endoscopic retrograde cholangiopancreatography. Artificial intelligence (AI), including machine learning and deep learning, is becoming integral in medical imaging and diagnostics, such as the detection of colorectal polyps. AI shows great potential in diagnosing pancreatobiliary diseases. Unlike machine learning, which requires feature extraction and selection, deep learning can utilize images directly as input. Accurate evaluation of AI performance is a complex task due to varied terminologies, evaluation methods, and development stages. Essential aspects of AI evaluation involve defining the AI's purpose, choosing appropriate gold standards, deciding on the validation phase, and selecting reliable validation methods. AI, particularly deep learning, is increasingly employed in endoscopic ultrasonography and endoscopic retrograde cholangiopancreatography diagnostics, achieving high accuracy levels in detecting and classifying various pancreatobiliary diseases. The AI often performs better than doctors, even in tasks like differentiating benign from malignant pancreatic tumors, cysts, and subepithelial lesions, identifying gallbladder lesions, assessing endoscopic retrograde cholangiopancreatography difficulty, and evaluating the biliary strictures. The potential for AI in diagnosing pancreatobiliary diseases, especially where other modalities have limitations, is considerable. However, a crucial constraint is the need for extensive, high-quality annotated data for AI training. Future advances in AI, such as large language models, promise further applications in the medical field.

Convolutional neural network applied to preoperative venous-phase CT images predicts risk category in patients with gastric gastrointestinal stromal tumors

OBJECTIVE

The risk category of gastric gastrointestinal stromal tumors (GISTs) are closely related to the surgical method, the scope of resection, and the need for preoperative chemotherapy. We aimed to develop and validate convolutional neural network (CNN) models based on preoperative venous-phase CT images to predict the risk category of gastric GISTs.

METHOD

A total of 425 patients pathologically diagnosed with gastric GISTs at the authors' medical centers between January 2012 and July 2021 were split into a training set (154, 84, and 59 with very low/low, intermediate, and high-risk, respectively) and a validation set (67, 35, and 26, respectively). Three CNN models were constructed by obtaining the upper and lower 1, 4, and 7 layers of the maximum tumour mask slice based on venous-phase CT Images and models of CNN_layer3, CNN_layer9, and CNN_layer15 established, respectively. The area under the receiver operating characteristics curve (AUROC) and the Obuchowski index were calculated to compare the diagnostic performance of the CNN models.

RESULTS

In the validation set, CNN_layer3, CNN_layer9, and CNN_layer15 had AUROCs of 0.89, 0.90, and 0.90, respectively, for low-risk gastric GISTs; 0.82, 0.83, and 0.83 for intermediate-risk gastric GISTs; and 0.86, 0.86, and 0.85 for high-risk gastric GISTs. In the validation dataset, CNN_layer3 (Obuchowski index, 0.871) provided similar performance than CNN_layer9 and CNN_layer15 (Obuchowski index, 0.875 and 0.873, respectively) in prediction of the gastric GIST risk category (All P >.05).

CONCLUSIONS

The CNN based on preoperative venous-phase CT images showed good performance for predicting the risk category of gastric GISTs.

Endoscopic ultrasound-based application system for predicting endoscopic resection-related outcomes and diagnosing subepithelial lesions: Multicenter prospective study

OBJECTIVES

Subepithelial lesions (SELs) are associated with various endoscopic resection (ER) outcomes and diagnostic challenges. We aimed to establish a tool for predicting ER-related outcomes and diagnosing SELs and to investigate the predictive value of endoscopic ultrasound (EUS).

METHODS

Phase 1 (system development) was performed in a retrospective cohort (n = 837) who underwent EUS before ER for SELs at eight hospitals. Prediction models for five key outcomes were developed using logistic regression. Models with satisfactory internal validation performance were included in a mobile application system, SEL endoscopic resection predictor (SELERP). In Phase 2, the models were externally validated in a prospective cohort of 200 patients.

RESULTS

An SELERP was developed using EUS characteristics, which included 10 models for five key outcomes: post-ER ulcer management, short procedure time, long hospital stay, high medication costs, and diagnosis of SELs. In Phase 1, 10 models were derived and validated (C-statistics, 0.67-0.99; calibration-in-the-large, -0.14-0.10; calibration slopes, 0.92-1.08). In Phase 2, the derived risk prediction models showed convincing discrimination (C-statistics, 0.64-0.73) and calibration (calibration-in-the-large, -0.02-0.05; calibration slopes, 1.01-1.09) in the prospective cohort. The sensitivities and specificities of the five diagnostic models were 68.3-95.7% and 64.1-83.3%, respectively.

CONCLUSION

We developed and prospectively validated an application system for the prediction of ER outcomes and diagnosis of SELs, which could aid clinical decision-making and facilitate patient-physician consultation. EUS features significantly contributed to the prediction.

TRIAL REGISTRATION

Chinese Clinical Trial Registry, http://www.chictr.org.cn (ChiCTR2000040118).

Data pyramid structure for optimizing EUS-based GISTs diagnosis in multi-center analysis with missing label

This study introduces the Data Pyramid Structure (DPS) to address data sparsity and missing labels in medical image analysis. The DPS optimizes multi-task learning and enables sustainable expansion of multi-center data analysis. Specifically, It facilitates attribute prediction and malignant tumor diagnosis tasks by implementing a segmentation and aggregation strategy on data with absent attribute labels. To leverage multi-center data, we propose the Unified Ensemble Learning Framework (UELF) and the Unified Federated Learning Framework (UFLF), which incorporate strategies for data transfer and incremental learning in scenarios with missing labels. The proposed method was evaluated on a challenging EUS patient dataset from five centers, achieving promising diagnostic performance. The average accuracy was 0.984 with an AUC of 0.927 for multi-center analysis, surpassing state-of-the-art approaches. The interpretability of the predictions further highlights the potential clinical relevance of our method.

Artificial Intelligence in the Prediction of Gastrointestinal Stromal Tumors on Endoscopic Ultrasonography Images: Development, Validation and Comparison with Endosonographers

Background/Aims

The accuracy of endosonographers in diagnosing gastric subepithelial lesions (SELs) using endoscopic ultrasonography (EUS) is influenced by experience and subjectivity. Artificial intelligence (AI) has achieved remarkable development in this field. This study aimed to develop an AI-based EUS diagnostic model for the diagnosis of SELs, and evaluated its efficacy with external validation.

Methods

We developed the EUS-AI model with ResNeSt50 using EUS images from two hospitals to predict the histopathology of the gastric SELs originating from muscularis propria. The diagnostic performance of the model was also validated using EUS images obtained from four other hospitals.

Results

A total of 2,057 images from 367 patients (375 SELs) were chosen to build the models, and 914 images from 106 patients (108 SELs) were chosen for external validation. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the model for differentiating gastrointestinal stromal tumors (GISTs) and non-GISTs in the external validation sets by images were 82.01%, 68.22%, 86.77%, 59.86%, and 78.12%, respectively. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy in the external validation set by tumors were 83.75%, 71.43%, 89.33%, 60.61%, and 80.56%, respectively. The EUS-AI model showed better performance (especially specificity) than some endosonographers. The model helped improve the sensitivity, specificity, and accuracy of certain endosonographers.

Conclusions

We developed an EUS-AI model to classify gastric SELs originating from muscularis propria into GISTs and non-GISTs with good accuracy. The model may help improve the diagnostic performance of endosonographers. Further work is required to develop a multi-modal EUS-AI system.

Artificial intelligence in endoscopy: Overview, applications, and future directions

Since the emergence of artificial intelligence (AI) in medicine, endoscopy applications in gastroenterology have been at the forefront of innovations. The ever-increasing number of studies necessitates the need to organize and classify applications in a useful way. Separating AI capabilities by computer aided detection (CADe), diagnosis (CADx), and quality assessment (CADq) allows for a systematic evaluation of each application. CADe studies have shown promise in accurate detection of esophageal, gastric and colonic neoplasia as well as identifying sources of bleeding and Crohn's disease in the small bowel. While more advanced CADx applications employ optical biopsies to give further information to characterize neoplasia and grade inflammatory disease, diverse CADq applications ensure quality and increase the efficiency of procedures. Future applications show promise in advanced therapeutic modalities and integrated systems that provide multimodal capabilities. AI is set to revolutionize clinical decision making and performance of endoscopy.

Endoscopic ultrasound artificial intelligence-assisted for prediction of gastrointestinal stromal tumors diagnosis: A systematic review and meta-analysis

BACKGROUND

Subepithelial lesions (SELs) are gastrointestinal tumors with heterogeneous malignant potential. Endoscopic ultrasonography (EUS) is the leading method for evaluation, but without histopathological analysis, precise differentiation of SEL risk is limited. Artificial intelligence (AI) is a promising aid for the diagnosis of gastrointestinal lesions in the absence of histopathology.

AIM

To determine the diagnostic accuracy of AI-assisted EUS in diagnosing SELs, especially lesions originating from the muscularis propria layer.

METHODS

Electronic databases including PubMed, EMBASE, and Cochrane Library were searched. Patients of any sex and > 18 years, with SELs assessed by EUS AI-assisted, with previous histopathological diagnosis, and presented sufficient data values which were extracted to construct a 2 × 2 table. The reference standard was histopathology. The primary outcome was the accuracy of AI for gastrointestinal stromal tumor (GIST). Secondary outcomes were AI-assisted EUS diagnosis for GIST vs gastrointestinal leiomyoma (GIL), the diagnostic performance of experienced endoscopists for GIST, and GIST vs GIL. Pooled sensitivity, specificity, positive, and negative predictive values were calculated. The corresponding summary receiver operating characteristic curve and post-test probability were also analyzed.

RESULTS

Eight retrospective studies with a total of 2355 patients and 44154 images were included in this meta-analysis. The AI-assisted EUS for GIST diagnosis showed a sensitivity of 92% [95% confidence interval (CI): 0.89-0.95; P < 0.01), specificity of 80% (95%CI: 0.75-0.85; P < 0.01), and area under the curve (AUC) of 0.949. For diagnosis of GIST vs GIL by AI-assisted EUS, specificity was 90% (95%CI: 0.88-0.95; P = 0.02) and AUC of 0.966. The experienced endoscopists' values were sensitivity of 72% (95%CI: 0.67-0.76; P < 0.01), specificity of 70% (95%CI: 0.64-0.76; P < 0.01), and AUC of 0.777 for GIST. Evaluating GIST vs GIL, the experts achieved a sensitivity of 73% (95%CI: 0.65-0.80; P < 0.01) and an AUC of 0.819.

CONCLUSION

AI-assisted EUS has high diagnostic accuracy for fourth-layer SELs, especially for GIST, demonstrating superiority compared to experienced endoscopists' and improving their diagnostic performance in the absence of invasive procedures.

Artificial intelligence in endoscopic ultrasonography: risk stratification of gastric gastrointestinal stromal tumors

Background

Previous studies have identified useful endoscopic ultrasonography (EUS) features to predict the malignant potential of gastrointestinal stromal tumors (GISTs). However, the results of the studies were not consistent. Artificial intelligence (AI) has shown promising results in medicine.

Objectives

We aimed to build a risk stratification EUS-AI model to predict the malignancy potential of GISTs.

Design

This was a retrospective study with external validation.

Methods

We developed two models using EUS images from two hospitals to predict the GIST risk category. Model 1 was the four-category risk EUS-AI model, and Model 2 was the two-category risk EUS-AI model. The diagnostic performance of the models was validated with external cohorts.

Results

A total of 1320 images (880 were very low-risk, 269 were low-risk, 68 were intermediate-risk, and 103 were high-risk) were finally chosen for building the models and test sets, and a total of 656 images (211 were very low-risk, 266 were low-risk, 88 were intermediate-risk, and 91 were high-risk) were chosen for external validation. The overall accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the four-category risk EUS-AI model in the external validation sets by tumor were 74.50%, 55.00%, 79.05%, 53.49%, and 81.63%, respectively. The accuracy, sensitivity, specificity, PPV, and NPV for the two-category risk EUS-AI model for the prediction of very low-risk GISTs in the external validation sets by tumor were 86.25%, 94.44%, 79.55%, 79.07%, and 94.59%, respectively.

Conclusion

We developed a EUS-AI model for the risk stratification of GISTs with promising results, which may complement current clinical practice in the management of GISTs.

Registration

The study has been registered in the Chinese Clinical Trial Registry (No. ChiCTR2100051191).

A Comprehensive Guide to Artificial Intelligence in Endoscopic Ultrasound

BACKGROUND

Endoscopic Ultrasound (EUS) is widely used for the diagnosis of bilio-pancreatic and gastrointestinal (GI) tract diseases, for the evaluation of subepithelial lesions, and for sampling of lymph nodes and solid masses located next to the GI tract. The role of Artificial Intelligence in healthcare in growing. This review aimed to provide an overview of the current state of AI in EUS from imaging to pathological diagnosis and training.

METHODS

AI algorithms can assist in lesion detection and characterization in EUS by analyzing EUS images and identifying suspicious areas that may require further clinical evaluation or biopsy sampling. Deep learning techniques, such as convolutional neural networks (CNNs), have shown great potential for tumor identification and subepithelial lesion (SEL) evaluation by extracting important features from EUS images and using them to classify or segment the images.

RESULTS

AI models with new features can increase the accuracy of diagnoses, provide faster diagnoses, identify subtle differences in disease presentation that may be missed by human eyes, and provide more information and insights into disease pathology.

CONCLUSIONS

The integration of AI in EUS images and biopsies has the potential to improve the diagnostic accuracy, leading to better patient outcomes and to a reduction in repeated procedures in case of non-diagnostic biopsies.

Artificial intelligence-assisted endoscopic ultrasound in the diagnosis of gastrointestinal stromal tumors: a meta-analysis

BACKGROUND AND AIMS

Endoscopic ultrasonography (EUS) is useful for the diagnosis of gastrointestinal stromal tumors (GISTs), but is limited by subjective interpretation. Studies on artificial intelligence (AI)-assisted diagnosis are under development. Here, we used a meta-analysis to evaluate the diagnostic performance of AI in the diagnosis of GISTs using EUS images.

METHODS

PubMed, Ovid Medline, Embase, Web of science, and the Cochrane Library databases were searched for studies based on the EUS using AI for the diagnosis of GISTs, and a meta-analysis was performed to examine the accuracy.

RESULTS

Overall, 7 studies were included in our meta-analysis. A total of 2431 patients containing more than 36,186 images were used as the overall dataset, of which 480 patients were used for the final testing. The pooled sensitivity, specificity, positive, and negative likelihood ratio (LR) of AI-assisted EUS for differentiating GISTs from other submucosal tumors (SMTs) were 0.92 (95% confidence interval [CI] 0.89-0.95), 0.82 (95% CI 0.75-0.87), 4.55 (95% CI 2.64-7.84), and 0.12 (95% CI 0.07-0.20), respectively. The summary diagnostic odds ratio (DOR) and the area under the curve were 64.70 (95% CI 23.83-175.69) and 0.950 (Q* = 0.891).

CONCLUSIONS

AI-assisted EUS showed high accuracy for the automatic endoscopic diagnosis of GISTs, which could be used as a valuable complementary method for the differentiation of SMTs in the future.

Endoscopic Ultrasound Advanced Techniques for Diagnosis of Gastrointestinal Stromal Tumours

Gastrointestinal Stromal Tumors (GISTs) are subepithelial lesions (SELs) that commonly develop in the gastrointestinal tract. GISTs, unlike other SELs, can exhibit malignant behavior, so differential diagnosis is critical to the decision-making process. Endoscopic ultrasound (EUS) is considered the most accurate imaging method for diagnosing and differentiating SELs in the gastrointestinal tract by assessing the lesions precisely and evaluating their malignant risk. Due to their overlapping imaging characteristics, endosonographers may have difficulty distinguishing GISTs from other SELs using conventional EUS alone, and the collection of tissue samples from these lesions may be technically challenging. Even though it appears to be less effective in the case of smaller lesions, histology is now the gold standard for achieving a final diagnosis and avoiding unnecessary and invasive treatment for benign SELs. The use of enhanced EUS modalities and elastography has improved the diagnostic ability of EUS. Furthermore, recent advancements in artificial intelligence systems that use EUS images have allowed them to distinguish GISTs from other SELs, thereby improving their diagnostic accuracy.

Query2: Query over queries for improving gastrointestinal stromal tumour detection in an endoscopic ultrasound

Gastrointestinal stromal tumour (GIST) lesions are mesenchymal neoplasms commonly found in the upper gastrointestinal tract, but non-invasive GIST detection during an endoscopy remains challenging because their ultrasonic images resemble several benign lesions. Techniques for automatic GIST detection and other lesions from endoscopic ultrasound (EUS) images offer great potential to advance the precision and automation of traditional endoscopy and treatment procedures. However, GIST recognition faces several intrinsic challenges, including the input restriction of a single image modality and the mismatch between tasks and models. To address these challenges, we propose a novel Query² (Query over Queries) framework to identify GISTs from ultrasound images. The proposed Query² framework applies an anatomical location embedding layer to break the single image modality. A cross-attention module is then applied to query the queries generated from the basic detection head. Moreover, a single-object restricted detection head is applied to infer the lesion categories. Meanwhile, to drive this network, we present GIST514-DB, a GIST dataset that will be made publicly available, which includes the ultrasound images, bounding boxes, categories and anatomical locations from 514 cases. Extensive experiments on the GIST514-DB demonstrate that the proposed Query² outperforms most of the state-of-the-art methods.

Deep learning and radiomics to predict the mitotic index of gastrointestinal stromal tumors based on multiparametric MRI

Introduction

Preoperative evaluation of the mitotic index (MI) of gastrointestinal stromal tumors (GISTs) represents the basis of individualized treatment of patients. However, the accuracy of conventional preoperative imaging methods is limited. The aim of this study was to develop a predictive model based on multiparametric MRI for preoperative MI prediction.

Methods

A total of 112 patients who were pathologically diagnosed with GIST were enrolled in this study. The dataset was subdivided into the development (n = 81) and test (n = 31) sets based on the time of diagnosis. With the use of T2-weighted imaging (T2WI) and apparent diffusion coefficient (ADC) map, a convolutional neural network (CNN)-based classifier was developed for MI prediction, which used a hybrid approach based on 2D tumor images and radiomics features from 3D tumor shape. The trained model was tested on an internal test set. Then, the hybrid model was comprehensively tested and compared with the conventional ResNet, shape radiomics classifier, and age plus diameter classifier.

Results

The hybrid model showed good MI prediction ability at the image level; the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and accuracy in the test set were 0.947 (95% confidence interval [CI]: 0.927-0.968), 0.964 (95% CI: 0.930-0.978), and 90.8 (95% CI: 88.0-93.0), respectively. With the average probabilities from multiple samples per patient, good performance was also achieved at the patient level, with AUROC, AUPRC, and accuracy of 0.930 (95% CI: 0.828-1.000), 0.941 (95% CI: 0.792-1.000), and 93.6% (95% CI: 79.3-98.2) in the test set, respectively.

Discussion

The deep learning-based hybrid model demonstrated the potential to be a good tool for the operative and non-invasive prediction of MI in GIST patients.

Efficacy of ultrasound endoscopy with artificial intelligence for the differential diagnosis of non-gastric gastrointestinal stromal tumors

Gastrointestinal stromal tumors (GISTs) are common subepithelial lesions (SELs) and require treatment considering their malignant potential. We recently developed an endoscopic ultrasound-based artificial intelligence (EUS-AI) system to differentiate GISTs from non-GISTs in gastric SELs, which were used to train the system. We assessed whether the EUS-AI system designed for diagnosing gastric GISTs could be applied to non-gastric GISTs. Between January 2015 and January 2021, 52 patients with non-gastric SELs (esophagus, n = 15; duodenum, n = 26; colon, n = 11) were enrolled. The ability of EUS-AI to differentiate GISTs from non-GISTs in non-gastric SELs was examined. The accuracy, sensitivity, and specificity of EUS-AI for discriminating GISTs from non-GISTs in non-gastric SELs were 94.4%, 100%, and 86.1%, respectively, with an area under the curve of 0.98 based on the cutoff value set using the Youden index. In the subanalysis, the accuracy, sensitivity, and specificity of EUS-AI were highest in the esophagus (100%, 100%, 100%; duodenum, 96.2%, 100%, 0%; colon, 90.9%, 100%, 0%); the cutoff values were determined using the Youden index or the value determined using stomach cases. The diagnostic accuracy of EUS-AI increased as lesion size increased, regardless of lesion location. EUS-AI based on gastric SELs had good diagnostic ability for non-gastric GISTs.

Diagnostic accuracy of endoscopic ultrasound with artificial intelligence for gastrointestinal stromal tumors: A meta-analysis

OBJECTIVES

Gastrointestinal stromal tumors (GISTs) are thought to have a malignant potential. However, utilizing endoscopic ultrasound (EUS) to differentiate GISTs from other types of subepithelial lesions (SELs) remains challenging. Artificial intelligence (AI)-based diagnostic systems for EUS have been reported to have a promising performance, although the results of the previous studies remain controversial. In this meta-analysis, we aimed to assess the diagnostic accuracy of AI-based EUS in distinguishing GISTs from other SELs.

METHODS

A literature search was conducted on MEDLINE and EMBASE databases to identify relevant articles. The sensitivity, specificity, and area under the summary receiver operating characteristic curve (AUROC) of eligible studies were analyzed.

RESULTS

Seven studies were eligible for the final analysis. The combined sensitivity and specificity of AI-based EUS were 0.93 (95% confidence interval [CI] 0.88-0.96) and 0.78 (95% CI 0.67-0.87), respectively. The overall diagnostic odds ratio of AI-based EUS for GISTs was 36.74 (95% CI 17.69-76.30) with an AUROC of 0.94.

CONCLUSIONS

AI-based EUS showed high diagnostic ability and might help better differentiate GISTs from other SELs. More prospective studies on the diagnosis of GISTs using AI-based EUS are warranted in clinical setting.

Application and prospect of artificial intelligence in digestive endoscopy

INTRODUCTION

With the progress of science and technology, artificial intelligence represented by deep learning has gradually begun to be applied in the medical field. Artificial intelligence has been applied to benign gastrointestinal lesions, tumors, early cancer, inflammatory bowel disease, gallbladder, pancreas, and other diseases. This review summarizes the latest research results on artificial intelligence in digestive endoscopy and discusses the prospect of artificial intelligence in digestive system diseases.

AREAS COVERED

We retrieved relevant documents on artificial intelligence in digestive tract diseases from PubMed and Medline. This review elaborates on the knowledge of computer-aided diagnosis in digestive endoscopy.

EXPERT OPINION

Artificial intelligence significantly improves diagnostic accuracy, reduces physicians' workload, and provides a shred of evidence for clinical diagnosis and treatment. Shortly, artificial intelligence will have high application value in the field of medicine.