|
2
|
Gong L, Wang M, Shu L, He J, Qin B, Xu J, Su W, Dong D, Hu H, Tian J, Zhou P. Automatic captioning of early gastric cancer using magnification endoscopy with narrow-band imaging. Gastrointest Endosc 2022; 96:929-942.e6. [PMID: 35917877 DOI: 10.1016/j.gie.2022.07.019] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 07/03/2022] [Accepted: 07/13/2022] [Indexed: 02/08/2023]
Abstract
BACKGROUND AND AIMS The detection rate for early gastric cancer (EGC) is unsatisfactory, and mastering the diagnostic skills of magnifying endoscopy with narrow-band imaging (ME-NBI) requires rich expertise and experience. We aimed to develop an EGC captioning model (EGCCap) to automatically describe the visual characteristics of ME-NBI images for endoscopists. METHODS ME-NBI images (n = 1886) from 294 cases were enrolled from multiple centers, and corresponding 5658 text data were designed following the simple EGC diagnostic algorithm. An EGCCap was developed using the multiscale meshed-memory transformer. We conducted comprehensive evaluations for EGCCap including the quantitative and quality of performance, generalization, robustness, interpretability, and assistant value analyses. The commonly used metrics were BLEUs, CIDEr, METEOR, ROUGE, SPICE, accuracy, sensitivity, and specificity. Two-sided statistical tests were conducted, and statistical significance was determined when P < .05. RESULTS EGCCap acquired satisfying captioning performance by outputting correctly and coherently clinically meaningful sentences in the internal test cohort (BLEU1 = 52.434, CIDEr = 36.734, METEOR = 27.823, ROUGE = 49.949, SPICE = 35.548) and maintained over 80% performance when applied to other centers or corrupted data. The diagnostic ability of endoscopists improved with the assistance of EGCCap, which was especially significant (P < .05) for junior endoscopists. Endoscopists gave EGCCap an average remarkable score of 7.182, showing acceptance of EGCCap. CONCLUSIONS EGCCap exhibited promising captioning performance and was proven with satisfying generalization, robustness, and interpretability. Our study showed potential value in aiding and improving the diagnosis of EGC and facilitating the development of automated reporting in the future.
Collapse
Affiliation(s)
- Lixin Gong
- College of Medicine and Biological Information Engineering School, Northeastern University, Shenyang, China; CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Min Wang
- Department of Gastroenterology, Hepatology and Nutrition, Shanghai Children's Hospital, Shanghai Jiaotong University, Shanghai, China
| | - Lei Shu
- Department of Gastroenterology, No. 1 Hospital of Wuhan, Wuhan, China
| | - Jie He
- Endoscopy Center, Zhongshan Hospital (Xiamen Branch), Fudan University, Xiamen, China; Department of Gastroenterology, The Affiliated Dongnan Hospital of Xiamen University, Zhangzhou, China
| | - Bin Qin
- Department of Gastroenterology, the Second Affiliated Hospital of Xi'an Jiaotong University, Zhangzhou, China
| | - Jiacheng Xu
- Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China
| | - Wei Su
- Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China
| | - Di Dong
- CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Hao Hu
- Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China; Department of Gastroenterology, Shigatse People's Hospital, Shigatse, China
| | - Jie Tian
- College of Medicine and Biological Information Engineering School, Northeastern University, Shenyang, China; CAS Key Laboratory of Molecular Imaging, Beijing Key Laboratory of Molecular Imaging, The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China; Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine and Engineering, Beihang University, Beijing, China
| | - Pinghong Zhou
- Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China
| |
Collapse
|