Ziegler J, Dobsch P, Rozema M, Zuber-Jerger I, Weigand K, Reuther S, Müller M, Kandulski A. Multimodal convolutional neural network-based algorithm for real-time detection and differentiation of malignant and inflammatory biliary strictures in cholangioscopy: a proof-of-concept study (with video).
Gastrointest Endosc 2025;
101:830-842.e2. [PMID:
39265745 DOI:
10.1016/j.gie.2024.09.001]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 08/12/2024] [Accepted: 09/01/2024] [Indexed: 09/14/2024]
Abstract
BACKGROUND AND AIMS
Deep learning algorithms gained attention for detection (computer-aided detection [CADe]) of biliary tract cancer in digital single-operator cholangioscopy (dSOC). We developed a multimodal convolutional neural network (CNN) for detection (CADe), characterization and discriminating (computer-aided diagnosis [CADx]) between malignant, inflammatory, and normal biliary tissue in raw dSOC videos. In addition, clinical metadata were included in the CNN algorithm to overcome limitations of image-only models.
METHODS
Based on dSOC videos and images of 111 patients (total of 15,158 still frames), a real-time CNN-based algorithm for CADe and CADx was developed and validated. We established an image-only model and metadata injection approach. In addition, frame-wise and case-based predictions on complete dSOC video sequences were validated. Model embeddings were visualized, and class activation maps highlighted relevant image regions.
RESULTS
The concatenation-based CADx approach achieved a per-frame area under the receiver-operating characteristic curve of .871, sensitivity of .809 (95% CI, .784-.832), specificity of .773 (95% CI, .761-.785), positive predictive value of .450 (95% CI, .423-.467), and negative predictive value of .946 (95% CI, .940-.954) with respect to malignancy on 5715 test frames from complete videos of 20 patients. For case-based diagnosis using average prediction scores, 6 of 8 malignant cases and all 12 benign cases were identified correctly.
CONCLUSIONS
Our algorithm distinguishes malignant and inflammatory bile duct lesions in dSOC videos, indicating the potential of CNN-based diagnostic support systems for both CADe and CADx. The integration of non-image data can improve CNN-based support systems, targeting current challenges in the assessment of biliary strictures.
Collapse