1
|
Meinikheim M, Mendel R, Palm C, Probst A, Muzalyova A, Scheppach MW, Nagl S, Schnoy E, Römmele C, Schulz DAH, Schlottmann J, Prinz F, Rauber D, Rückert T, Matsumura T, Fernández-Esparrach G, Parsa N, Byrne MF, Messmann H, Ebigbo A. Influence of artificial intelligence on the diagnostic performance of endoscopists in the assessment of Barrett's esophagus: a tandem randomized and video trial. Endoscopy 2024. [PMID: 38547927 DOI: 10.1055/a-2296-5696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/04/2024]
Abstract
BACKGROUND This study evaluated the effect of an artificial intelligence (AI)-based clinical decision support system on the performance and diagnostic confidence of endoscopists in their assessment of Barrett's esophagus (BE). METHODS 96 standardized endoscopy videos were assessed by 22 endoscopists with varying degrees of BE experience from 12 centers. Assessment was randomized into two video sets: group A (review first without AI and second with AI) and group B (review first with AI and second without AI). Endoscopists were required to evaluate each video for the presence of Barrett's esophagus-related neoplasia (BERN) and then decide on a spot for a targeted biopsy. After the second assessment, they were allowed to change their clinical decision and confidence level. RESULTS AI had a stand-alone sensitivity, specificity, and accuracy of 92.2%, 68.9%, and 81.3%, respectively. Without AI, BE experts had an overall sensitivity, specificity, and accuracy of 83.3%, 58.1%, and 71.5%, respectively. With AI, BE nonexperts showed a significant improvement in sensitivity and specificity when videos were assessed a second time with AI (sensitivity 69.8% [95%CI 65.2%-74.2%] to 78.0% [95%CI 74.0%-82.0%]; specificity 67.3% [95%CI 62.5%-72.2%] to 72.7% [95%CI 68.2%-77.3%]). In addition, the diagnostic confidence of BE nonexperts improved significantly with AI. CONCLUSION BE nonexperts benefitted significantly from additional AI. BE experts and nonexperts remained significantly below the stand-alone performance of AI, suggesting that there may be other factors influencing endoscopists' decisions to follow or discard AI advice.
Collapse
Affiliation(s)
- Michael Meinikheim
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Robert Mendel
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Christoph Palm
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Andreas Probst
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Anna Muzalyova
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Markus W Scheppach
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Sandra Nagl
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Elisabeth Schnoy
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Christoph Römmele
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Dominik A H Schulz
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Jakob Schlottmann
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Friederike Prinz
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - David Rauber
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Tobias Rückert
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Tomoaki Matsumura
- Department of Gastroenterology, Chiba University Graduate School of Medicine, Chiba, Japan
| | - Glòria Fernández-Esparrach
- Endoscopy Unit, Gastroenterology Department, ICMDM, Hospital Clínic de Barcelona, Barcelona, Spain
- Faculty of Medicine, University of Barcelona, Barcelona, Spain
- Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- Centro de Investigación Biomédica en Red de Enfermedades Hepáticas y Digestivas (CIBEREHD), Barcelona, Spain
| | - Nasim Parsa
- Division of Gastroenterology and Hepatology, Mayo Clinic, Scottsdale, United States
- Satisfai Health, Vancouver, Canada
| | - Michael F Byrne
- Satisfai Health, Vancouver, Canada
- Gastroenterology, Vancouver General Hospital, The University of British Columbia, Vancouver, Canada
| | - Helmut Messmann
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Alanna Ebigbo
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| |
Collapse
|
2
|
Ren J, Che J, Gong P, Wang X, Li X, Li A, Xiao C. Cross comparison representation learning for semi-supervised segmentation of cellular nuclei in immunofluorescence staining. Comput Biol Med 2024; 171:108102. [PMID: 38350398 DOI: 10.1016/j.compbiomed.2024.108102] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Revised: 01/29/2024] [Accepted: 02/04/2024] [Indexed: 02/15/2024]
Abstract
The morphological analysis of cells from optical images is vital for interpreting brain function in disease states. Extracting comprehensive cell morphology from intricate backgrounds, common in neural and some medical images, poses a significant challenge. Due to the huge workload of manual recognition, automated neuron cell segmentation using deep learning algorithms with labeled data is integral to neural image analysis tools. To combat the high cost of acquiring labeled data, we propose a novel semi-supervised cell segmentation algorithm for immunofluorescence-stained cell image datasets (ISC), utilizing a mean-teacher semi-supervised learning framework. We include a "cross comparison representation learning block" to enhance the teacher-student model comparison on high-dimensional channels, thereby improving feature compactness and separability, which results in the extraction of higher-dimensional features from unlabeled data. We also suggest a new network, the Multi Pooling Layer Attention Dense Network (MPAD-Net), serving as the backbone of the student model to augment segmentation accuracy. Evaluations on the immunofluorescence staining datasets and the public CRAG dataset illustrate our method surpasses other top semi-supervised learning methods, achieving average Jaccard, Dice and Normalized Surface Dice (NSD) indicators of 83.22%, 90.95% and 81.90% with only 20% labeled data. The datasets and code are available on the website at https://github.com/Brainsmatics/CCRL.
Collapse
Affiliation(s)
- Jianran Ren
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China
| | - Jingyi Che
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China
| | - Peicong Gong
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China
| | - Xiaojun Wang
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China
| | - Xiangning Li
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China
| | - Anan Li
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China; Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Chi Xiao
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Sanya 572025, China; Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Sanya 572025, China.
| |
Collapse
|
3
|
Rueckert T, Rueckert D, Palm C. Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art. Comput Biol Med 2024; 169:107929. [PMID: 38184862 DOI: 10.1016/j.compbiomed.2024.107929] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 12/02/2023] [Accepted: 01/01/2024] [Indexed: 01/09/2024]
Abstract
In the field of computer- and robot-assisted minimally invasive surgery, enormous progress has been made in recent years based on the recognition of surgical instruments in endoscopic images and videos. In particular, the determination of the position and type of instruments is of great interest. Current work involves both spatial and temporal information, with the idea that predicting the movement of surgical tools over time may improve the quality of the final segmentations. The provision of publicly available datasets has recently encouraged the development of new methods, mainly based on deep learning. In this review, we identify and characterize datasets used for method development and evaluation and quantify their frequency of use in the literature. We further present an overview of the current state of research regarding the segmentation and tracking of minimally invasive surgical instruments in endoscopic images and videos. The paper focuses on methods that work purely visually, without markers of any kind attached to the instruments, considering both single-frame semantic and instance segmentation approaches, as well as those that incorporate temporal information. The publications analyzed were identified through the platforms Google Scholar, Web of Science, and PubMed. The search terms used were "instrument segmentation", "instrument tracking", "surgical tool segmentation", and "surgical tool tracking", resulting in a total of 741 articles published between 01/2015 and 07/2023, of which 123 were included using systematic selection criteria. A discussion of the reviewed literature is provided, highlighting existing shortcomings and emphasizing the available potential for future developments.
Collapse
Affiliation(s)
- Tobias Rueckert
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany.
| | - Daniel Rueckert
- Artificial Intelligence in Healthcare and Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany; Department of Computing, Imperial College London, UK
| | - Christoph Palm
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany; Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg, Germany
| |
Collapse
|
4
|
Wang J, Ding X, Xiao J. Poisson-based image editing for semi-supervised vitiligo lesion segmentation with limited annotations. Comput Biol Med 2023; 165:107320. [PMID: 37625258 DOI: 10.1016/j.compbiomed.2023.107320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2023] [Revised: 07/17/2023] [Accepted: 08/07/2023] [Indexed: 08/27/2023]
Abstract
Vitiligo lesion segmentation is crucial for the assessment and treatment of vitiligo. There are two significant challenges in this problem, namely, the availability of dense segmentation annotations and the collection of large amounts of vitiligo images, which are also major challenges in medical image analysis (MIA). However, most existing methods often heavily rely on the availability of large-scale labeled datasets and high-quality annotations. Consequently, the performance of these models may not be easily reproducible or transferable to those domains with limited data availability. As a result, there is a need to develop alternative approaches that can leverage unlabeled datasets for segmentation with a small-scale training set. In this paper, we propose a data augmentation strategy based on image editing, which can synthesize a large number of samples using a small number of annotated data. The synthesized examples are of high visual quality and enforce the segmentation performance without any cost. Besides, we also adapt the Mean-Teacher framework for reliable predictions mining from unlabeled samples to alleviate the demands of densely annotated segmentations. We obtain pseudo-labels for unlabeled samples by utilizing highly confident pixels. On the other hand, we proposed a new Bimodal Vitiligo Lesions Segmentation (BVLS) dataset containing fine-grain segmentation masks and bimodal images usually used for vitiligo diagnosis to mitigate the lack of a vitiligo segmentation dataset. Extensive experiments conducted on the BLVS dataset demonstrate that our approach can achieve significant improvements (+17.27%) compared with previous data augmentation methods on the UNet backbone. Furthermore, the semi-supervised framework can reach an IoU of 49.71% with only 10% annotated images. Our code and dataset are availabel at https://github.com/JcWang20/BLVS.
Collapse
Affiliation(s)
- Jiacong Wang
- School of Artificial Intelligence, University of Chinese Academy of Sciences, China
| | - Xiaolan Ding
- Department of Dermatology, Peking University People's Hospital, China
| | - Jun Xiao
- School of Artificial Intelligence, University of Chinese Academy of Sciences, China.
| |
Collapse
|
5
|
Tang Y, Wang S, Qu Y, Cui Z, Zhang W. Consistency and adversarial semi-supervised learning for medical image segmentation. Comput Biol Med 2023; 161:107018. [PMID: 37216776 DOI: 10.1016/j.compbiomed.2023.107018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 04/15/2023] [Accepted: 05/05/2023] [Indexed: 05/24/2023]
Abstract
Medical image segmentation based on deep learning has made enormous progress in recent years. However, the performance of existing methods generally heavily relies on a large amount of labeled data, which are commonly expensive and time-consuming to obtain. To settle above issue, in this paper, a novel semi-supervised medical image segmentation method is proposed, in which the adversarial training mechanism and the collaborative consistency learning strategy are introduced into the mean teacher model. With the adversarial training mechanism, the discriminator can generate confidence maps for unlabeled data, such that more reliable supervised information for the student network is exploited. In the process of adversarial training, we further propose a collaborative consistency learning strategy by which the auxiliary discriminator can assist the primary discriminator in achieving supervised information with higher quality. We extensively evaluate our method on three representative yet challenging medical image segmentation tasks: (1) skin lesion segmentation from dermoscopy images in the International Skin Imaging Collaboration (ISIC) 2017 dataset; (2) optic cup and optic disk (OC/OD) segmentation from fundus images in the Retinal Fundus Glaucoma Challenge (REFUGE) dataset; and (3) tumor segmentation from lower-grade glioma (LGG) tumors images. The experimental results validate the superiority and effectiveness of our proposal when compared with the state-of-the-art semi-supervised medical image segmentation methods.
Collapse
Affiliation(s)
- Yongqiang Tang
- State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China.
| | - Shilei Wang
- School of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan, Shanxi, 030024, China.
| | - Yuxun Qu
- College of Computer Science, Nankai University, Tianjin, 300350, China.
| | - Zhihua Cui
- School of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan, Shanxi, 030024, China.
| | - Wensheng Zhang
- State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China.
| |
Collapse
|