1
|
Saragih DG, Hibi A, Tyrrell PN. Using diffusion models to generate synthetic labeled data for medical image segmentation. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03213-z. [PMID: 38900372 DOI: 10.1007/s11548-024-03213-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 06/04/2024] [Indexed: 06/21/2024]
Abstract
PURPOSE Medical image analysis has become a prominent area where machine learning has been applied. However, high-quality, publicly available data are limited either due to patient privacy laws or the time and cost required for experts to annotate images. In this retrospective study, we designed and evaluated a pipeline to generate synthetic labeled polyp images for augmenting medical image segmentation models with the aim of reducing this data scarcity. METHODS We trained diffusion models on the HyperKvasir dataset, comprising 1000 images of polyps in the human GI tract from 2008 to 2016. Qualitative expert review, Fréchet Inception Distance (FID), and Multi-Scale Structural Similarity (MS-SSIM) were tested for evaluation. Additionally, various segmentation models were trained with the generated data and evaluated using Dice score (DS) and Intersection over Union (IoU). RESULTS Our pipeline produced images more akin to real polyp images based on FID scores. Segmentation model performance also showed improvements over GAN methods when trained entirely, or partially, with synthetic data, despite requiring less compute for training. Moreover, the improvement persists when tested on different datasets, showcasing the transferability of the generated images. CONCLUSIONS The proposed pipeline produced realistic image and mask pairs which could reduce the need for manual data annotation when performing a machine learning task. We support this use case by showing that the methods proposed in this study enhanced segmentation model performance, as measured by Dice and IoU scores, when trained fully or partially on synthetic data.
Collapse
Affiliation(s)
- Daniel G Saragih
- Department of Medical Imaging, University of Toronto, 263 McCaul Street, Toronto, M5T 1W7, ON, Canada
| | - Atsuhiro Hibi
- Department of Medical Imaging, University of Toronto, 263 McCaul Street, Toronto, M5T 1W7, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
| | - Pascal N Tyrrell
- Department of Medical Imaging, University of Toronto, 263 McCaul Street, Toronto, M5T 1W7, ON, Canada.
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada.
- Department of Statistical Sciences, University of Toronto, Toronto, ON, Canada.
| |
Collapse
|
2
|
Boland PA, Hardy NP, Moynihan A, McEntee PD, Loo C, Fenlon H, Cahill RA. Intraoperative near infrared functional imaging of rectal cancer using artificial intelligence methods - now and near future state of the art. Eur J Nucl Med Mol Imaging 2024:10.1007/s00259-024-06731-9. [PMID: 38858280 DOI: 10.1007/s00259-024-06731-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2023] [Accepted: 04/15/2024] [Indexed: 06/12/2024]
Abstract
Colorectal cancer remains a major cause of cancer death and morbidity worldwide. Surgery is a major treatment modality for primary and, increasingly, secondary curative therapy. However, with more patients being diagnosed with early stage and premalignant disease manifesting as large polyps, greater accuracy in diagnostic and therapeutic precision is needed right from the time of first endoscopic encounter. Rapid advancements in the field of artificial intelligence (AI), coupled with widespread availability of near infrared imaging (currently based around indocyanine green (ICG)) can enable colonoscopic tissue classification and prognostic stratification for significant polyps, in a similar manner to contemporary dynamic radiological perfusion imaging but with the advantage of being able to do so directly within interventional procedural time frames. It can provide an explainable method for immediate digital biopsies that could guide or even replace traditional forceps biopsies and provide guidance re margins (both areas where current practice is only approximately 80% accurate prior to definitive excision). Here, we discuss the concept and practice of AI enhanced ICG perfusion analysis for rectal cancer surgery while highlighting recent and essential near-future advancements. These include breakthrough developments in computer vision and time series analysis that allow for real-time quantification and classification of fluorescent perfusion signals of rectal cancer tissue intraoperatively that accurately distinguish between normal, benign, and malignant tissues in situ endoscopically, which are now undergoing international prospective validation (the Horizon Europe CLASSICA study). Next stage advancements may include detailed digital characterisation of small rectal malignancy based on intraoperative assessment of specific intratumoral fluorescent signal pattern. This could include T staging and intratumoral molecular process profiling (e.g. regarding angiogenesis, differentiation, inflammatory component, and tumour to stroma ratio) with the potential to accurately predict the microscopic local response to nonsurgical treatment enabling personalised therapy via decision support tools. Such advancements are also applicable to the next generation fluorophores and imaging agents currently emerging from clinical trials. In addition, by providing an understandable, applicable method for detailed tissue characterisation visually, such technology paves the way for acceptance of other AI methodology during surgery including, potentially, deep learning methods based on whole screen/video detailing.
Collapse
Affiliation(s)
- Patrick A Boland
- UCD Centre for Precision Surgery, School of Medicine, University College Dublin, 47 Eccles Street, Dublin 7, Dublin, Ireland
- Department of Colorectal Surgery, Mater Misericordiae University Hospital, Dublin, Ireland
| | - N P Hardy
- UCD Centre for Precision Surgery, School of Medicine, University College Dublin, 47 Eccles Street, Dublin 7, Dublin, Ireland
- Department of Colorectal Surgery, Mater Misericordiae University Hospital, Dublin, Ireland
| | - A Moynihan
- UCD Centre for Precision Surgery, School of Medicine, University College Dublin, 47 Eccles Street, Dublin 7, Dublin, Ireland
- Department of Colorectal Surgery, Mater Misericordiae University Hospital, Dublin, Ireland
| | - P D McEntee
- UCD Centre for Precision Surgery, School of Medicine, University College Dublin, 47 Eccles Street, Dublin 7, Dublin, Ireland
- Department of Colorectal Surgery, Mater Misericordiae University Hospital, Dublin, Ireland
| | - C Loo
- UCD Centre for Precision Surgery, School of Medicine, University College Dublin, 47 Eccles Street, Dublin 7, Dublin, Ireland
| | - H Fenlon
- Department of Radiology, Mater Misericordiae University Hospital, Dublin, Ireland
| | - R A Cahill
- UCD Centre for Precision Surgery, School of Medicine, University College Dublin, 47 Eccles Street, Dublin 7, Dublin, Ireland.
- Department of Colorectal Surgery, Mater Misericordiae University Hospital, Dublin, Ireland.
| |
Collapse
|
3
|
Biffi C, Antonelli G, Bernhofer S, Hassan C, Hirata D, Iwatate M, Maieron A, Salvagnini P, Cherubini A. REAL-Colon: A dataset for developing real-world AI applications in colonoscopy. Sci Data 2024; 11:539. [PMID: 38796533 PMCID: PMC11127922 DOI: 10.1038/s41597-024-03359-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 05/10/2024] [Indexed: 05/28/2024] Open
Abstract
Detection and diagnosis of colon polyps are key to preventing colorectal cancer. Recent evidence suggests that AI-based computer-aided detection (CADe) and computer-aided diagnosis (CADx) systems can enhance endoscopists' performance and boost colonoscopy effectiveness. However, most available public datasets primarily consist of still images or video clips, often at a down-sampled resolution, and do not accurately represent real-world colonoscopy procedures. We introduce the REAL-Colon (Real-world multi-center Endoscopy Annotated video Library) dataset: a compilation of 2.7 M native video frames from sixty full-resolution, real-world colonoscopy recordings across multiple centers. The dataset contains 350k bounding-box annotations, each created under the supervision of expert gastroenterologists. Comprehensive patient clinical data, colonoscopy acquisition information, and polyp histopathological information are also included in each video. With its unprecedented size, quality, and heterogeneity, the REAL-Colon dataset is a unique resource for researchers and developers aiming to advance AI research in colonoscopy. Its openness and transparency facilitate rigorous and reproducible research, fostering the development and benchmarking of more accurate and reliable colonoscopy-related algorithms and models.
Collapse
Affiliation(s)
- Carlo Biffi
- Cosmo Intelligent Medical Devices, Dublin, Ireland.
| | - Giulio Antonelli
- Gastroenterology and Digestive Endoscopy Unit, Ospedale dei Castelli (N.O.C.), Rome, Italy
| | - Sebastian Bernhofer
- Karl Landsteiner University of Health Sciences, Krems, Austria
- Department of Internal Medicine 2, University Hospital St. Pölten, St. Pölten, Austria
| | - Cesare Hassan
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Endoscopy Unit, Humanitas Clinical and Research Center IRCCS, Rozzano, Italy
| | - Daizen Hirata
- Gastrointestinal Center, Sano Hospital, Hyogo, Japan
| | - Mineo Iwatate
- Gastrointestinal Center, Sano Hospital, Hyogo, Japan
| | - Andreas Maieron
- Karl Landsteiner University of Health Sciences, Krems, Austria
- Department of Internal Medicine 2, University Hospital St. Pölten, St. Pölten, Austria
| | | | - Andrea Cherubini
- Cosmo Intelligent Medical Devices, Dublin, Ireland.
- Milan Center for Neuroscience, University of Milano-Bicocca, Milano, Italy.
| |
Collapse
|
4
|
Xu C, Fan K, Mo W, Cao X, Jiao K. Dual ensemble system for polyp segmentation with submodels adaptive selection ensemble. Sci Rep 2024; 14:6152. [PMID: 38485963 PMCID: PMC10940608 DOI: 10.1038/s41598-024-56264-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 03/04/2024] [Indexed: 03/18/2024] Open
Abstract
Colonoscopy is one of the main methods to detect colon polyps, and its detection is widely used to prevent and diagnose colon cancer. With the rapid development of computer vision, deep learning-based semantic segmentation methods for colon polyps have been widely researched. However, the accuracy and stability of some methods in colon polyp segmentation tasks show potential for further improvement. In addition, the issue of selecting appropriate sub-models in ensemble learning for the colon polyp segmentation task still needs to be explored. In order to solve the above problems, we first implement the utilization of multi-complementary high-level semantic features through the Multi-Head Control Ensemble. Then, to solve the sub-model selection problem in training, we propose SDBH-PSO Ensemble for sub-model selection and optimization of ensemble weights for different datasets. The experiments were conducted on the public datasets CVC-ClinicDB, Kvasir, CVC-ColonDB, ETIS-LaribPolypDB and PolypGen. The results show that the DET-Former, constructed based on the Multi-Head Control Ensemble and the SDBH-PSO Ensemble, consistently provides improved accuracy across different datasets. Among them, the Multi-Head Control Ensemble demonstrated superior feature fusion capability in the experiments, and the SDBH-PSO Ensemble demonstrated excellent sub-model selection capability. The sub-model selection capabilities of the SDBH-PSO Ensemble will continue to have significant reference value and practical utility as deep learning networks evolve.
Collapse
Affiliation(s)
- Cun Xu
- Guilin University of Electronic Technology, Guilin, 541000, China
| | - Kefeng Fan
- China Electronics Standardization Institute, Beijing, 100007, China.
| | - Wei Mo
- Guilin University of Electronic Technology, Guilin, 541000, China
| | - Xuguang Cao
- Guilin University of Electronic Technology, Guilin, 541000, China
| | - Kaijie Jiao
- Guilin University of Electronic Technology, Guilin, 541000, China
| |
Collapse
|
5
|
Mozaffari J, Amirkhani A, Shokouhi SB. ColonGen: an efficient polyp segmentation system for generalization improvement using a new comprehensive dataset. Phys Eng Sci Med 2024; 47:309-325. [PMID: 38224384 DOI: 10.1007/s13246-023-01368-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 12/06/2023] [Indexed: 01/16/2024]
Abstract
Colorectal cancer (CRC) is one of the most common causes of cancer-related deaths. While polyp detection is important for diagnosing CRC, high miss rates for polyps have been reported during colonoscopy. Most deep learning methods extract features from images using convolutional neural networks (CNNs). In recent years, vision transformer (ViT) models have been employed for image processing and have been successful in image segmentation. It is possible to improve image processing by using transformer models that can extract spatial location information, and CNNs that are capable of aggregating local information. Despite this, recent research shows limited effectiveness in increasing data diversity and generalization accuracy. This paper investigates the generalization proficiency of polyp image segmentation based on transformer architecture and proposes a novel approach using two different ViT architectures. This allows the model to learn representations from different perspectives, which can then be combined to create a richer feature representation. Additionally, a more universal and comprehensive dataset has been derived from the datasets presented in the related research, which can be used for improving generalizations. We first evaluated the generalization of our proposed model using three distinct training-testing scenarios. Our experimental results demonstrate that our ColonGen-V1 outperforms other state-of-the-art methods in all scenarios. As a next step, we used the comprehensive dataset for improving the performance of the model against in- and out-of-domain data. The results show that our ColonGen-V2 outperforms state-of-the-art studies by 5.1%, 1.3%, and 1.1% in ETIS-Larib, Kvasir-Seg, and CVC-ColonDB datasets, respectively. The inclusive dataset and the model introduced in this paper are available to the public through this link: https://github.com/javadmozaffari/Polyp_segmentation .
Collapse
Affiliation(s)
- Javad Mozaffari
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, 16846-13114, Iran
| | - Abdollah Amirkhani
- School of Automotive Engineering, Iran University of Science and Technology, Tehran, 16846-13114, Iran.
| | - Shahriar B Shokouhi
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, 16846-13114, Iran
| |
Collapse
|
6
|
Yue G, Zhuo G, Yan W, Zhou T, Tang C, Yang P, Wang T. Boundary uncertainty aware network for automated polyp segmentation. Neural Netw 2024; 170:390-404. [PMID: 38029720 DOI: 10.1016/j.neunet.2023.11.050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 07/15/2023] [Accepted: 11/22/2023] [Indexed: 12/01/2023]
Abstract
Recently, leveraging deep neural networks for automated colorectal polyp segmentation has emerged as a hot topic due to the favored advantages in evading the limitations of visual inspection, e.g., overwork and subjectivity. However, most existing methods do not pay enough attention to the uncertain areas of colonoscopy images and often provide unsatisfactory segmentation performance. In this paper, we propose a novel boundary uncertainty aware network (BUNet) for precise and robust colorectal polyp segmentation. Specifically, considering that polyps vary greatly in size and shape, we first adopt a pyramid vision transformer encoder to learn multi-scale feature representations. Then, a simple yet effective boundary exploration module (BEM) is proposed to explore boundary cues from the low-level features. To make the network focus on the ambiguous area where the prediction score is biased to neither the foreground nor the background, we further introduce a boundary uncertainty aware module (BUM) that explores error-prone regions from the high-level features with the assistance of boundary cues provided by the BEM. Through the top-down hybrid deep supervision, our BUNet implements coarse-to-fine polyp segmentation and finally localizes polyp regions precisely. Extensive experiments on five public datasets show that BUNet is superior to thirteen competing methods in terms of both effectiveness and generalization ability.
Collapse
Affiliation(s)
- Guanghui Yue
- National-Reginoal Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Marshall Laboratory of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518060, China
| | - Guibin Zhuo
- National-Reginoal Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Marshall Laboratory of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518060, China
| | - Weiqing Yan
- School of Computer and Control Engineering, Yantai University, Yantai 264005, China
| | - Tianwei Zhou
- College of Management, Shenzhen University, Shenzhen 518060, China.
| | - Chang Tang
- School of Computer Science, China University of Geosciences, Wuhan 430074, China
| | - Peng Yang
- National-Reginoal Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Marshall Laboratory of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518060, China
| | - Tianfu Wang
- National-Reginoal Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Marshall Laboratory of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen 518060, China
| |
Collapse
|
7
|
Ali S, Ghatwary N, Jha D, Isik-Polat E, Polat G, Yang C, Li W, Galdran A, Ballester MÁG, Thambawita V, Hicks S, Poudel S, Lee SW, Jin Z, Gan T, Yu C, Yan J, Yeo D, Lee H, Tomar NK, Haithami M, Ahmed A, Riegler MA, Daul C, Halvorsen P, Rittscher J, Salem OE, Lamarque D, Cannizzaro R, Realdon S, de Lange T, East JE. Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge. Sci Rep 2024; 14:2032. [PMID: 38263232 PMCID: PMC10805888 DOI: 10.1038/s41598-024-52063-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Accepted: 01/12/2024] [Indexed: 01/25/2024] Open
Abstract
Polyps are well-known cancer precursors identified by colonoscopy. However, variability in their size, appearance, and location makes the detection of polyps challenging. Moreover, colonoscopy surveillance and removal of polyps are highly operator-dependent procedures and occur in a highly complex organ topology. There exists a high missed detection rate and incomplete removal of colonic polyps. To assist in clinical procedures and reduce missed rates, automated methods for detecting and segmenting polyps using machine learning have been achieved in past years. However, the major drawback in most of these methods is their ability to generalise to out-of-sample unseen datasets from different centres, populations, modalities, and acquisition systems. To test this hypothesis rigorously, we, together with expert gastroenterologists, curated a multi-centre and multi-population dataset acquired from six different colonoscopy systems and challenged the computational expert teams to develop robust automated detection and segmentation methods in a crowd-sourcing Endoscopic computer vision challenge. This work put forward rigorous generalisability tests and assesses the usability of devised deep learning methods in dynamic and actual clinical colonoscopy procedures. We analyse the results of four top performing teams for the detection task and five top performing teams for the segmentation task. Our analyses demonstrate that the top-ranking teams concentrated mainly on accuracy over the real-time performance required for clinical applicability. We further dissect the devised methods and provide an experiment-based hypothesis that reveals the need for improved generalisability to tackle diversity present in multi-centre datasets and routine clinical procedures.
Collapse
Affiliation(s)
- Sharib Ali
- School of Computing, Faculty of Engineering and Physical Sciences, University of Leeds, Leeds, LS2 9JT, UK.
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, OX3 7DQ, UK.
- Oxford National Institute for Health Research Biomedical Research Centre, Oxford, OX4 2PG, UK.
| | - Noha Ghatwary
- Computer Engineering Department, Arab Academy for Science and Technology, Smart Village, Giza, Egypt
| | - Debesh Jha
- SimulaMet, 0167, Oslo, Norway
- Department of Computer Science, UiT The Arctic University of Norway, Hansine Hansens veg 18, 9019, Tromsø, Norway
| | - Ece Isik-Polat
- Graduate School of Informatics, Middle East Technical University, 06800, Ankara, Turkey
| | - Gorkem Polat
- Graduate School of Informatics, Middle East Technical University, 06800, Ankara, Turkey
| | - Chen Yang
- City University of Hong Kong, Kowloon, Hong Kong
| | - Wuyang Li
- City University of Hong Kong, Kowloon, Hong Kong
| | - Adrian Galdran
- BCN MedTech, Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08018, Barcelona, Spain
| | - Miguel-Ángel González Ballester
- BCN MedTech, Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08018, Barcelona, Spain
- ICREA, Barcelona, Spain
| | | | | | - Sahadev Poudel
- Department of IT Convergence Engineering, Gachon University, Seongnam, 13120, Republic of Korea
| | - Sang-Woong Lee
- Department of IT Convergence Engineering, Gachon University, Seongnam, 13120, Republic of Korea
| | - Ziyi Jin
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, China
| | - Tianyuan Gan
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, China
| | - ChengHui Yu
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - JiangPeng Yan
- Department of Automation, Tsinghua University, Beijing, 100084, China
| | - Doyeob Yeo
- Smart Sensing and Diagnosis Research Division, Korea Atomic Energy Research Institute, Taejon, 34057, Republic of Korea
| | - Hyunseok Lee
- Daegu-Gyeongbuk Medical Innovation Foundation, Medical Device Development Center, Taegu, 427724, Republic of Korea
| | - Nikhil Kumar Tomar
- NepAL Applied Mathematics and Informatics Institute for Research (NAAMII), Kathmandu, Nepal
| | - Mahmood Haithami
- Computer Science Department, University of Nottingham, Malaysia Campus, 43500, Semenyih, Malaysia
| | - Amr Ahmed
- Computer Science, Edge Hill University, Lancashire, United Kingdom
| | - Michael A Riegler
- SimulaMet, 0167, Oslo, Norway
- Department of Computer Science, UiT The Arctic University of Norway, Hansine Hansens veg 18, 9019, Tromsø, Norway
| | - Christian Daul
- CRAN UMR 7039, Université de Lorraine and CNRS, 54500, Vandœuvre-Lès-Nancy, France
| | - Pål Halvorsen
- SimulaMet, 0167, Oslo, Norway
- Oslo Metropolitan University, Pilestredet 46, 0167, Oslo, Norway
| | - Jens Rittscher
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, OX3 7DQ, UK
| | - Osama E Salem
- Faculty of Medicine, University of Alexandria, Alexandria, 21131, Egypt
| | - Dominique Lamarque
- Université de Versailles St-Quentin en Yvelines, Hôpital Ambroise Paré, 9 Av. Charles de Gaulle, 92100, Boulogne-Billancourt, France
| | - Renato Cannizzaro
- CRO Centro Riferimento Oncologico IRCCS Aviano Italy, Via Franco Gallini, 2, 33081, Aviano, PN, Italy
| | - Stefano Realdon
- CRO Centro Riferimento Oncologico IRCCS Aviano Italy, Via Franco Gallini, 2, 33081, Aviano, PN, Italy
- Veneto Institute of Oncology IOV-IRCCS, Via Gattamelata, 64, 35128, Padua, Italy
| | - Thomas de Lange
- Medical Department, Sahlgrenska University Hospital-Mölndal, Blå stråket 5, 413 45, Göteborg, Sweden
- Department of Molecular and Clinical Medicine, Sahlgrenska Academy, University of Gothenburg, 41345, Göteborg, Sweden
- Augere Medical, Nedre Vaskegang 6, Oslo, 0186, Norway
| | - James E East
- Oxford National Institute for Health Research Biomedical Research Centre, Oxford, OX4 2PG, UK
- Translational Gastroenterology Unit, Nuffield Department of Medicine, Experimental Medicine Division, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| |
Collapse
|
8
|
Wang J, Jin Y, Stoyanov D, Wang L. FedDP: Dual Personalization in Federated Medical Image Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:297-308. [PMID: 37494156 DOI: 10.1109/tmi.2023.3299206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/28/2023]
Abstract
Personalized federated learning (PFL) addresses the data heterogeneity challenge faced by general federated learning (GFL). Rather than learning a single global model, with PFL a collection of models are adapted to the unique feature distribution of each site. However, current PFL methods rarely consider self-attention networks which can handle data heterogeneity by long-range dependency modeling and they do not utilize prediction inconsistencies in local models as an indicator of site uniqueness. In this paper, we propose FedDP, a novel fed erated learning scheme with d ual p ersonalization, which improves model personalization from both feature and prediction aspects to boost image segmentation results. We leverage long-range dependencies by designing a local query (LQ) that decouples the query embedding layer out of each local model, whose parameters are trained privately to better adapt to the respective feature distribution of the site. We then propose inconsistency-guided calibration (IGC), which exploits the inter-site prediction inconsistencies to accommodate the model learning concentration. By encouraging a model to penalize pixels with larger inconsistencies, we better tailor prediction-level patterns to each local site. Experimentally, we compare FedDP with the state-of-the-art PFL methods on two popular medical image segmentation tasks with different modalities, where our results consistently outperform others on both tasks. Our code and models are available at https://github.com/jcwang123/PFL-Seg-Trans.
Collapse
|
9
|
Zhu S, Gao J, Liu L, Yin M, Lin J, Xu C, Xu C, Zhu J. Public Imaging Datasets of Gastrointestinal Endoscopy for Artificial Intelligence: a Review. J Digit Imaging 2023; 36:2578-2601. [PMID: 37735308 PMCID: PMC10584770 DOI: 10.1007/s10278-023-00844-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 05/03/2023] [Accepted: 05/03/2023] [Indexed: 09/23/2023] Open
Abstract
With the advances in endoscopic technologies and artificial intelligence, a large number of endoscopic imaging datasets have been made public to researchers around the world. This study aims to review and introduce these datasets. An extensive literature search was conducted to identify appropriate datasets in PubMed, and other targeted searches were conducted in GitHub, Kaggle, and Simula to identify datasets directly. We provided a brief introduction to each dataset and evaluated the characteristics of the datasets included. Moreover, two national datasets in progress were discussed. A total of 40 datasets of endoscopic images were included, of which 34 were accessible for use. Basic and detailed information on each dataset was reported. Of all the datasets, 16 focus on polyps, and 6 focus on small bowel lesions. Most datasets (n = 16) were constructed by colonoscopy only, followed by normal gastrointestinal endoscopy and capsule endoscopy (n = 9). This review may facilitate the usage of public dataset resources in endoscopic research.
Collapse
Affiliation(s)
- Shiqi Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Jingwen Gao
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Lu Liu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Minyue Yin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Jiaxi Lin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Chang Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China
| | - Chunfang Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China.
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China.
| | - Jinzhou Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou , Jiangsu, 215000, China.
- Suzhou Clinical Center of Digestive Diseases, Suzhou, 215000, China.
| |
Collapse
|
10
|
Farooq MS, Tehseen R, Sabir M, Atal Z. Detection of autism spectrum disorder (ASD) in children and adults using machine learning. Sci Rep 2023; 13:9605. [PMID: 37311766 DOI: 10.1038/s41598-023-35910-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2022] [Accepted: 05/25/2023] [Indexed: 06/15/2023] Open
Abstract
Autism spectrum disorder (ASD) presents a neurological and developmental disorder that has an impact on the social and cognitive skills of children causing repetitive behaviours, restricted interests, communication problems and difficulty in social interaction. Early diagnosis of ASD can prevent from its severity and prolonged effects. Federated learning (FL) is one of the most recent techniques that can be applied for accurate ASD diagnoses in early stages or prevention of its long-term effects. In this article, FL technique has been uniquely applied for autism detection by training two different ML classifiers including logistic regression and support vector machine locally for classification of ASD factors and detection of ASD in children and adults. Due to FL, results obtained from these classifiers have been transmitted to central server where meta classifier is trained to determine which approach is most accurate in the detection of ASD in children and adults. Four different ASD patient datasets, each containing more than 600 records of effected children and adults have been obtained from different repository for features extraction. The proposed model predicted ASD with 98% accuracy (in children) and 81% accuracy (in adults).
Collapse
Affiliation(s)
- Muhammad Shoaib Farooq
- Department of Artificial Intelligence, University of Management and Technology, Lahore, 54000, Pakistan
| | - Rabia Tehseen
- Department of Computer Science, University of Central Punjab, Lahore, 54000, Pakistan
| | - Maidah Sabir
- Department of Artificial Intelligence, University of Management and Technology, Lahore, 54000, Pakistan
| | - Zabihullah Atal
- Department of Computer Science, Kardan University, Kabul, 1007, Afghanistan.
| |
Collapse
|
11
|
ELKarazle K, Raman V, Then P, Chua C. Detection of Colorectal Polyps from Colonoscopy Using Machine Learning: A Survey on Modern Techniques. SENSORS (BASEL, SWITZERLAND) 2023; 23:1225. [PMID: 36772263 PMCID: PMC9953705 DOI: 10.3390/s23031225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Revised: 01/08/2023] [Accepted: 01/17/2023] [Indexed: 06/18/2023]
Abstract
Given the increased interest in utilizing artificial intelligence as an assistive tool in the medical sector, colorectal polyp detection and classification using deep learning techniques has been an active area of research in recent years. The motivation for researching this topic is that physicians miss polyps from time to time due to fatigue and lack of experience carrying out the procedure. Unidentified polyps can cause further complications and ultimately lead to colorectal cancer (CRC), one of the leading causes of cancer mortality. Although various techniques have been presented recently, several key issues, such as the lack of enough training data, white light reflection, and blur affect the performance of such methods. This paper presents a survey on recently proposed methods for detecting polyps from colonoscopy. The survey covers benchmark dataset analysis, evaluation metrics, common challenges, standard methods of building polyp detectors and a review of the latest work in the literature. We conclude this paper by providing a precise analysis of the gaps and trends discovered in the reviewed literature for future work.
Collapse
Affiliation(s)
- Khaled ELKarazle
- School of Information and Communication Technologies, Swinburne University of Technology, Sarawak Campus, Kuching 93350, Malaysia
| | - Valliappan Raman
- Department of Artificial Intelligence and Data Science, Coimbatore Institute of Technology, Coimbatore 641014, India
| | - Patrick Then
- School of Information and Communication Technologies, Swinburne University of Technology, Sarawak Campus, Kuching 93350, Malaysia
| | - Caslon Chua
- Department of Computer Science and Software Engineering, Swinburne University of Technology, Melbourne 3122, Australia
| |
Collapse
|