1
|
Chang J, Hatfield B. Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond. Adv Cancer Res 2024; 161:431-478. [PMID: 39032956 DOI: 10.1016/bs.acr.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/23/2024]
Abstract
The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.
Collapse
Affiliation(s)
- Justin Chang
- Virginia Commonwealth University Health System, Richmond, VA, United States
| | - Bryce Hatfield
- Virginia Commonwealth University Health System, Richmond, VA, United States.
| |
Collapse
|
2
|
McGenity C, Clarke EL, Jennings C, Matthews G, Cartlidge C, Freduah-Agyemang H, Stocken DD, Treanor D. Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy. NPJ Digit Med 2024; 7:114. [PMID: 38704465 PMCID: PMC11069583 DOI: 10.1038/s41746-024-01106-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 04/12/2024] [Indexed: 05/06/2024] Open
Abstract
Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.
Collapse
Affiliation(s)
- Clare McGenity
- University of Leeds, Leeds, UK.
- Leeds Teaching Hospitals NHS Trust, Leeds, UK.
| | - Emily L Clarke
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Charlotte Jennings
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | | | | | | | - Darren Treanor
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
- Department of Clinical Pathology and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden
- Centre for Medical Image Science and Visualization (CMIV), Linköping University, Linköping, Sweden
| |
Collapse
|
3
|
Yengec-Tasdemir SB, Aydin Z, Akay E, Dogan S, Yilmaz B. An effective colorectal polyp classification for histopathological images based on supervised contrastive learning. Comput Biol Med 2024; 172:108267. [PMID: 38479197 DOI: 10.1016/j.compbiomed.2024.108267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Revised: 03/06/2024] [Accepted: 03/06/2024] [Indexed: 03/26/2024]
Abstract
Early detection of colon adenomatous polyps is pivotal in reducing colon cancer risk. In this context, accurately distinguishing between adenomatous polyp subtypes, especially tubular and tubulovillous, from hyperplastic variants is crucial. This study introduces a cutting-edge computer-aided diagnosis system optimized for this task. Our system employs advanced Supervised Contrastive learning to ensure precise classification of colon histopathology images. Significantly, we have integrated the Big Transfer model, which has gained prominence for its exemplary adaptability to visual tasks in medical imaging. Our novel approach discerns between in-class and out-of-class images, thereby elevating its discriminatory power for polyp subtypes. We validated our system using two datasets: a specially curated one and the publicly accessible UniToPatho dataset. The results reveal that our model markedly surpasses traditional deep convolutional neural networks, registering classification accuracies of 87.1% and 70.3% for the custom and UniToPatho datasets, respectively. Such results emphasize the transformative potential of our model in polyp classification endeavors.
Collapse
Affiliation(s)
- Sena Busra Yengec-Tasdemir
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, BT39DT, United Kingdom.
| | - Zafer Aydin
- Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey; Department of Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey
| | - Ebru Akay
- Pathology Clinic, Kayseri City Hospital, Kayseri, 38080, Turkey
| | - Serkan Dogan
- Gastroenterology Clinic, Kayseri City Hospital, Kayseri, 38080, Turkey
| | - Bulent Yilmaz
- Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey; Department of Electrical Engineering, Gulf University for Science and Technology, Mishref, 40005, Kuwait
| |
Collapse
|
4
|
Neto PC, Montezuma D, Oliveira SP, Oliveira D, Fraga J, Monteiro A, Monteiro J, Ribeiro L, Gonçalves S, Reinhard S, Zlobec I, Pinto IM, Cardoso JS. An interpretable machine learning system for colorectal cancer diagnosis from pathology slides. NPJ Precis Oncol 2024; 8:56. [PMID: 38443695 PMCID: PMC10914836 DOI: 10.1038/s41698-024-00539-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 02/08/2024] [Indexed: 03/07/2024] Open
Abstract
Considering the profound transformation affecting pathology practice, we aimed to develop a scalable artificial intelligence (AI) system to diagnose colorectal cancer from whole-slide images (WSI). For this, we propose a deep learning (DL) system that learns from weak labels, a sampling strategy that reduces the number of training samples by a factor of six without compromising performance, an approach to leverage a small subset of fully annotated samples, and a prototype with explainable predictions, active learning features and parallelisation. Noting some problems in the literature, this study is conducted with one of the largest WSI colorectal samples dataset with approximately 10,500 WSIs. Of these samples, 900 are testing samples. Furthermore, the robustness of the proposed method is assessed with two additional external datasets (TCGA and PAIP) and a dataset of samples collected directly from the proposed prototype. Our proposed method predicts, for the patch-based tiles, a class based on the severity of the dysplasia and uses that information to classify the whole slide. It is trained with an interpretable mixed-supervision scheme to leverage the domain knowledge introduced by pathologists through spatial annotations. The mixed-supervision scheme allowed for an intelligent sampling strategy effectively evaluated in several different scenarios without compromising the performance. On the internal dataset, the method shows an accuracy of 93.44% and a sensitivity between positive (low-grade and high-grade dysplasia) and non-neoplastic samples of 0.996. On the external test samples varied with TCGA being the most challenging dataset with an overall accuracy of 84.91% and a sensitivity of 0.996.
Collapse
Affiliation(s)
- Pedro C Neto
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
- Faculty of Engineering, University of Porto (FEUP), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
| | - Diana Montezuma
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal.
- Cancer Biology and Epigenetics Group, Research Center of IPO Porto (CI-IPOP) / RISE@CI-IPOP (Health Research Network), Portuguese Oncology Institute of Porto (IPO Porto) / Porto Comprehensive Cancer Center (Porto.CCC), R. Dr. António Bernardino de Almeida 865, Porto, 4200-072, Porto, Portugal.
- Doctoral Programme in Medical Sciences, School of Medicine and Biomedical Sciences - University of Porto (ICBAS-UP), R. Jorge de Viterbo Ferreira 228, Porto, 4050-313, Porto, Portugal.
| | - Sara P Oliveira
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
- Faculty of Engineering, University of Porto (FEUP), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
| | - Domingos Oliveira
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - João Fraga
- Department of Pathology, IPO-Porto, R. Dr. António Bernardino de Almeida 865, Porto, 4200-072, Porto, Portugal
| | - Ana Monteiro
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - João Monteiro
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Liliana Ribeiro
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Sofia Gonçalves
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Stefan Reinhard
- Institute of Pathology, University of Bern, Uni Bern, Murtenstrasse 31, Bern, 3008, Bern, Switzerland
| | - Inti Zlobec
- Institute of Pathology, University of Bern, Uni Bern, Murtenstrasse 31, Bern, 3008, Bern, Switzerland
| | - Isabel M Pinto
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Jaime S Cardoso
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal
- Faculty of Engineering, University of Porto (FEUP), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal
| |
Collapse
|
5
|
Hua H, Zhou Y, Li W, Zhang J, Deng Y, Khoo BL. Microfluidics-based patient-derived disease detection tool for deep learning-assisted precision medicine. BIOMICROFLUIDICS 2024; 18:014101. [PMID: 38223546 PMCID: PMC10787641 DOI: 10.1063/5.0172146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Accepted: 12/11/2023] [Indexed: 01/16/2024]
Abstract
Cancer spatial and temporal heterogeneity fuels resistance to therapies. To realize the routine assessment of cancer prognosis and treatment, we demonstrate the development of an Intelligent Disease Detection Tool (IDDT), a microfluidic-based tumor model integrated with deep learning-assisted algorithmic analysis. IDDT was clinically validated with liquid blood biopsy samples (n = 71) from patients with various types of cancers (e.g., breast, gastric, and lung cancer) and healthy donors, requiring low sample volume (∼200 μl) and a high-throughput 3D tumor culturing system (∼300 tumor clusters). To support automated algorithmic analysis, intelligent decision-making, and precise segmentation, we designed and developed an integrative deep neural network, which includes Mask Region-Based Convolutional Neural Network (Mask R-CNN), vision transformer, and Segment Anything Model (SAM). Our approach significantly reduces the manual labeling time by up to 90% with a high mean Intersection Over Union (mIoU) of 0.902 and immediate results (<2 s per image) for clinical cohort classification. The IDDT can accurately stratify healthy donors (n = 12) and cancer patients (n = 55) within their respective treatment cycle and cancer stage, resulting in high precision (∼99.3%) and high sensitivity (∼98%). We envision that our patient-centric IDDT provides an intelligent, label-free, and cost-effective approach to help clinicians make precise medical decisions and tailor treatment strategies for each patient.
Collapse
Affiliation(s)
| | - Yunlan Zhou
- Department of Clinical Laboratory, Xinhua Hospital, Shanghai Jiaotong University School of Medicine, Shanghai 200092, China
| | | | - Jing Zhang
- Department of Biomedical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong 999077, China
| | - Yanlin Deng
- Department of Biomedical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong 999077, China
| | - Bee Luan Khoo
- Authors to whom correspondence should be addressed:; ; and
| |
Collapse
|
6
|
Kolasa K, Admassu B, Hołownia-Voloskova M, Kędzior KJ, Poirrier JE, Perni S. Systematic reviews of machine learning in healthcare: a literature review. Expert Rev Pharmacoecon Outcomes Res 2024; 24:63-115. [PMID: 37955147 DOI: 10.1080/14737167.2023.2279107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 10/31/2023] [Indexed: 11/14/2023]
Abstract
INTRODUCTION The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery. METHODS A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted. RESULTS In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively). EXPERT OPINION The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.
Collapse
Affiliation(s)
- Katarzyna Kolasa
- Division of Health Economics and Healthcare Management, Kozminski University, Warsaw, Poland
| | - Bisrat Admassu
- Division of Health Economics and Healthcare Management, Kozminski University, Warsaw, Poland
| | | | | | | | | |
Collapse
|
7
|
Griem J, Eich ML, Schallenberg S, Pryalukhin A, Bychkov A, Fukuoka J, Zayats V, Hulla W, Munkhdelger J, Seper A, Tsvetkov T, Mukhopadhyay A, Sanner A, Stieber J, Fuchs M, Babendererde N, Schömig-Markiefka B, Klein S, Buettner R, Quaas A, Tolkach Y. Artificial Intelligence-Based Tool for Tumor Detection and Quantitative Tissue Analysis in Colorectal Specimens. Mod Pathol 2023; 36:100327. [PMID: 37683932 DOI: 10.1016/j.modpat.2023.100327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 08/11/2023] [Accepted: 08/31/2023] [Indexed: 09/10/2023]
Abstract
Digital pathology adoption allows for applying computational algorithms to routine pathology tasks. Our study aimed to develop a clinical-grade artificial intelligence (AI) tool for precise multiclass tissue segmentation in colorectal specimens (resections and biopsies) and clinically validate the tool for tumor detection in biopsy specimens. The training data set included 241 precisely manually annotated whole-slide images (WSIs) from multiple institutions. The algorithm was trained for semantic segmentation of 11 tissue classes with an additional module for biopsy WSI classification. Six case cohorts from 5 pathology departments (4 countries) were used for formal and clinical validation, digitized by 4 different scanning systems. The developed algorithm showed high precision of segmentation of different tissue classes in colorectal specimens with composite multiclass Dice score of up to 0.895 and pixel-wise tumor detection specificity and sensitivity of up to 0.958 and 0.987, respectively. In the clinical validation study on multiple external cohorts, the AI tool reached sensitivity of 1.0 and specificity of up to 0.969 for tumor detection in biopsy WSI. The AI tool analyzes most biopsy cases in less than 1 minute, allowing effective integration into clinical routine. We developed and extensively validated a highly accurate, clinical-grade tool for assistive diagnostic processing of colorectal specimens. This tool allows for quantitative deciphering of colorectal cancer tissue for development of prognostic and predictive biomarkers and personalization of oncologic care. This study is a foundation for a SemiCOL computational challenge. We open-source multiple manually annotated and weakly labeled test data sets, representing a significant contribution to the colorectal cancer computational pathology field.
Collapse
Affiliation(s)
- Johanna Griem
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | - Alexey Pryalukhin
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | - Andrey Bychkov
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Junya Fukuoka
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Vitaliy Zayats
- Laboratory for Medical Artificial Intelligence, The Resource Center for Universal Design and Rehabilitation Technologies (RCUD and RT), Moscow, Russia
| | - Wolfgang Hulla
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | | | - Alexander Seper
- Danube Private University, Medical Faculty, Krems-Stein, Austria
| | - Tsvetan Tsvetkov
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | | | | | - Moritz Fuchs
- Technical University Darmstadt, Darmstadt, Germany
| | | | | | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Reinhard Buettner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany.
| |
Collapse
|
8
|
Bilal M, Tsang YW, Ali M, Graham S, Hero E, Wahab N, Dodd K, Sahota H, Wu S, Lu W, Jahanifar M, Robinson A, Azam A, Benes K, Nimir M, Hewitt K, Bhalerao A, Eldaly H, Raza SEA, Gopalakrishnan K, Minhas F, Snead D, Rajpoot N. Development and validation of artificial intelligence-based prescreening of large-bowel biopsies taken in the UK and Portugal: a retrospective cohort study. Lancet Digit Health 2023; 5:e786-e797. [PMID: 37890902 DOI: 10.1016/s2589-7500(23)00148-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 07/10/2023] [Accepted: 07/25/2023] [Indexed: 10/29/2023]
Abstract
BACKGROUND Histopathological examination is a crucial step in the diagnosis and treatment of many major diseases. Aiming to facilitate diagnostic decision making and improve the workload of pathologists, we developed an artificial intelligence (AI)-based prescreening tool that analyses whole-slide images (WSIs) of large-bowel biopsies to identify typical, non-neoplastic, and neoplastic biopsies. METHODS This retrospective cohort study was conducted with an internal development cohort of slides acquired from a hospital in the UK and three external validation cohorts of WSIs acquired from two hospitals in the UK and one clinical laboratory in Portugal. To learn the differential histological patterns from digitised WSIs of large-bowel biopsy slides, our proposed weakly supervised deep-learning model (Colorectal AI Model for Abnormality Detection [CAIMAN]) used slide-level diagnostic labels and no detailed cell or region-level annotations. The method was developed with an internal development cohort of 5054 biopsy slides from 2080 patients that were labelled with corresponding diagnostic categories assigned by pathologists. The three external validation cohorts, with a total of 1536 slides, were used for independent validation of CAIMAN. Each WSI was classified into one of three classes (ie, typical, atypical non-neoplastic, and atypical neoplastic). Prediction scores of image tiles were aggregated into three prediction scores for the whole slide, one for its likelihood of being typical, one for its likelihood of being non-neoplastic, and one for its likelihood of being neoplastic. The assessment of the external validation cohorts was conducted by the trained and frozen CAIMAN model. To evaluate model performance, we calculated area under the convex hull of the receiver operating characteristic curve (AUROC), area under the precision-recall curve, and specificity compared with our previously published iterative draw and rank sampling (IDaRS) algorithm. We also generated heat maps and saliency maps to analyse and visualise the relationship between the WSI diagnostic labels and spatial features of the tissue microenvironment. The main outcome of this study was the ability of CAIMAN to accurately identify typical and atypical WSIs of colon biopsies, which could potentially facilitate automatic removing of typical biopsies from the diagnostic workload in clinics. FINDINGS A randomly selected subset of all large bowel biopsies was obtained between Jan 1, 2012, and Dec 31, 2017. The AI training, validation, and assessments were done between Jan 1, 2021, and Sept 30, 2022. WSIs with diagnostic labels were collected between Jan 1 and Sept 30, 2022. Our analysis showed no statistically significant differences across prediction scores from CAIMAN for typical and atypical classes based on anatomical sites of the biopsy. At 0·99 sensitivity, CAIMAN (specificity 0·5592) was more accurate than an IDaRS-based weakly supervised WSI-classification pipeline (0·4629) in identifying typical and atypical biopsies on cross-validation in the internal development cohort (p<0·0001). At 0·99 sensitivity, CAIMAN was also more accurate than IDaRS for two external validation cohorts (p<0·0001), but not for a third external validation cohort (p=0·10). CAIMAN provided higher specificity than IDaRS at some high-sensitivity thresholds (0·7763 vs 0·6222 for 0·95 sensitivity, 0·7126 vs 0·5407 for 0·97 sensitivity, and 0·5615 vs 0·3970 for 0·99 sensitivity on one of the external validation cohorts) and showed high classification performance in distinguishing between neoplastic biopsies (AUROC 0·9928, 95% CI 0·9927-0·9929), inflammatory biopsies (0·9658, 0·9655-0·9661), and atypical biopsies (0·9789, 0·9786-0·9792). On the three external validation cohorts, CAIMAN had AUROC values of 0·9431 (95% CI 0·9165-0·9697), 0·9576 (0·9568-0·9584), and 0·9636 (0·9615-0·9657) for the detection of atypical biopsies. Saliency maps supported the representation of disease heterogeneity in model predictions and its association with relevant histological features. INTERPRETATION CAIMAN, with its high sensitivity in detecting atypical large-bowel biopsies, might be a promising improvement in clinical workflow efficiency and diagnostic decision making in prescreening of typical colorectal biopsies. FUNDING The Pathology Image Data Lake for Analytics, Knowledge and Education Centre of Excellence; the UK Government's Industrial Strategy Challenge Fund; and Innovate UK on behalf of UK Research and Innovation.
Collapse
Affiliation(s)
- Mohsin Bilal
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Artificial Intelligence and Data Science, National University of Computer and Emerging Sciences, Islamabad, Pakistan
| | - Yee Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Mahmoud Ali
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Histofy, Birmingham, UK
| | - Emily Hero
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Department of Pathology, University Hospitals of Leicester National Health Service Trust, Leicester, UK
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Katherine Dodd
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Harvir Sahota
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Shaobin Wu
- Department of Pathology, East Suffolk and North Essex National Health Service Foundation Trust, Colchester, UK
| | - Wenqi Lu
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Andrew Robinson
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Ayesha Azam
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Ksenija Benes
- Department of Pathology, The Royal Wolverhampton National Health Service Trust, Wolverhampton, UK
| | - Mohammed Nimir
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Katherine Hewitt
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Hesham Eldaly
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Kishore Gopalakrishnan
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - David Snead
- Warwick Medical School, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Histofy, Birmingham, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Histofy, Birmingham, UK; The Alan Turing Institute, London, UK.
| |
Collapse
|
9
|
Ruiz Sarrias O, Gónzalez Deza C, Rodríguez Rodríguez J, Arrizibita Iriarte O, Vizcay Atienza A, Zumárraga Lizundia T, Sayar Beristain O, Aldaz Pastor A. Predicting Severe Haematological Toxicity in Gastrointestinal Cancer Patients Undergoing 5-FU-Based Chemotherapy: A Bayesian Network Approach. Cancers (Basel) 2023; 15:4206. [PMID: 37686482 PMCID: PMC10486471 DOI: 10.3390/cancers15174206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 07/28/2023] [Accepted: 08/16/2023] [Indexed: 09/10/2023] Open
Abstract
PURPOSE Severe toxicity is reported in about 30% of gastrointestinal cancer patients receiving 5-Fluorouracil (5-FU)-based chemotherapy. To date, limited tools exist to identify at risk patients in this setting. The objective of this study was to address this need by designing a predictive model using a Bayesian network, a probabilistic graphical model offering robust, explainable predictions. METHODS We utilized a dataset of 267 gastrointestinal cancer patients, conducting preprocessing, and splitting it into TRAIN and TEST sets (80%:20% ratio). The RandomForest algorithm assessed variable importance based on MeanDecreaseGini coefficient. The bnlearn R library helped design a Bayesian network model using a 10-fold cross-validation on the TRAIN set and the aic-cg method for network structure optimization. The model's performance was gauged based on accuracy, sensitivity, and specificity, using cross-validation on the TRAIN set and independent validation on the TEST set. RESULTS The model demonstrated satisfactory performance with an average accuracy of 0.85 (±0.05) and 0.80 on TRAIN and TEST datasets, respectively. The sensitivity and specificity were 0.82 (±0.14) and 0.87 (±0.07) for the TRAIN dataset, and 0.71 and 0.83 for the TEST dataset, respectively. A user-friendly tool was developed for clinical implementation. CONCLUSIONS Despite several limitations, our Bayesian network model demonstrated a high level of accuracy in predicting the risk of developing severe haematological toxicity in gastrointestinal cancer patients receiving 5-FU-based chemotherapy. Future research should aim at model validation in larger cohorts of patients and different clinical settings.
Collapse
Affiliation(s)
- Oskitz Ruiz Sarrias
- Department of Mathematics and Statistic, NNBi, 31191 Esquiroz, Navarra, Spain; (O.R.S.)
| | - Cristina Gónzalez Deza
- Department of Medical Oncology, Clínica Universidad De Navarra, 31008 Pamplona, Navarra, Spain; (C.G.D.); (J.R.R.); (T.Z.L.)
| | - Javier Rodríguez Rodríguez
- Department of Medical Oncology, Clínica Universidad De Navarra, 31008 Pamplona, Navarra, Spain; (C.G.D.); (J.R.R.); (T.Z.L.)
| | | | - Angel Vizcay Atienza
- Department of Medical Oncology, Clínica Universidad De Navarra, 31008 Pamplona, Navarra, Spain; (C.G.D.); (J.R.R.); (T.Z.L.)
| | - Teresa Zumárraga Lizundia
- Department of Medical Oncology, Clínica Universidad De Navarra, 31008 Pamplona, Navarra, Spain; (C.G.D.); (J.R.R.); (T.Z.L.)
| | | | | |
Collapse
|
10
|
Yengec-Tasdemir SB, Aydin Z, Akay E, Dogan S, Yilmaz B. Improved classification of colorectal polyps on histopathological images with ensemble learning and stain normalization. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 232:107441. [PMID: 36905748 DOI: 10.1016/j.cmpb.2023.107441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/14/2023] [Revised: 02/05/2023] [Accepted: 02/21/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Early detection of colon adenomatous polyps is critically important because correct detection of it significantly reduces the potential of developing colon cancers in the future. The key challenge in the detection of adenomatous polyps is differentiating it from its visually similar counterpart, non-adenomatous tissues. Currently, it solely depends on the experience of the pathologist. To assist the pathologists, the objective of this work is to provide a novel non-knowledge-based Clinical Decision Support System (CDSS) for improved detection of adenomatous polyps on colon histopathology images. METHODS The domain shift problem arises when the train and test data are coming from different distributions of diverse settings and unequal color levels. This problem, which can be tackled by stain normalization techniques, restricts the machine learning models to attain higher classification accuracies. In this work, the proposed method integrates stain normalization techniques with ensemble of competitively accurate, scalable and robust variants of CNNs, ConvNexts. The improvement is empirically analyzed for five widely employed stain normalization techniques. The classification performance of the proposed method is evaluated on three datasets comprising more than 10k colon histopathology images. RESULTS The comprehensive experiments demonstrate that the proposed method outperforms the state-of-the-art deep convolutional neural network based models by attaining 95% classification accuracy on the curated dataset, and 91.1% and 90% on EBHI and UniToPatho public datasets, respectively. CONCLUSIONS These results show that the proposed method can accurately classify colon adenomatous polyps on histopathology images. It retains remarkable performance scores even for different datasets coming from different distributions. This indicates that the model has a notable generalization ability.
Collapse
Affiliation(s)
- Sena Busra Yengec-Tasdemir
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, BT39DT, United Kingdom; Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey.
| | - Zafer Aydin
- Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey; Department of Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey
| | - Ebru Akay
- Pathology Clinic, Kayseri City Hospital, Kayseri, 38080, Turkey
| | - Serkan Dogan
- Gastroenterology Clinic, Kayseri City Hospital, Kayseri, 38080, Turkey
| | - Bulent Yilmaz
- Department of Electrical Engineering, Gulf University for Science and Technology, Mishref, 40005, Kuwait; Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, 38080, Turkey.
| |
Collapse
|
11
|
Cai H, Feng X, Yin R, Zhao Y, Guo L, Fan X, Liao J. MIST: multiple instance learning network based on Swin Transformer for whole slide image classification of colorectal adenomas. J Pathol 2023; 259:125-135. [PMID: 36318158 DOI: 10.1002/path.6027] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2022] [Revised: 09/30/2022] [Accepted: 10/28/2022] [Indexed: 12/12/2022]
Abstract
Colorectal adenoma is a recognized precancerous lesion of colorectal cancer (CRC), and at least 80% of colorectal cancers are malignantly transformed from it. Therefore, it is essential to distinguish benign from malignant adenomas in the early screening of colorectal cancer. Many deep learning computational pathology studies based on whole slide images (WSIs) have been proposed. Most approaches require manual annotation of lesion regions on WSIs, which is time-consuming and labor-intensive. This study proposes a new approach, MIST - Multiple Instance learning network based on the Swin Transformer, which can accurately classify colorectal adenoma WSIs only with slide-level labels. MIST uses the Swin Transformer as the backbone to extract features of images through self-supervised contrastive learning and uses a dual-stream multiple instance learning network to predict the class of slides. We trained and validated MIST on 666 WSIs collected from 480 colorectal adenoma patients in the Department of Pathology, The Affiliated Drum Tower Hospital of Nanjing University Medical School. These slides contained six common types of colorectal adenomas. The accuracy of external validation on 273 newly collected WSIs from Nanjing First Hospital was 0.784, which was superior to the existing methods and reached a level comparable to that of the local pathologist's accuracy of 0.806. Finally, we analyzed the interpretability of MIST and observed that the lesion areas of interest in MIST were generally consistent with those of interest to local pathologists. In conclusion, MIST is a low-burden, interpretable, and effective approach that can be used in colorectal cancer screening and may lead to a potential reduction in the mortality of CRC patients by assisting clinicians in the decision-making process. © 2022 The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Hongbin Cai
- School of Science, China Pharmaceutical University, Nanjing, PR China
| | - Xiaobing Feng
- College of Electrical and Information Engineering, Hunan University, Changsha, PR China
| | - Ruomeng Yin
- School of Science, China Pharmaceutical University, Nanjing, PR China
| | - Youcai Zhao
- Department of Pathology, Nanjing First Hospital, Nanjing, PR China
| | - Lingchuan Guo
- Department of Pathology, The First Affiliated Hospital of Soochow University, Soochow, PR China
| | - Xiangshan Fan
- Department of Pathology, The Affiliated Drum Tower Hospital of Nanjing University Medical School, Nanjing, PR China
| | - Jun Liao
- School of Science, China Pharmaceutical University, Nanjing, PR China
| |
Collapse
|
12
|
Wong ANN, He Z, Leung KL, To CCK, Wong CY, Wong SCC, Yoo JS, Chan CKR, Chan AZ, Lacambra MD, Yeung MHY. Current Developments of Artificial Intelligence in Digital Pathology and Its Future Clinical Applications in Gastrointestinal Cancers. Cancers (Basel) 2022; 14:3780. [PMID: 35954443 PMCID: PMC9367360 DOI: 10.3390/cancers14153780] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 07/27/2022] [Accepted: 08/01/2022] [Indexed: 02/05/2023] Open
Abstract
The implementation of DP will revolutionize current practice by providing pathologists with additional tools and algorithms to improve workflow. Furthermore, DP will open up opportunities for development of AI-based tools for more precise and reproducible diagnosis through computational pathology. One of the key features of AI is its capability to generate perceptions and recognize patterns beyond the human senses. Thus, the incorporation of AI into DP can reveal additional morphological features and information. At the current rate of AI development and adoption of DP, the interest in computational pathology is expected to rise in tandem. There have already been promising developments related to AI-based solutions in prostate cancer detection; however, in the GI tract, development of more sophisticated algorithms is required to facilitate histological assessment of GI specimens for early and accurate diagnosis. In this review, we aim to provide an overview of the current histological practices in AP laboratories with respect to challenges faced in image preprocessing, present the existing AI-based algorithms, discuss their limitations and present clinical insight with respect to the application of AI in early detection and diagnosis of GI cancer.
Collapse
Affiliation(s)
- Alex Ngai Nick Wong
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Zebang He
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Ka Long Leung
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Curtis Chun Kit To
- Department of Anatomical and Cellular Pathology, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong SAR, China; (C.C.K.T.); (C.K.R.C.); (M.D.L.)
| | - Chun Yin Wong
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Sze Chuen Cesar Wong
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Jung Sun Yoo
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Cheong Kin Ronald Chan
- Department of Anatomical and Cellular Pathology, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong SAR, China; (C.C.K.T.); (C.K.R.C.); (M.D.L.)
| | - Angela Zaneta Chan
- Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, Shatin, Hong Kong SAR, China;
| | - Maribel D. Lacambra
- Department of Anatomical and Cellular Pathology, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong SAR, China; (C.C.K.T.); (C.K.R.C.); (M.D.L.)
| | - Martin Ho Yin Yeung
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| |
Collapse
|
13
|
Deep Neural Network Models for Colon Cancer Screening. Cancers (Basel) 2022; 14:cancers14153707. [PMID: 35954370 PMCID: PMC9367621 DOI: 10.3390/cancers14153707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 07/26/2022] [Accepted: 07/27/2022] [Indexed: 12/24/2022] Open
Abstract
Simple Summary Deep learning models have been shown to achieve high performance in diagnosing colon cancer compared to conventional image processing and hand-crafted machine learning methods. Hence, several studies have focused on developing hybrid learning, end-to-end, and transfer learning techniques to reduce manual interaction and for labelling the regions of interest. However, these weak learning techniques do not always provide a clear diagnosis. Therefore, it is necessary to develop a clear explainable learning method that can highlight factors and form the basis of clinical decisions. However, there has been little research carried out employing such transparent approaches. This study discussed the aforementioned models for colon cancer diagnosis. Abstract Early detection of colorectal cancer can significantly facilitate clinicians’ decision-making and reduce their workload. This can be achieved using automatic systems with endoscopic and histological images. Recently, the success of deep learning has motivated the development of image- and video-based polyp identification and segmentation. Currently, most diagnostic colonoscopy rooms utilize artificial intelligence methods that are considered to perform well in predicting invasive cancer. Convolutional neural network-based architectures, together with image patches and preprocesses are often widely used. Furthermore, learning transfer and end-to-end learning techniques have been adopted for detection and localization tasks, which improve accuracy and reduce user dependence with limited datasets. However, explainable deep networks that provide transparency, interpretability, reliability, and fairness in clinical diagnostics are preferred. In this review, we summarize the latest advances in such models, with or without transparency, for the prediction of colorectal cancer and also address the knowledge gap in the upcoming technology.
Collapse
|
14
|
Neto PC, Oliveira SP, Montezuma D, Fraga J, Monteiro A, Ribeiro L, Gonçalves S, Pinto IM, Cardoso JS. iMIL4PATH: A Semi-Supervised Interpretable Approach for Colorectal Whole-Slide Images. Cancers (Basel) 2022; 14:cancers14102489. [PMID: 35626093 PMCID: PMC9139905 DOI: 10.3390/cancers14102489] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 05/13/2022] [Accepted: 05/17/2022] [Indexed: 02/04/2023] Open
Abstract
Simple Summary Nowadays, colorectal cancer is the third most incident cancer worldwide and, although it can be detected by imaging techniques, diagnosis is always based on biopsy samples. This assessment includes neoplasia grading, a subjective yet important task for pathologists. With the growing availability of digital slides, the development of robust and high-performance computer vision algorithms can help to tackle such a task. In this work, we propose an approach to automatically detect and grade lesions in colorectal biopsies with high sensitivity. The presented model attempts to support slide decision reasoning in terms of the spatial distribution of lesions, focusing the pathologist’s attention on key areas. Thus, it can be integrated into clinical practice as a second opinion or as a flag for details that may have been missed at first glance. Abstract Colorectal cancer (CRC) diagnosis is based on samples obtained from biopsies, assessed in pathology laboratories. Due to population growth and ageing, as well as better screening programs, the CRC incidence rate has been increasing, leading to a higher workload for pathologists. In this sense, the application of AI for automatic CRC diagnosis, particularly on whole-slide images (WSI), is of utmost relevance, in order to assist professionals in case triage and case review. In this work, we propose an interpretable semi-supervised approach to detect lesions in colorectal biopsies with high sensitivity, based on multiple-instance learning and feature aggregation methods. The model was developed on an extended version of the recent, publicly available CRC dataset (the CRC+ dataset with 4433 WSI), using 3424 slides for training and 1009 slides for evaluation. The proposed method attained 90.19% classification ACC, 98.8% sensitivity, 85.7% specificity, and a quadratic weighted kappa of 0.888 at slide-based evaluation. Its generalisation capabilities are also studied on two publicly available external datasets.
Collapse
Affiliation(s)
- Pedro C. Neto
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal; (S.P.O.); (J.S.C.)
- Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal
- Correspondence:
| | - Sara P. Oliveira
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal; (S.P.O.); (J.S.C.)
- Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal
| | - Diana Montezuma
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
- School of Medicine and Biomedical Sciences, University of Porto (ICBAS), 4050-313 Porto, Portugal
- Cancer Biology and Epigenetics Group, IPO-Porto, 4200-072 Porto, Portugal
| | - João Fraga
- Department of Pathology, IPO-Porto, 4200-072 Porto, Portugal;
| | - Ana Monteiro
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Liliana Ribeiro
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Sofia Gonçalves
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Isabel M. Pinto
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Jaime S. Cardoso
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal; (S.P.O.); (J.S.C.)
- Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal
| |
Collapse
|
15
|
Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review. Diagnostics (Basel) 2022; 12:diagnostics12040837. [PMID: 35453885 PMCID: PMC9028395 DOI: 10.3390/diagnostics12040837] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 03/22/2022] [Accepted: 03/25/2022] [Indexed: 02/04/2023] Open
Abstract
Colorectal cancer (CRC) is the second most common cancer in women and the third most common in men, with an increasing incidence. Pathology diagnosis complemented with prognostic and predictive biomarker information is the first step for personalized treatment. The increased diagnostic load in the pathology laboratory, combined with the reported intra- and inter-variability in the assessment of biomarkers, has prompted the quest for reliable machine-based methods to be incorporated into the routine practice. Recently, Artificial Intelligence (AI) has made significant progress in the medical field, showing potential for clinical applications. Herein, we aim to systematically review the current research on AI in CRC image analysis. In histopathology, algorithms based on Deep Learning (DL) have the potential to assist in diagnosis, predict clinically relevant molecular phenotypes and microsatellite instability, identify histological features related to prognosis and correlated to metastasis, and assess the specific components of the tumor microenvironment.
Collapse
|
16
|
Deep Learning Approaches to Colorectal Cancer Diagnosis: A Review. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app112210982] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Unprecedented breakthroughs in the development of graphical processing systems have led to great potential for deep learning (DL) algorithms in analyzing visual anatomy from high-resolution medical images. Recently, in digital pathology, the use of DL technologies has drawn a substantial amount of attention for use in the effective diagnosis of various cancer types, especially colorectal cancer (CRC), which is regarded as one of the dominant causes of cancer-related deaths worldwide. This review provides an in-depth perspective on recently published research articles on DL-based CRC diagnosis and prognosis. Overall, we provide a retrospective synopsis of simple image-processing-based and machine learning (ML)-based computer-aided diagnosis (CAD) systems, followed by a comprehensive appraisal of use cases with different types of state-of-the-art DL algorithms for detecting malignancies. We first list multiple standardized and publicly available CRC datasets from two imaging types: colonoscopy and histopathology. Secondly, we categorize the studies based on the different types of CRC detected (tumor tissue, microsatellite instability, and polyps), and we assess the data preprocessing steps and the adopted DL architectures before presenting the optimum diagnostic results. CRC diagnosis with DL algorithms is still in the preclinical phase, and therefore, we point out some open issues and provide some insights into the practicability and development of robust diagnostic systems in future health care and oncology.
Collapse
|
17
|
Kuntz S, Krieghoff-Henning E, Kather JN, Jutzi T, Höhn J, Kiehl L, Hekler A, Alwers E, von Kalle C, Fröhling S, Utikal JS, Brenner H, Hoffmeister M, Brinker TJ. Gastrointestinal cancer classification and prognostication from histology using deep learning: Systematic review. Eur J Cancer 2021; 155:200-215. [PMID: 34391053 DOI: 10.1016/j.ejca.2021.07.012] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2021] [Accepted: 07/06/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Gastrointestinal cancers account for approximately 20% of all cancer diagnoses and are responsible for 22.5% of cancer deaths worldwide. Artificial intelligence-based diagnostic support systems, in particular convolutional neural network (CNN)-based image analysis tools, have shown great potential in medical computer vision. In this systematic review, we summarise recent studies reporting CNN-based approaches for digital biomarkers for characterization and prognostication of gastrointestinal cancer pathology. METHODS Pubmed and Medline were screened for peer-reviewed papers dealing with CNN-based gastrointestinal cancer analyses from histological slides, published between 2015 and 2020.Seven hundred and ninety titles and abstracts were screened, and 58 full-text articles were assessed for eligibility. RESULTS Sixteen publications fulfilled our inclusion criteria dealing with tumor or precursor lesion characterization or prognostic and predictive biomarkers: 14 studies on colorectal or rectal cancer, three studies on gastric cancer and none on esophageal cancer. These studies were categorised according to their end-points: polyp characterization, tumor characterization and patient outcome. Regarding the translation into clinical practice, we identified several studies demonstrating generalization of the classifier with external tests and comparisons with pathologists, but none presenting clinical implementation. CONCLUSIONS Results of recent studies on CNN-based image analysis in gastrointestinal cancer pathology are promising, but studies were conducted in observational and retrospective settings. Large-scale trials are needed to assess performance and predict clinical usefulness. Furthermore, large-scale trials are required for approval of CNN-based prediction models as medical devices.
Collapse
Affiliation(s)
- Sara Kuntz
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jakob N Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Tanja Jutzi
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Julia Höhn
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Lennard Kiehl
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Elizabeth Alwers
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christof von Kalle
- Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany
| | - Stefan Fröhling
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jochen S Utikal
- Department of Dermatology, Heidelberg University, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Division of Preventive Oncology, German Cancer Research Center (DKFZ), National Center for Tumor Diseases (NCT), Heidelberg, Germany; German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
18
|
Oliveira SP, Neto PC, Fraga J, Montezuma D, Monteiro A, Monteiro J, Ribeiro L, Gonçalves S, Pinto IM, Cardoso JS. CAD systems for colorectal cancer from WSI are still not ready for clinical acceptance. Sci Rep 2021; 11:14358. [PMID: 34257363 PMCID: PMC8277780 DOI: 10.1038/s41598-021-93746-z] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Accepted: 06/28/2021] [Indexed: 02/07/2023] Open
Abstract
Most oncological cases can be detected by imaging techniques, but diagnosis is based on pathological assessment of tissue samples. In recent years, the pathology field has evolved to a digital era where tissue samples are digitised and evaluated on screen. As a result, digital pathology opened up many research opportunities, allowing the development of more advanced image processing techniques, as well as artificial intelligence (AI) methodologies. Nevertheless, despite colorectal cancer (CRC) being the second deadliest cancer type worldwide, with increasing incidence rates, the application of AI for CRC diagnosis, particularly on whole-slide images (WSI), is still a young field. In this review, we analyse some relevant works published on this particular task and highlight the limitations that hinder the application of these works in clinical practice. We also empirically investigate the feasibility of using weakly annotated datasets to support the development of computer-aided diagnosis systems for CRC from WSI. Our study underscores the need for large datasets in this field and the use of an appropriate learning methodology to gain the most benefit from partially annotated datasets. The CRC WSI dataset used in this study, containing 1,133 colorectal biopsy and polypectomy samples, is available upon reasonable request.
Collapse
Affiliation(s)
- Sara P Oliveira
- INESCTEC, 4200-465, Porto, Portugal.
- Faculty of Engineering (FEUP), University of Porto, 4200-465, Porto, Portugal.
| | - Pedro C Neto
- INESCTEC, 4200-465, Porto, Portugal
- Faculty of Engineering (FEUP), University of Porto, 4200-465, Porto, Portugal
| | - João Fraga
- IMP Diagnostics, 4150-146, Porto, Portugal
| | - Diana Montezuma
- IMP Diagnostics, 4150-146, Porto, Portugal
- ICBAS, University of Porto, 4050-313, Porto , Portugal
- Cancer Biology and Epigenetics Group, IPO-Porto, 4200-072, Porto, Portugal
| | | | | | | | | | | | - Jaime S Cardoso
- INESCTEC, 4200-465, Porto, Portugal
- Faculty of Engineering (FEUP), University of Porto, 4200-465, Porto, Portugal
| |
Collapse
|
19
|
Cao JS, Lu ZY, Chen MY, Zhang B, Juengpanich S, Hu JH, Li SJ, Topatana W, Zhou XY, Feng X, Shen JL, Liu Y, Cai XJ. Artificial intelligence in gastroenterology and hepatology: Status and challenges. World J Gastroenterol 2021; 27:1664-1690. [PMID: 33967550 PMCID: PMC8072192 DOI: 10.3748/wjg.v27.i16.1664] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Revised: 02/11/2021] [Accepted: 03/17/2021] [Indexed: 02/06/2023] Open
Abstract
Originally proposed by John McCarthy in 1955, artificial intelligence (AI) has achieved a breakthrough and revolutionized the processing methods of clinical medicine with the increasing workloads of medical records and digital images. Doctors are paying attention to AI technologies for various diseases in the fields of gastroenterology and hepatology. This review will illustrate AI technology procedures for medical image analysis, including data processing, model establishment, and model validation. Furthermore, we will summarize AI applications in endoscopy, radiology, and pathology, such as detecting and evaluating lesions, facilitating treatment, and predicting treatment response and prognosis with excellent model performance. The current challenges for AI in clinical application include potential inherent bias in retrospective studies that requires larger samples for validation, ethics and legal concerns, and the incomprehensibility of the output results. Therefore, doctors and researchers should cooperate to address the current challenges and carry out further investigations to develop more accurate AI tools for improved clinical applications.
Collapse
Affiliation(s)
- Jia-Sheng Cao
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Zi-Yi Lu
- Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Ming-Yu Chen
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Bin Zhang
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Sarun Juengpanich
- Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Jia-Hao Hu
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Shi-Jie Li
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Win Topatana
- Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Xue-Yin Zhou
- School of Medicine, Wenzhou Medical University, Wenzhou 325035, Zhejiang Province, China
| | - Xu Feng
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Ji-Liang Shen
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Yu Liu
- College of Life Sciences, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Xiu-Jun Cai
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| |
Collapse
|