1
|
Kildal W, Cyll K, Kalsnes J, Islam R, Julbø FM, Pradhan M, Ersvær E, Shepherd N, Vlatkovic L, Tekpli X, Garred Ø, Kristensen GB, Askautrud HA, Hveem TS, Danielsen HE. Deep learning for automated scoring of immunohistochemically stained tumour tissue sections - Validation across tumour types based on patient outcomes. Heliyon 2024; 10:e32529. [PMID: 39040241 PMCID: PMC11261074 DOI: 10.1016/j.heliyon.2024.e32529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2024] [Accepted: 06/05/2024] [Indexed: 07/24/2024] Open
Abstract
We aimed to develop deep learning (DL) models to detect protein expression in immunohistochemically (IHC) stained tissue-sections, and to compare their accuracy and performance with manually scored clinically relevant proteins in common cancer types. Five cancer patient cohorts (colon, two prostate, breast, and endometrial) were included. We developed separate DL models for scoring IHC-stained tissue-sections with nuclear, cytoplasmic, and membranous staining patterns. For training, we used images with annotations of cells with positive and negative staining from the colon cohort stained for Ki-67 and PMS2 (nuclear model), the prostate cohort 1 stained for PTEN (cytoplasmic model) and β-catenin (membranous model). The nuclear DL model was validated for MSH6 in the colon, MSH6 and PMS2 in the endometrium, Ki-67 and CyclinB1 in prostate, and oestrogen and progesterone receptors in the breast cancer cohorts. The cytoplasmic DL model was validated for PTEN and Mapre2, and the membranous DL model for CD44 and Flotillin1, all in prostate cohorts. When comparing the results of manual and DL scores in the validation sets, using manual scores as the ground truth, we observed an average correct classification rate of 91.5 % (76.9-98.5 %) for the nuclear model, 85.6 % (73.3-96.6 %) for the cytoplasmic model, and 78.4 % (75.5-84.3 %) for the membranous model. In survival analyses, manual and DL scores showed similar prognostic impact, with similar hazard ratios and p-values for all DL models. Our findings demonstrate that DL models offer a promising alternative to manual IHC scoring, providing efficiency and reproducibility across various data sources and markers.
Collapse
Affiliation(s)
- Wanja Kildal
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Karolina Cyll
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Joakim Kalsnes
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Rakibul Islam
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Frida M. Julbø
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Manohar Pradhan
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Elin Ersvær
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Neil Shepherd
- Gloucestershire Cellular Pathology Laboratory, Gloucester, GL53 7AN, UK
| | - Ljiljana Vlatkovic
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - OSBREAC
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
- Gloucestershire Cellular Pathology Laboratory, Gloucester, GL53 7AN, UK
- Department of Medical Genetics, Institute of Clinical Medicine, Faculty of Medicine, University of Oslo and Oslo University Hospital, NO-0450, Oslo, Norway
- Department of Pathology, Oslo University Hospital, NO-0424, Oslo, Norway
- Nuffield Division of Clinical Laboratory Sciences, University of Oxford, Oxford, OX3 9DU, UK
| | - Xavier Tekpli
- Department of Medical Genetics, Institute of Clinical Medicine, Faculty of Medicine, University of Oslo and Oslo University Hospital, NO-0450, Oslo, Norway
| | - Øystein Garred
- Department of Pathology, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Gunnar B. Kristensen
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Hanne A. Askautrud
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Tarjei S. Hveem
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Håvard E. Danielsen
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
- Nuffield Division of Clinical Laboratory Sciences, University of Oxford, Oxford, OX3 9DU, UK
| |
Collapse
|
2
|
Juez-Castillo G, Valencia-Vidal B, Orrego LM, Cabello-Donayre M, Montosa-Hidalgo L, Pérez-Victoria JM. FiCRoN, a deep learning-based algorithm for the automatic determination of intracellular parasite burden from fluorescence microscopy images. Med Image Anal 2024; 91:103036. [PMID: 38016388 DOI: 10.1016/j.media.2023.103036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 06/27/2023] [Accepted: 11/13/2023] [Indexed: 11/30/2023]
Abstract
Protozoan parasites are responsible for dramatic, neglected diseases. The automatic determination of intracellular parasite burden from fluorescence microscopy images is a challenging problem. Recent advances in deep learning are transforming this process, however, high-performance algorithms have not been developed. The limitations in image acquisition, especially for intracellular parasites, make this process complex. For this reason, traditional image-processing methods are not easily transferred between different datasets and segmentation-based strategies do not have a high performance. Here, we propose a novel method FiCRoN, based on fully convolutional regression networks (FCRNs), as a promising new tool for estimating intracellular parasite burden. This estimation requires three values, intracellular parasites, infected cells and uninfected cells. FiCRoN solves this problem as multi-task learning: counting by regression at two scales, a smaller one for intracellular parasites and a larger one for host cells. It does not use segmentation or detection, resulting in a higher generalization of counting tasks and, therefore, a decrease in error propagation. Linear regression reveals an excellent correlation coefficient between manual and automatic methods. FiCRoN is an innovative freedom-respecting image analysis software based on deep learning, designed to provide a fast and accurate quantification of parasite burden, also potentially useful as a single-cell counter.
Collapse
Affiliation(s)
- Graciela Juez-Castillo
- Instituto de Parasitología y Biomedicina "López-Neyra", Consejo Superior de Investigaciones Cientìficas, (IPBLN-CSIC), PTS Granada, 18016 Granada, Spain; Research Group Osiris&Bioaxis, Faculty of Engineering, El Bosque University, 110121 Bogotá, Colombia
| | - Brayan Valencia-Vidal
- Research Group Osiris&Bioaxis, Faculty of Engineering, El Bosque University, 110121 Bogotá, Colombia; Department of Computer Engineering, Automation and Robotics, Research Centre for Information and Communication Technologies, University of Granada, 18014 Granada, Spain.
| | - Lina M Orrego
- Instituto de Parasitología y Biomedicina "López-Neyra", Consejo Superior de Investigaciones Cientìficas, (IPBLN-CSIC), PTS Granada, 18016 Granada, Spain
| | - María Cabello-Donayre
- Instituto de Parasitología y Biomedicina "López-Neyra", Consejo Superior de Investigaciones Cientìficas, (IPBLN-CSIC), PTS Granada, 18016 Granada, Spain; Universidad Internacional de la Rioja, 26006 La Rioja, Spain
| | - Laura Montosa-Hidalgo
- Instituto de Parasitología y Biomedicina "López-Neyra", Consejo Superior de Investigaciones Cientìficas, (IPBLN-CSIC), PTS Granada, 18016 Granada, Spain
| | - José M Pérez-Victoria
- Instituto de Parasitología y Biomedicina "López-Neyra", Consejo Superior de Investigaciones Cientìficas, (IPBLN-CSIC), PTS Granada, 18016 Granada, Spain.
| |
Collapse
|
3
|
Pantelis AG, Panagopoulou PA, Lapatsanis DP. Artificial Intelligence and Machine Learning in the Diagnosis and Management of Gastroenteropancreatic Neuroendocrine Neoplasms—A Scoping Review. Diagnostics (Basel) 2022; 12:diagnostics12040874. [PMID: 35453922 PMCID: PMC9027316 DOI: 10.3390/diagnostics12040874] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2022] [Revised: 03/27/2022] [Accepted: 03/29/2022] [Indexed: 12/21/2022] Open
Abstract
Neuroendocrine neoplasms (NENs) and tumors (NETs) are rare neoplasms that may affect any part of the gastrointestinal system. In this scoping review, we attempt to map existing evidence on the role of artificial intelligence, machine learning and deep learning in the diagnosis and management of NENs of the gastrointestinal system. After implementation of inclusion and exclusion criteria, we retrieved 44 studies with 53 outcome analyses. We then classified the papers according to the type of studied NET (26 Pan-NETs, 59.1%; 3 metastatic liver NETs (6.8%), 2 small intestinal NETs, 4.5%; colorectal, rectal, non-specified gastroenteropancreatic and non-specified gastrointestinal NETs had from 1 study each, 2.3%). The most frequently used AI algorithms were Supporting Vector Classification/Machine (14 analyses, 29.8%), Convolutional Neural Network and Random Forest (10 analyses each, 21.3%), Random Forest (9 analyses, 19.1%), Logistic Regression (8 analyses, 17.0%), and Decision Tree (6 analyses, 12.8%). There was high heterogeneity on the description of the prediction model, structure of datasets, and performance metrics, whereas the majority of studies did not report any external validation set. Future studies should aim at incorporating a uniform structure in accordance with existing guidelines for purposes of reproducibility and research quality, which are prerequisites for integration into clinical practice.
Collapse
Affiliation(s)
- Athanasios G. Pantelis
- 4th Department of Surgery, Evaggelismos General Hospital of Athens, 10676 Athens, Greece;
- Correspondence:
| | | | - Dimitris P. Lapatsanis
- 4th Department of Surgery, Evaggelismos General Hospital of Athens, 10676 Athens, Greece;
| |
Collapse
|
4
|
Xing F, Cornish TC, Bennett TD, Ghosh D. Bidirectional Mapping-Based Domain Adaptation for Nucleus Detection in Cross-Modality Microscopy Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:2880-2896. [PMID: 33284750 PMCID: PMC8543886 DOI: 10.1109/tmi.2020.3042789] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Cell or nucleus detection is a fundamental task in microscopy image analysis and has recently achieved state-of-the-art performance by using deep neural networks. However, training supervised deep models such as convolutional neural networks (CNNs) usually requires sufficient annotated image data, which is prohibitively expensive or unavailable in some applications. Additionally, when applying a CNN to new datasets, it is common to annotate individual cells/nuclei in those target datasets for model re-learning, leading to inefficient and low-throughput image analysis. To tackle these problems, we present a bidirectional, adversarial domain adaptation method for nucleus detection on cross-modality microscopy image data. Specifically, the method learns a deep regression model for individual nucleus detection with both source-to-target and target-to-source image translation. In addition, we explicitly extend this unsupervised domain adaptation method to a semi-supervised learning situation and further boost the nucleus detection performance. We evaluate the proposed method on three cross-modality microscopy image datasets, which cover a wide variety of microscopy imaging protocols or modalities, and obtain a significant improvement in nucleus detection compared to reference baseline approaches. In addition, our semi-supervised method is very competitive with recent fully supervised learning models trained with all real target training labels.
Collapse
|
5
|
Detection of Ki67 Hot-Spots of Invasive Breast Cancer Based on Convolutional Neural Networks Applied to Mutual Information of H&E and Ki67 Whole Slide Images. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10217761] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
Ki67 hot-spot detection and its evaluation in invasive breast cancer regions play a significant role in routine medical practice. The quantification of cellular proliferation assessed by Ki67 immunohistochemistry is an established prognostic and predictive biomarker that determines the choice of therapeutic protocols. In this paper, we present three deep learning-based approaches to automatically detect and quantify Ki67 hot-spot areas by means of the Ki67 labeling index. To this end, a dataset composed of 100 whole slide images (WSIs) belonging to 50 breast cancer cases (Ki67 and H&E WSI pairs) was used. Three methods based on CNN classification were proposed and compared to create the tumor proliferation map. The best results were obtained by applying the CNN to the mutual information acquired from the color deconvolution of both the Ki67 marker and the H&E WSIs. The overall accuracy of this approach was 95%. The agreement between the automatic Ki67 scoring and the manual analysis is promising with a Spearman’s ρ correlation of 0.92. The results illustrate the suitability of this CNN-based approach for detecting hot-spots areas of invasive breast cancer in WSI.
Collapse
|