1
|
Westhaeusser F, Fuhlert P, Dietrich E, Lennartz M, Khatri R, Kaiser N, Röbeck P, Bülow R, von Stillfried S, Witte A, Ladjevardi S, Drotte A, Severgardh P, Baumbach J, Puelles VG, Häggman M, Brehler M, Boor P, Walhagen P, Dragomir A, Busch C, Graefen M, Bengtsson E, Sauter G, Zimmermann M, Bonn S. Robust, credible, and interpretable AI-based histopathological prostate cancer grading. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.07.09.24310082. [PMID: 39040171 PMCID: PMC11261944 DOI: 10.1101/2024.07.09.24310082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/24/2024]
Abstract
Background Prostate cancer (PCa) is among the most common cancers in men and its diagnosis requires the histopathological evaluation of biopsies by human experts. While several recent artificial intelligence-based (AI) approaches have reached human expert-level PCa grading, they often display significantly reduced performance on external datasets. This reduced performance can be caused by variations in sample preparation, for instance the staining protocol, section thickness, or scanner used. Another limiting factor of contemporary AI-based PCa grading is the prediction of ISUP grades, which leads to the perpetuation of human annotation errors. Methods We developed the prostate cancer aggressiveness index (PCAI), an AI-based PCa detection and grading framework that is trained on objective patient outcome, rather than subjective ISUP grades. We designed PCAI as a clinical application, containing algorithmic modules that offer robustness to data variation, medical interpretability, and a measure of prediction confidence. To train and evaluate PCAI, we generated a multicentric, retrospective, observational trial consisting of six cohorts with 25,591 patients, 83,864 images, and 5 years of median follow-up from 5 different centers and 3 countries. This includes a high-variance dataset of 8,157 patients and 28,236 images with variations in sample thickness, staining protocol, and scanner, allowing for the systematic evaluation and optimization of model robustness to data variation. The performance of PCAI was assessed on three external test cohorts from two countries, comprising 2,255 patients and 9,437 images. Findings Using our high-variance datasets, we show how differences in sample processing, particularly slide thickness and staining time, significantly reduce the performance of AI-based PCa grading by up to 6.2 percentage points in the concordance index (C-index). We show how a select set of algorithmic improvements, including domain adversarial training, conferred robustness to data variation, interpretability, and a measure of credibility to PCAI. These changes lead to significant prediction improvement across two biopsy cohorts and one TMA cohort, systematically exceeding expert ISUP grading in C-index and AUROC by up to 22 percentage points. Interpretation Data variation poses serious risks for AI-based histopathological PCa grading, even when models are trained on large datasets. Algorithmic improvements for model robustness, interpretability, credibility, and training on high-variance data as well as outcome-based severity prediction gives rise to robust models with above ISUP-level PCa grading performance.
Collapse
Affiliation(s)
- Fabian Westhaeusser
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Spearpoint Analytics AB, Stockholm, Sweden
| | - Patrick Fuhlert
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Spearpoint Analytics AB, Stockholm, Sweden
| | - Esther Dietrich
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Maximilian Lennartz
- Institute of Pathology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Robin Khatri
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Nico Kaiser
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- III. Department of Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Pontus Röbeck
- Department of Urology, Uppsala University Hospital, Uppsala, Sweden
| | - Roman Bülow
- Institute of Pathology, RWTH Aachen University Hospital, Aachen, Germany
| | | | - Anja Witte
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Sam Ladjevardi
- Department of Urology, Uppsala University Hospital, Uppsala, Sweden
| | | | | | - Jan Baumbach
- Institute of Computational Systems Biology, University of Hamburg, Germany
| | - Victor G. Puelles
- III. Department of Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Pathology, Aarhus University Hospital, Aarhus, Denmark
| | - Michael Häggman
- Department of Urology, Uppsala University Hospital, Uppsala, Sweden
| | - Michael Brehler
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Peter Boor
- Institute of Pathology, RWTH Aachen University Hospital, Aachen, Germany
| | | | - Anca Dragomir
- Department of Pathology, Uppsala University Hospital and Department of Immunology, Genetics and Pathology, Uppsala University, Uppsala, Sweden
| | - Christer Busch
- Spearpoint Analytics AB, Stockholm, Sweden
- Department of Urology, Uppsala University Hospital, Uppsala, Sweden
| | - Markus Graefen
- Martini-Klinik Prostate Cancer Center, University Hospital Hamburg-Eppendorf, Hamburg, Germany
| | - Ewert Bengtsson
- Spearpoint Analytics AB, Stockholm, Sweden
- Uppsala University, Department of Information Technology, Centre for Image Analysis, Uppsala, Sweden
| | - Guido Sauter
- Institute of Pathology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Marina Zimmermann
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Stefan Bonn
- Institute of Medical Systems Biology, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Spearpoint Analytics AB, Stockholm, Sweden
| |
Collapse
|
2
|
Shephard AJ, Bashir RMS, Mahmood H, Jahanifar M, Minhas F, Raza SEA, McCombe KD, Craig SG, James J, Brooks J, Nankivell P, Mehanna H, Khurram SA, Rajpoot NM. A fully automated and explainable algorithm for predicting malignant transformation in oral epithelial dysplasia. NPJ Precis Oncol 2024; 8:137. [PMID: 38942998 PMCID: PMC11213925 DOI: 10.1038/s41698-024-00624-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 05/29/2024] [Indexed: 06/30/2024] Open
Abstract
Oral epithelial dysplasia (OED) is a premalignant histopathological diagnosis given to lesions of the oral cavity. Its grading suffers from significant inter-/intra-observer variability, and does not reliably predict malignancy progression, potentially leading to suboptimal treatment decisions. To address this, we developed an artificial intelligence (AI) algorithm, that assigns an Oral Malignant Transformation (OMT) risk score based on the Haematoxylin and Eosin (H&E) stained whole slide images (WSIs). Our AI pipeline leverages an in-house segmentation model to detect and segment both nuclei and epithelium. Subsequently, a shallow neural network utilises interpretable morphological and spatial features, emulating histological markers, to predict progression. We conducted internal cross-validation on our development cohort (Sheffield; n = 193 cases) and independent validation on two external cohorts (Birmingham and Belfast; n = 89 cases). On external validation, the proposed OMTscore achieved an AUROC = 0.75 (Recall = 0.92) in predicting OED progression, outperforming other grading systems (Binary: AUROC = 0.72, Recall = 0.85). Survival analyses showed the prognostic value of our OMTscore (C-index = 0.60, p = 0.02), compared to WHO (C-index = 0.64, p = 0.003) and binary grades (C-index = 0.65, p < 0.001). Nuclear analyses elucidated the presence of peri-epithelial and intra-epithelial lymphocytes in highly predictive patches of transforming cases (p < 0.001). This is the first study to propose a completely automated, explainable, and externally validated algorithm for predicting OED transformation. Our algorithm shows comparable-to-human-level performance, offering a promising solution to the challenges of grading OED in routine clinical practice.
Collapse
Affiliation(s)
- Adam J Shephard
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | | | - Hanya Mahmood
- School of Clinical Dentistry, University of Sheffield, Sheffield, UK
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Kris D McCombe
- Precision Medicine Centre, Patrick G. Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, UK
| | - Stephanie G Craig
- Precision Medicine Centre, Patrick G. Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, UK
| | - Jacqueline James
- Precision Medicine Centre, Patrick G. Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, UK
| | - Jill Brooks
- Institute of Head and Neck Studies and Education, Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK
| | - Paul Nankivell
- Institute of Head and Neck Studies and Education, Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK
| | - Hisham Mehanna
- Institute of Head and Neck Studies and Education, Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK
| | - Syed Ali Khurram
- School of Clinical Dentistry, University of Sheffield, Sheffield, UK
| | - Nasir M Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK.
| |
Collapse
|
3
|
Nakhli R, Rich K, Zhang A, Darbandsari A, Shenasa E, Hadjifaradji A, Thiessen S, Milne K, Jones SJM, McAlpine JN, Nelson BH, Gilks CB, Farahani H, Bashashati A. VOLTA: an enVironment-aware cOntrastive ceLl represenTation leArning for histopathology. Nat Commun 2024; 15:3942. [PMID: 38729933 PMCID: PMC11087497 DOI: 10.1038/s41467-024-48062-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 04/19/2024] [Indexed: 05/12/2024] Open
Abstract
In clinical oncology, many diagnostic tasks rely on the identification of cells in histopathology images. While supervised machine learning techniques necessitate the need for labels, providing manual cell annotations is time-consuming. In this paper, we propose a self-supervised framework (enVironment-aware cOntrastive cell represenTation learning: VOLTA) for cell representation learning in histopathology images using a technique that accounts for the cell's mutual relationship with its environment. We subject our model to extensive experiments on data collected from multiple institutions comprising over 800,000 cells and six cancer types. To showcase the potential of our proposed framework, we apply VOLTA to ovarian and endometrial cancers and demonstrate that our cell representations can be utilized to identify the known histotypes of ovarian cancer and provide insights that link histopathology and molecular subtypes of endometrial cancer. Unlike supervised models, we provide a framework that can empower discoveries without any annotation data, even in situations where sample sizes are limited.
Collapse
Affiliation(s)
- Ramin Nakhli
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Katherine Rich
- Bioinformatics Graduate Program, University of British Columbia, Vancouver, Canada
| | - Allen Zhang
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Amirali Darbandsari
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Elahe Shenasa
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Amir Hadjifaradji
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Sidney Thiessen
- Deeley Research Centre, BC Cancer Agency, Victoria, BC, Canada
| | - Katy Milne
- Deeley Research Centre, BC Cancer Agency, Victoria, BC, Canada
| | - Steven J M Jones
- Canada's Michael Smith Genome Sciences Centre, BC Cancer Research Institute, Vancouver, Canada
- Department of Medical Genetics, University of British Columbia, Vancouver, Canada
| | - Jessica N McAlpine
- Department of Obstetrics and Gynecology, University of British Columbia, Vancouver, BC, Canada
| | - Brad H Nelson
- Deeley Research Centre, BC Cancer Agency, Victoria, BC, Canada
| | - C Blake Gilks
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Hossein Farahani
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Ali Bashashati
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.
- Canada's Michael Smith Genome Sciences Centre, BC Cancer Research Institute, Vancouver, Canada.
| |
Collapse
|
4
|
Bui DC, Song B, Kim K, Kwak JT. DAX-Net: A dual-branch dual-task adaptive cross-weight feature fusion network for robust multi-class cancer classification in pathology images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 248:108112. [PMID: 38479146 DOI: 10.1016/j.cmpb.2024.108112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 02/15/2024] [Accepted: 03/01/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND AND OBJECTIVE Multi-class cancer classification has been extensively studied in digital and computational pathology due to its importance in clinical decision-making. Numerous computational tools have been proposed for various types of cancer classification. Many of them are built based on convolutional neural networks. Recently, Transformer-style networks have shown to be effective for cancer classification. Herein, we present a hybrid design that leverages both convolutional neural networks and transformer architecture to obtain superior performance in cancer classification. METHODS We propose a dual-branch dual-task adaptive cross-weight feature fusion network, called DAX-Net, which exploits heterogeneous feature representations from the convolutional neural network and Transformer network, adaptively combines them to boost their representation power, and conducts cancer classification as categorical classification and ordinal classification. For an efficient and effective optimization of the proposed model, we introduce two loss functions that are tailored to the two classification tasks. RESULTS To evaluate the proposed method, we employed colorectal and prostate cancer datasets, of which each contains both in-domain and out-of-domain test sets. For colorectal cancer, the proposed method obtained an accuracy of 88.4%, a quadratic kappa score of 0.945, and an F1 score of 0.831 for the in-domain test set, and 84.4%, 0.910, and 0.768 for the out-of-domain test set. For prostate cancer, it achieved an accuracy of 71.6%, a kappa score of 0.635, and an F1 score of 0.655 for the in-domain test set, 79.2% accuracy, 0.721 kappa score, and 0.686 F1 score for the first out-of-domain test set, and 58.1% accuracy, 0.564 kappa score, and 0.493 F1 score for the second out-of-domain test set. It is worth noting that the performance of the proposed method outperformed other competitors by significant margins, in particular, with respect to the out-of-domain test sets. CONCLUSIONS The experimental results demonstrate that the proposed method is not only accurate but also robust to varying conditions of the test sets in comparison to several, related methods. These results suggest that the proposed method can facilitate automated cancer classification in various clinical settings.
Collapse
Affiliation(s)
- Doanh C Bui
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Boram Song
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Kyungeun Kim
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Jin Tae Kwak
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea.
| |
Collapse
|
5
|
Aubreville M, Stathonikos N, Donovan TA, Klopfleisch R, Ammeling J, Ganz J, Wilm F, Veta M, Jabari S, Eckstein M, Annuscheit J, Krumnow C, Bozaba E, Çayır S, Gu H, Chen X'A, Jahanifar M, Shephard A, Kondo S, Kasai S, Kotte S, Saipradeep VG, Lafarge MW, Koelzer VH, Wang Z, Zhang Y, Yang S, Wang X, Breininger K, Bertram CA. Domain generalization across tumor types, laboratories, and species - Insights from the 2022 edition of the Mitosis Domain Generalization Challenge. Med Image Anal 2024; 94:103155. [PMID: 38537415 DOI: 10.1016/j.media.2024.103155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 01/19/2024] [Accepted: 03/20/2024] [Indexed: 04/16/2024]
Abstract
Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022). The challenge provided annotated histologic tumor images from six different domains and evaluated the algorithmic approaches for mitotic figure detection provided by nine challenge participants on ten independent domains. Ground truth for mitotic figure detection was established in two ways: a three-expert majority vote and an independent, immunohistochemistry-assisted set of labels. This work represents an overview of the challenge tasks, the algorithmic strategies employed by the participants, and potential factors contributing to their success. With an F1 score of 0.764 for the top-performing team, we summarize that domain generalization across various tumor domains is possible with today's deep learning-based recognition pipelines. However, we also found that domain characteristics not present in the training set (feline as new species, spindle cell shape as new morphology and a new scanner) led to small but significant decreases in performance. When assessed against the immunohistochemistry-assisted reference standard, all methods resulted in reduced recall scores, with only minor changes in the order of participants in the ranking.
Collapse
Affiliation(s)
| | | | - Taryn A Donovan
- Department of Anatomic Pathology, The Schwarzman Animal Medical Center, NY, USA
| | - Robert Klopfleisch
- Institute of Veterinary Pathology, Freie Universität Berlin, Berlin, Germany
| | | | - Jonathan Ganz
- Technische Hochschule Ingolstadt, Ingolstadt, Germany
| | - Frauke Wilm
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Mitko Veta
- Computational Pathology Group, Radboud UMC Nijmegen, The Netherlands
| | - Samir Jabari
- Institute of Neuropathology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Markus Eckstein
- Institute of Pathology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nünberg, Erlangen, Germany
| | | | | | - Engin Bozaba
- Artificial Intelligence Research Team, Virasoft Corporation, NY, USA
| | - Sercan Çayır
- Artificial Intelligence Research Team, Virasoft Corporation, NY, USA
| | - Hongyan Gu
- University of California, Los Angeles, USA
| | | | | | | | | | - Satoshi Kasai
- Niigata University of Health and Welfare, Niigata, Japan
| | - Sujatha Kotte
- TCS Research, Tata Consultancy Services Ltd, Hyderabad, India
| | - V G Saipradeep
- TCS Research, Tata Consultancy Services Ltd, Hyderabad, India
| | - Maxime W Lafarge
- Department of Pathology and Molecular Pathology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Viktor H Koelzer
- Department of Pathology and Molecular Pathology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Ziyue Wang
- Harbin Institute of Technology, Shenzhen, China
| | | | - Sen Yang
- College of Biomedical Engineering, Sichuan University, Chengdu, China
| | - Xiyue Wang
- Department of Radiation Oncology, Stanford University School of Medicine, Palo Alto, USA
| | - Katharina Breininger
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Christof A Bertram
- Institute of Pathology, University of Veterinary Medicine, Vienna, Austria
| |
Collapse
|
6
|
Leon-Ferre RA, Carter JM, Zahrieh D, Sinnwell JP, Salgado R, Suman VJ, Hillman DW, Boughey JC, Kalari KR, Couch FJ, Ingle JN, Balkenhol M, Ciompi F, van der Laak J, Goetz MP. Automated mitotic spindle hotspot counts are highly associated with clinical outcomes in systemically untreated early-stage triple-negative breast cancer. NPJ Breast Cancer 2024; 10:25. [PMID: 38553444 PMCID: PMC10980681 DOI: 10.1038/s41523-024-00629-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 03/08/2024] [Indexed: 04/02/2024] Open
Abstract
Operable triple-negative breast cancer (TNBC) has a higher risk of recurrence and death compared to other subtypes. Tumor size and nodal status are the primary clinical factors used to guide systemic treatment, while biomarkers of proliferation have not demonstrated value. Recent studies suggest that subsets of TNBC have a favorable prognosis, even without systemic therapy. We evaluated the association of fully automated mitotic spindle hotspot (AMSH) counts with recurrence-free (RFS) and overall survival (OS) in two separate cohorts of patients with early-stage TNBC who did not receive systemic therapy. AMSH counts were obtained from areas with the highest mitotic density in digitized whole slide images processed with a convolutional neural network trained to detect mitoses. In 140 patients from the Mayo Clinic TNBC cohort, AMSH counts were significantly associated with RFS and OS in a multivariable model controlling for nodal status, tumor size, and tumor-infiltrating lymphocytes (TILs) (p < 0.0001). For every 10-point increase in AMSH counts, there was a 16% increase in the risk of an RFS event (HR 1.16, 95% CI 1.08-1.25), and a 7% increase in the risk of death (HR 1.07, 95% CI 1.00-1.14). We corroborated these findings in a separate cohort of systemically untreated TNBC patients from Radboud UMC in the Netherlands. Our findings suggest that AMSH counts offer valuable prognostic information in patients with early-stage TNBC who did not receive systemic therapy, independent of tumor size, nodal status, and TILs. If further validated, AMSH counts could help inform future systemic therapy de-escalation strategies.
Collapse
Affiliation(s)
| | | | | | | | - Roberto Salgado
- GZA-ZNA-Hospitals, Antwerp, Belgium
- Peter Mac Callum Cancer Centre, Melbourne, Australia
| | | | | | | | | | | | | | | | | | - Jeroen van der Laak
- Radboud University Medical Center, Nijmegen, Netherlands
- Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | | |
Collapse
|
7
|
Bertram CA, Donovan TA, Bartel A. Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in canine tumors. Vet Pathol 2024:3009858241239565. [PMID: 38533804 DOI: 10.1177/03009858241239565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/28/2024]
Abstract
One of the most relevant prognostic indices for tumors is cellular proliferation, which is most commonly measured by the mitotic activity in routine tumor sections. The goal of this systematic review was to analyze the methods and prognostic relevance of histologically measuring mitotic activity that have been reported for canine tumors in the literature. A total of 137 articles that correlated the mitotic activity in canine tumors with patient outcome were identified through a systematic (PubMed and Scopus) and nonsystematic (Google Scholar) literature search and eligibility screening process. Mitotic activity methods encompassed the mitotic count (MC, number of mitotic figures per tumor area) in 126 studies, presumably the MC (method not specified) in 6 studies, and the mitotic index (MI, number of mitotic figures per number of tumor cells) in 5 studies. A particularly high risk of bias was identified based on the available details of the MC methods and statistical analyses, which often did not quantify the prognostic discriminative ability of the MC and only reported P values. A significant association of the MC with survival was found in 72 of 109 (66%) studies. However, survival was evaluated by at least 3 studies in only 7 tumor types/groups, of which a prognostic relevance is apparent for mast cell tumors of the skin, cutaneous melanoma, and soft tissue tumor of the skin and subcutis. None of the studies using the MI found a prognostic relevance. This review highlights the need for more studies with standardized methods and appropriate analysis of the discriminative ability to prove the prognostic value of the MC and MI in various tumor types. Future studies are needed to evaluate the influence of the performance of individual pathologists on the appropriateness of prognostic thresholds and investigate methods to improve interobserver reproducibility.
Collapse
|
8
|
Bertram CA, Donovan TA, Bartel A. Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in feline tumors. Vet Pathol 2024:3009858241239566. [PMID: 38533803 DOI: 10.1177/03009858241239566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/28/2024]
Abstract
Increased proliferation is a driver of tumorigenesis, and quantification of mitotic activity is a standard task for prognostication. This systematic review is an analysis of all available references on mitotic activity in feline tumors to provide an overview of the assessment methods and prognostic value. A systematic literature search in PubMed and Scopus and a nonsystematic search in Google Scholar were conducted. All articles on feline tumors that correlated mitotic activity with patient outcome were identified. Data analysis revealed that of the 42 eligible articles, mitotic count (MC, mitotic figures/tumor area) was evaluated in 39 studies, and mitotic index (MI, mitotic figures/tumor cells) in 3 studies. The risk of bias was considered high for most studies (26/42, 62%) based on small study populations, insufficient details of the MC/MI methods, and lack of statistical measures for diagnostic accuracy or effect on outcome. The MC/MI methods varied between studies. A significant association of MC with survival was determined in 20 of 28 (71%) studies (10 studies evaluated other outcome metrics or provided individual patient data), while 1 study found an inverse effect. Three tumor types had at least 4 studies, and a prognostic association with survival was found in 5 of 6 studies on mast cell tumors, 5 of 5 on mammary tumors, and 3 of 4 on soft-tissue sarcomas. MI was shown to correlate with survival for mammary tumors by 2 research groups; however, comparisons to MC were not conducted. Further studies with standardized mitotic activity methods and appropriate statistical analysis for discriminant ability of patient outcome are needed to infer the prognostic value of MC and MI.
Collapse
|
9
|
Fernandez-Martín C, Silva-Rodriguez J, Kiraz U, Morales S, Janssen EAM, Naranjo V. Uninformed Teacher-Student for hard-samples distillation in weakly supervised mitosis localization. Comput Med Imaging Graph 2024; 112:102328. [PMID: 38244279 DOI: 10.1016/j.compmedimag.2024.102328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 11/02/2023] [Accepted: 12/12/2023] [Indexed: 01/22/2024]
Abstract
BACKGROUND AND OBJECTIVE Mitotic activity is a crucial biomarker for diagnosing and predicting outcomes for different types of cancers, particularly breast cancer. However, manual mitosis counting is challenging and time-consuming for pathologists, with moderate reproducibility due to biopsy slide size, low mitotic cell density, and pattern heterogeneity. In recent years, deep learning methods based on convolutional neural networks (CNNs) have been proposed to address these limitations. Nonetheless, these methods have been hampered by the available data labels, which usually consist only of the centroids of mitosis, and by the incoming noise from annotated hard negatives. As a result, complex algorithms with multiple stages are often required to refine the labels at the pixel level and reduce the number of false positives. METHODS This article presents a novel weakly supervised approach for mitosis detection that utilizes only image-level labels on histological hematoxylin and eosin (H&E) images, avoiding the need for complex labeling scenarios. Also, an Uninformed Teacher-Student (UTS) pipeline is introduced to detect and distill hard samples by comparing weakly supervised localizations and the annotated centroids, using strong augmentations to enhance uncertainty. Additionally, an automatic proliferation score is proposed that mimicks the pathologist-annotated mitotic activity index (MAI). The proposed approach is evaluated on three publicly available datasets for mitosis detection on breast histology samples, and two datasets for mitotic activity counting in whole-slide images. RESULTS The proposed framework achieves competitive performance with relevant prior literature in all the datasets used for evaluation without explicitly using the mitosis location information during training. This approach challenges previous methods that rely on strong mitosis location information and multiple stages to refine false positives. Furthermore, the proposed pipeline for hard-sample distillation demonstrates promising dataset-specific improvements. Concretely, when the annotation has not been thoroughly refined by multiple pathologists, the UTS model offers improvements of up to ∼4% in mitosis localization, thanks to the detection and distillation of uncertain cases. Concerning the mitosis counting task, the proposed automatic proliferation score shows a moderate positive correlation with the MAI annotated by pathologists at the biopsy level on two external datasets. CONCLUSIONS The proposed Uninformed Teacher-Student pipeline leverages strong augmentations to distill uncertain samples and measure dissimilarities between predicted and annotated mitosis. Results demonstrate the feasibility of the weakly supervised approach and highlight its potential as an objective evaluation tool for tumor proliferation.
Collapse
Affiliation(s)
- Claudio Fernandez-Martín
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain.
| | | | - Umay Kiraz
- Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Stavanger, Norway; Department of Pathology, Stavanger University Hospital, Stavanger, Norway
| | - Sandra Morales
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain
| | - Emiel A M Janssen
- Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Stavanger, Norway; Department of Pathology, Stavanger University Hospital, Stavanger, Norway
| | - Valery Naranjo
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain
| |
Collapse
|
10
|
Ibrahim A, Jahanifar M, Wahab N, Toss MS, Makhlouf S, Atallah N, Lashen AG, Katayama A, Graham S, Bilal M, Bhalerao A, Ahmed Raza SE, Snead D, Minhas F, Rajpoot N, Rakha E. Artificial Intelligence-Based Mitosis Scoring in Breast Cancer: Clinical Application. Mod Pathol 2024; 37:100416. [PMID: 38154653 DOI: 10.1016/j.modpat.2023.100416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 10/27/2023] [Accepted: 12/14/2023] [Indexed: 12/30/2023]
Abstract
In recent years, artificial intelligence (AI) has demonstrated exceptional performance in mitosis identification and quantification. However, the implementation of AI in clinical practice needs to be evaluated against the existing methods. This study is aimed at assessing the optimal method of using AI-based mitotic figure scoring in breast cancer (BC). We utilized whole slide images from a large cohort of BC with extended follow-up comprising a discovery (n = 1715) and a validation (n = 859) set (Nottingham cohort). The Cancer Genome Atlas of breast invasive carcinoma (TCGA-BRCA) cohort (n = 757) was used as an external test set. Employing automated mitosis detection, the mitotic count was assessed using 3 different methods, the mitotic count per tumor area (MCT; calculated by dividing the number of mitotic figures by the total tumor area), the mitotic index (MI; defined as the average number of mitotic figures per 1000 malignant cells), and the mitotic activity index (MAI; defined as the number of mitotic figures in 3 mm2 area within the mitotic hotspot). These automated metrics were evaluated and compared based on their correlation with the well-established visual scoring method of the Nottingham grading system and Ki67 score, clinicopathologic parameters, and patient outcomes. AI-based mitotic scores derived from the 3 methods (MCT, MI, and MAI) were significantly correlated with the clinicopathologic characteristics and patient survival (P < .001). However, the mitotic counts and the derived cutoffs varied significantly between the 3 methods. Only MAI and MCT were positively correlated with the gold standard visual scoring method used in Nottingham grading system (r = 0.8 and r = 0.7, respectively) and Ki67 scores (r = 0.69 and r = 0.55, respectively), and MAI was the only independent predictor of survival (P < .05) in multivariate Cox regression analysis. For clinical applications, the optimum method of scoring mitosis using AI needs to be considered. MAI can provide reliable and reproducible results and can accurately quantify mitotic figures in BC.
Collapse
Affiliation(s)
- Asmaa Ibrahim
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom; Department of Pathology, Faculty of Medicine, Suez Canal University, Egypt
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Noorul Wahab
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Michael S Toss
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom; Histopathology Department, Royal Hallamshire Hospital, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Nehal Atallah
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Ayat G Lashen
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Ayaka Katayama
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Simon Graham
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Mohsin Bilal
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - David Snead
- Cellular Pathology, University Hospitals Coventry and Warwickshire NHS Trust, United Kingdom
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, United Kingdom.
| | - Emad Rakha
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom; Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom; Pathology Department, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
11
|
Rai T, Morisi A, Bacci B, Bacon NJ, Dark MJ, Aboellail T, Thomas SA, La Ragione RM, Wells K. Keeping Pathologists in the Loop and an Adaptive F1-Score Threshold Method for Mitosis Detection in Canine Perivascular Wall Tumours. Cancers (Basel) 2024; 16:644. [PMID: 38339394 PMCID: PMC10854568 DOI: 10.3390/cancers16030644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 01/17/2024] [Accepted: 01/29/2024] [Indexed: 02/12/2024] Open
Abstract
Performing a mitosis count (MC) is the diagnostic task of histologically grading canine Soft Tissue Sarcoma (cSTS). However, mitosis count is subject to inter- and intra-observer variability. Deep learning models can offer a standardisation in the process of MC used to histologically grade canine Soft Tissue Sarcomas. Subsequently, the focus of this study was mitosis detection in canine Perivascular Wall Tumours (cPWTs). Generating mitosis annotations is a long and arduous process open to inter-observer variability. Therefore, by keeping pathologists in the loop, a two-step annotation process was performed where a pre-trained Faster R-CNN model was trained on initial annotations provided by veterinary pathologists. The pathologists reviewed the output false positive mitosis candidates and determined whether these were overlooked candidates, thus updating the dataset. Faster R-CNN was then trained on this updated dataset. An optimal decision threshold was applied to maximise the F1-score predetermined using the validation set and produced our best F1-score of 0.75, which is competitive with the state of the art in the canine mitosis domain.
Collapse
Affiliation(s)
- Taranpreet Rai
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, UK;
- Surrey DataHub, University of Surrey, Guildford GU2 7AL, UK
| | - Ambra Morisi
- School of Veterinary Medicine, University of Surrey, Guildford GU2 7AL, UK; (A.M.); (R.M.L.R.)
| | - Barbara Bacci
- Department of Veterinary Medical Sciences, University of Bologna, 40126 Bologna, Italy;
| | | | - Michael J. Dark
- Department of Comparative, Diagnostic and Population Medicine, College of Veterinary Medicine, University of Florida, Gainesville, FL 32611, USA;
| | - Tawfik Aboellail
- Department of Diagnostic Pathology and Pathobiology, Kansas State University, Manhattan, KS 66506, USA;
| | - Spencer A. Thomas
- Department of Computer Science, University of Surrey, Guildford GU2 7XH, UK;
- National Physical Laboratory, London TW11 0LW, UK
| | - Roberto M. La Ragione
- School of Veterinary Medicine, University of Surrey, Guildford GU2 7AL, UK; (A.M.); (R.M.L.R.)
- School of Biosciences, University of Surrey, Guildford GU2 7XH, UK
| | - Kevin Wells
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, UK;
- Surrey DataHub, University of Surrey, Guildford GU2 7AL, UK
| |
Collapse
|
12
|
Hirling D, Tasnadi E, Caicedo J, Caroprese MV, Sjögren R, Aubreville M, Koos K, Horvath P. Segmentation metric misinterpretations in bioimage analysis. Nat Methods 2024; 21:213-216. [PMID: 37500758 PMCID: PMC10864175 DOI: 10.1038/s41592-023-01942-8] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Accepted: 06/06/2023] [Indexed: 07/29/2023]
Abstract
Quantitative evaluation of image segmentation algorithms is crucial in the field of bioimage analysis. The most common assessment scores, however, are often misinterpreted and multiple definitions coexist with the same name. Here we present the ambiguities of evaluation metrics for segmentation algorithms and show how these misinterpretations can alter leaderboards of influential competitions. We also propose guidelines for how the currently existing problems could be tackled.
Collapse
Affiliation(s)
- Dominik Hirling
- Biological Research Centre, Eötvös Loránd Research Network (ELKH), Szeged, Hungary
- Doctoral School of Computer Science, University of Szeged, Szeged, Hungary
| | - Ervin Tasnadi
- Biological Research Centre, Eötvös Loránd Research Network (ELKH), Szeged, Hungary
- Doctoral School of Computer Science, University of Szeged, Szeged, Hungary
| | - Juan Caicedo
- Broad Institute of Harvard and MIT, Cambridge, MA, USA
| | | | - Rickard Sjögren
- Sartorius, Corporate Research, Umeå, Sweden
- CellVoyant Technologies Ltd, Bristol, UK
| | | | - Krisztian Koos
- Biological Research Centre, Eötvös Loránd Research Network (ELKH), Szeged, Hungary
| | - Peter Horvath
- Biological Research Centre, Eötvös Loránd Research Network (ELKH), Szeged, Hungary.
- Single-Cell Technologies Ltd, Szeged, Hungary.
- Institute for Molecular Medicine Finland (FIMM), University of Helsinki, Helsinki, Finland.
| |
Collapse
|
13
|
Fernandez G, Zeineh J, Prastawa M, Scott R, Madduri AS, Shtabsky A, Jaffer S, Feliz A, Veremis B, Mejias JC, Charytonowicz E, Gladoun N, Koll G, Cruz K, Malinowski D, Donovan MJ. Analytical Validation of the PreciseDx Digital Prognostic Breast Cancer Test in Early-Stage Breast Cancer. Clin Breast Cancer 2024; 24:93-102.e6. [PMID: 38114366 DOI: 10.1016/j.clbc.2023.10.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Revised: 10/19/2023] [Accepted: 10/29/2023] [Indexed: 12/21/2023]
Abstract
BACKGROUND PreciseDx Breast (PDxBr) is a digital test that predicts early-stage breast cancer recurrence within 6-years of diagnosis. MATERIALS AND METHODS Using hematoxylin and eosin-stained whole slide images of invasive breast cancer (IBC) and artificial intelligence-enabled morphology feature array, microanatomic features are generated. Morphometric attributes in combination with patient's age, tumor size, stage, and lymph node status predict disease free survival using a proprietary algorithm. Here, analytical validation of the automated annotation process and extracted histologic digital features of the PDxBr test, including impact of methodologic variability on the composite risk score is presented. Studies of precision, repeatability, reproducibility and interference were performed on morphology feature array-derived features. The final risk score was assessed over 20-days with 2-operators, 2-runs/day, and 2-replicates across 8-patients, allowing for calculation of within-run repeatability, between-run and within-laboratory reproducibility. RESULTS Analytical validation of features derived from whole slide images demonstrated a high degree of precision for tumor segmentation (0.98, 0.98), lymphocyte detection (0.91, 0.93), and mitotic figures (0.85, 0.84). Correlation of variation of the assay risk score for both reproducibility and repeatability were less than 2%, and interference from variation in hematoxylin and eosin staining or tumor thickness was not observed demonstrating assay robustness across standard histopathology preparations. CONCLUSION In summary, the analytical validation of the digital IBC risk assessment test demonstrated a strong performance across all features in the model and complimented the clinical validation of the assay previously shown to accurately predict recurrence within 6-years in early-stage invasive breast cancer patients.
Collapse
Affiliation(s)
- Gerardo Fernandez
- PreciseDx, New York, NY; Icahn School of Medicine at Mount Sinai, New York, NY
| | | | | | | | | | | | | | | | - Brandon Veremis
- PreciseDx, New York, NY; Icahn School of Medicine at Mount Sinai, New York, NY
| | | | | | - Nataliya Gladoun
- PreciseDx, New York, NY; Icahn School of Medicine at Mount Sinai, New York, NY
| | | | | | | | - Michael J Donovan
- PreciseDx, New York, NY; Icahn School of Medicine at Mount Sinai, New York, NY; University of Miami, Pathology, Miami, FL.
| |
Collapse
|
14
|
Schumann Y, Dottermusch M, Schweizer L, Krech M, Lempertz T, Schüller U, Neumann P, Neumann JE. Morphology-based molecular classification of spinal cord ependymomas using deep neural networks. Brain Pathol 2024:e13239. [PMID: 38205683 DOI: 10.1111/bpa.13239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Accepted: 12/31/2023] [Indexed: 01/12/2024] Open
Abstract
Based on DNA-methylation, ependymomas growing in the spinal cord comprise two major molecular types termed spinal (SP-EPN) and myxopapillary ependymomas (MPE(-A/B)), which differ with respect to their clinical features and prognosis. Due to the existing discrepancy between histomorphogical diagnoses and classification using methylation data, we asked whether deep neural networks can predict the DNA methylation class of spinal cord ependymomas from hematoxylin and eosin stained whole-slide images. Using explainable AI, we further aimed to prospectively improve the consistency of histology-based diagnoses with DNA methylation profiling by identifying and quantifying distinct morphological patterns of these molecular ependymoma types. We assembled a case series of 139 molecularly characterized spinal cord ependymomas (nMPE = 84, nSP-EPN = 55). Self-supervised and weakly-supervised neural networks were used for classification. We employed attention analysis and supervised machine-learning methods for the discovery and quantification of morphological features and their correlation to the diagnoses of experienced neuropathologists. Our best performing model predicted the DNA methylation class with 98% test accuracy and used self-supervised learning to outperform pretrained encoder-networks (86% test accuracy). In contrast, the diagnoses of neuropathologists matched the DNA methylation class in only 83% of cases. Domain-adaptation techniques improved model generalization to an external validation cohort by up to 22%. Statistically significant morphological features were identified per molecular type and quantitatively correlated to human diagnoses. The approach was extended to recently defined subtypes of myxopapillary ependymomas (MPE-(A/B), 80% test accuracy). In summary, we demonstrated the accurate prediction of the DNA methylation class of spinal cord ependymomas (SP-EPN, MPE(-A/B)) using hematoxylin and eosin stained whole-slide images. Our approach may prospectively serve as a supplementary resource for integrated diagnostics and may even help to establish a standardized, high-quality level of histology-based diagnostics across institutions-in particular in low-income countries, where expensive DNA-methylation analyses may not be readily available.
Collapse
Affiliation(s)
- Yannis Schumann
- Chair for High Performance Computing, Helmut-Schmidt-University Hamburg, Hamburg, Germany
| | - Matthias Dottermusch
- Center for Molecular Neurobiology (ZMNH), University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
- Institute of Neuropathology, UKE, Hamburg, Germany
| | - Leonille Schweizer
- Institute of Neurology (Edinger Institute), University Hospital Frankfurt, Goethe University, Frankfurt am Main, Germany
- German Cancer Consortium (DKTK), Partner Site Frankfurt/Mainz, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Frankfurt Cancer Institute (FCI), Frankfurt am Main, Germany
| | - Maja Krech
- Institute for Neuropathology, Charité Berlin, Berlin, Germany
| | - Tasja Lempertz
- Center for Molecular Neurobiology (ZMNH), University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
| | - Ulrich Schüller
- Institute of Neuropathology, UKE, Hamburg, Germany
- Research Institute Children's Cancer Center Hamburg, UKE, Hamburg, Germany
- Department of Pediatric Hematology and Oncology, UKE, Hamburg, Germany
| | - Philipp Neumann
- Chair for High Performance Computing, Helmut-Schmidt-University Hamburg, Hamburg, Germany
| | - Julia E Neumann
- Center for Molecular Neurobiology (ZMNH), University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
- Institute of Neuropathology, UKE, Hamburg, Germany
| |
Collapse
|
15
|
Gu H, Yang C, Al-Kharouf I, Magaki S, Lakis N, Williams CK, Alrosan SM, Onstott EK, Yan W, Khanlou N, Cobos I, Zhang XR, Zarrin-Khameh N, Vinters HV, Chen XA, Haeri M. Enhancing mitosis quantification and detection in meningiomas with computational digital pathology. Acta Neuropathol Commun 2024; 12:7. [PMID: 38212848 PMCID: PMC10782692 DOI: 10.1186/s40478-023-01707-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 12/10/2023] [Indexed: 01/13/2024] Open
Abstract
Mitosis is a critical criterion for meningioma grading. However, pathologists' assessment of mitoses is subject to significant inter-observer variation due to challenges in locating mitosis hotspots and accurately detecting mitotic figures. To address this issue, we leverage digital pathology and propose a computational strategy to enhance pathologists' mitosis assessment. The strategy has two components: (1) A depth-first search algorithm that quantifies the mathematically maximum mitotic count in 10 consecutive high-power fields, which can enhance the preciseness, especially in cases with borderline mitotic count. (2) Implementing a collaborative sphere to group a set of pathologists to detect mitoses under each high-power field, which can mitigate subjective random errors in mitosis detection originating from individual detection errors. By depth-first search algorithm (1) , we analyzed 19 meningioma slides and discovered that the proposed algorithm upgraded two borderline cases verified at consensus conferences. This improvement is attributed to the algorithm's ability to quantify the mitotic count more comprehensively compared to other conventional methods of counting mitoses. In implementing a collaborative sphere (2) , we evaluated the correctness of mitosis detection from grouped pathologists and/or pathology residents, where each member of the group annotated a set of 48 high-power field images for mitotic figures independently. We report that groups with sizes of three can achieve an average precision of 0.897 and sensitivity of 0.699 in mitosis detection, which is higher than an average pathologist in this study (precision: 0.750, sensitivity: 0.667). The proposed computational strategy can be integrated with artificial intelligence workflow, which envisions the future of achieving a rapid and robust mitosis assessment by interactive assisting algorithms that can ultimately benefit patient management.
Collapse
Affiliation(s)
- Hongyan Gu
- Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, 90095, USA
| | - Chunxu Yang
- Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, 90095, USA
| | - Issa Al-Kharouf
- Pathology and Laboratory Medicine, The University of Kansas Medical Center, Kansas City, KS, 66160, USA
| | - Shino Magaki
- Pathology and Laboratory Medicine, UCLA David Geffen School of Medicine, Los Angeles, CA, 90095, USA
| | - Nelli Lakis
- Pathology and Laboratory Medicine, The University of Kansas Medical Center, Kansas City, KS, 66160, USA
| | - Christopher Kazu Williams
- Pathology and Laboratory Medicine, UCLA David Geffen School of Medicine, Los Angeles, CA, 90095, USA
| | - Sallam Mohammad Alrosan
- Pathology and Laboratory Medicine, The University of Kansas Medical Center, Kansas City, KS, 66160, USA
| | - Ellie Kate Onstott
- Pathology and Laboratory Medicine, The University of Kansas Medical Center, Kansas City, KS, 66160, USA
| | - Wenzhong Yan
- Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, 90095, USA
| | - Negar Khanlou
- Pathology and Laboratory Medicine, UCLA David Geffen School of Medicine, Los Angeles, CA, 90095, USA
| | - Inma Cobos
- Department of Pathology, Stanford Medical School, Stanford, CA, 94305, USA
| | | | | | - Harry V Vinters
- Pathology and Laboratory Medicine, UCLA David Geffen School of Medicine, Los Angeles, CA, 90095, USA
| | - Xiang Anthony Chen
- Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, 90095, USA.
| | - Mohammad Haeri
- Pathology and Laboratory Medicine, The University of Kansas Medical Center, Kansas City, KS, 66160, USA.
| |
Collapse
|
16
|
Dawood M, Vu QD, Young LS, Branson K, Jones L, Rajpoot N, Minhas FUAA. Cancer drug sensitivity prediction from routine histology images. NPJ Precis Oncol 2024; 8:5. [PMID: 38184744 PMCID: PMC10771481 DOI: 10.1038/s41698-023-00491-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 12/08/2023] [Indexed: 01/08/2024] Open
Abstract
Drug sensitivity prediction models can aid in personalising cancer therapy, biomarker discovery, and drug design. Such models require survival data from randomised controlled trials which can be time consuming and expensive. In this proof-of-concept study, we demonstrate for the first time that deep learning can link histological patterns in whole slide images (WSIs) of Haematoxylin & Eosin (H&E) stained breast cancer sections with drug sensitivities inferred from cell lines. We employ patient-wise drug sensitivities imputed from gene expression-based mapping of drug effects on cancer cell lines to train a deep learning model that predicts patients' sensitivity to multiple drugs from WSIs. We show that it is possible to use routine WSIs to predict the drug sensitivity profile of a cancer patient for a number of approved and experimental drugs. We also show that the proposed approach can identify cellular and histological patterns associated with drug sensitivity profiles of cancer patients.
Collapse
Affiliation(s)
- Muhammad Dawood
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK.
| | - Quoc Dang Vu
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Lawrence S Young
- Warwick Medical School, University of Warwick, Coventry, UK
- Cancer Research Centre, University of Warwick, Coventry, UK
| | - Kim Branson
- Artificial Intelligence & Machine Learning, GlaxoSmithKline, San Francisco, CA, USA
| | - Louise Jones
- Barts Cancer Institute, Queen Mary University of London, London, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
- Cancer Research Centre, University of Warwick, Coventry, UK
- The Alan Turing Institute, London, UK
| | - Fayyaz Ul Amir Afsar Minhas
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
- Cancer Research Centre, University of Warwick, Coventry, UK
| |
Collapse
|
17
|
Farooq H, Saleem S, Aleem I, Iftikhar A, Sheikh UN, Naveed H. Toward interpretable and generalized mitosis detection in digital pathology using deep learning. Digit Health 2024; 10:20552076241255471. [PMID: 38778869 PMCID: PMC11110526 DOI: 10.1177/20552076241255471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 05/01/2024] [Indexed: 05/25/2024] Open
Abstract
Objective The mitotic activity index is an important prognostic factor in the diagnosis of cancer. The task of mitosis detection is difficult as the nuclei are microscopic in size and partially labeled, and there are many more non-mitotic nuclei compared to mitotic ones. In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method to tackle these challenges. Methods Our proposed methodology is inspired from recent research on deep learning and an extensive analysis on the dataset and training pipeline. We first used the MiDoG'22 dataset for training, validation, and testing. We then tested the methodology without fine-tuning on the TUPAC'16 dataset and on a real-time case from Shaukat Khanum Memorial Cancer Hospital and Research Centre. Results Our methodology has shown promising results both quantitatively and qualitatively. Quantitatively, our methodology achieved an F1-score of 0.87 on the MiDoG'22 dataset and an F1-score of 0.83 on the TUPAC dataset. Qualitatively, our methodology is generalizable and interpretable across various datasets and clinical settings. Conclusion In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method that can accurately predict mitotic nuclei. We illustrate the accuracy, generalizability, and interpretability of our approach across various datasets and clinical settings. Our methodology can speed up the adoption of computer-aided digital pathology in clinical settings.
Collapse
Affiliation(s)
- Hasan Farooq
- Computational Biology Research Lab, National University of Computer & Emerging Sciences, Islamabad, Pakistan
| | - Saira Saleem
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Iffat Aleem
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Ayesha Iftikhar
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Umer Nisar Sheikh
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Hammad Naveed
- Computational Biology Research Lab, National University of Computer & Emerging Sciences, Islamabad, Pakistan
| |
Collapse
|
18
|
Aubreville M, Wilm F, Stathonikos N, Breininger K, Donovan TA, Jabari S, Veta M, Ganz J, Ammeling J, van Diest PJ, Klopfleisch R, Bertram CA. A comprehensive multi-domain dataset for mitotic figure detection. Sci Data 2023; 10:484. [PMID: 37491536 PMCID: PMC10368709 DOI: 10.1038/s41597-023-02327-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 06/22/2023] [Indexed: 07/27/2023] Open
Abstract
The prognostic value of mitotic figures in tumor tissue is well-established for many tumor types and automating this task is of high research interest. However, especially deep learning-based methods face performance deterioration in the presence of domain shifts, which may arise from different tumor types, slide preparation and digitization devices. We introduce the MIDOG++ dataset, an extension of the MIDOG 2021 and 2022 challenge datasets. We provide region of interest images from 503 histological specimens of seven different tumor types with variable morphology with in total labels for 11,937 mitotic figures: breast carcinoma, lung carcinoma, lymphosarcoma, neuroendocrine tumor, cutaneous mast cell tumor, cutaneous melanoma, and (sub)cutaneous soft tissue sarcoma. The specimens were processed in several laboratories utilizing diverse scanners. We evaluated the extent of the domain shift by using state-of-the-art approaches, observing notable differences in single-domain training. In a leave-one-domain-out setting, generalizability improved considerably. This mitotic figure dataset is the first that incorporates a wide domain shift based on different tumor types, laboratories, whole slide image scanners, and species.
Collapse
Affiliation(s)
| | - Frauke Wilm
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Nikolas Stathonikos
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Katharina Breininger
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | | - Samir Jabari
- Department of Neuropathology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Mitko Veta
- Medical Image Analysis Group, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Jonathan Ganz
- Technische Hochschule Ingolstadt, Ingolstadt, Germany
| | | | - Paul J van Diest
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Robert Klopfleisch
- Institute of Veterinary Pathology, Freie Universität Berlin, Berlin, Germany
| | - Christof A Bertram
- Institute of Pathology, University of Veterinary Medicine Vienna, Vienna, Austria
| |
Collapse
|
19
|
Bai B, Yang X, Li Y, Zhang Y, Pillar N, Ozcan A. Deep learning-enabled virtual histological staining of biological samples. LIGHT, SCIENCE & APPLICATIONS 2023; 12:57. [PMID: 36864032 PMCID: PMC9981740 DOI: 10.1038/s41377-023-01104-7] [Citation(s) in RCA: 33] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 02/10/2023] [Accepted: 02/14/2023] [Indexed: 06/18/2023]
Abstract
Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to as virtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.
Collapse
Affiliation(s)
- Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Xilin Yang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA.
| |
Collapse
|