1
|
Jahanifar M, Shephard A, Zamanitajeddin N, Graham S, Raza SEA, Minhas F, Rajpoot N. Mitosis detection, fast and slow: Robust and efficient detection of mitotic figures. Med Image Anal 2024; 94:103132. [PMID: 38442527 DOI: 10.1016/j.media.2024.103132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 02/28/2024] [Accepted: 03/01/2024] [Indexed: 03/07/2024]
Abstract
Counting of mitotic figures is a fundamental step in grading and prognostication of several cancers. However, manual mitosis counting is tedious and time-consuming. In addition, variation in the appearance of mitotic figures causes a high degree of discordance among pathologists. With advances in deep learning models, several automatic mitosis detection algorithms have been proposed but they are sensitive to domain shift often seen in histology images. We propose a robust and efficient two-stage mitosis detection framework, which comprises mitosis candidate segmentation (Detecting Fast) and candidate refinement (Detecting Slow) stages. The proposed candidate segmentation model, termed EUNet, is fast and accurate due to its architectural design. EUNet can precisely segment candidates at a lower resolution to considerably speed up candidate detection. Candidates are then refined using a deeper classifier network, EfficientNet-B7, in the second stage. We make sure both stages are robust against domain shift by incorporating domain generalization methods. We demonstrate state-of-the-art performance and generalizability of the proposed model on the three largest publicly available mitosis datasets, winning the two mitosis domain generalization challenge contests (MIDOG21 and MIDOG22). Finally, we showcase the utility of the proposed algorithm by processing the TCGA breast cancer cohort (1,124 whole-slide images) to generate and release a repository of more than 620K potential mitotic figures (not exhaustively validated).
Collapse
Affiliation(s)
- Mostafa Jahanifar
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK.
| | - Adam Shephard
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Neda Zamanitajeddin
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Simon Graham
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK; Histofy Ltd, Birmingham, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Fayyaz Minhas
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Nasir Rajpoot
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK; Histofy Ltd, Birmingham, UK.
| |
Collapse
|
2
|
Ibrahim A, Jahanifar M, Wahab N, Toss MS, Makhlouf S, Atallah N, Lashen AG, Katayama A, Graham S, Bilal M, Bhalerao A, Ahmed Raza SE, Snead D, Minhas F, Rajpoot N, Rakha E. Artificial Intelligence-Based Mitosis Scoring in Breast Cancer: Clinical Application. Mod Pathol 2024; 37:100416. [PMID: 38154653 DOI: 10.1016/j.modpat.2023.100416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 10/27/2023] [Accepted: 12/14/2023] [Indexed: 12/30/2023]
Abstract
In recent years, artificial intelligence (AI) has demonstrated exceptional performance in mitosis identification and quantification. However, the implementation of AI in clinical practice needs to be evaluated against the existing methods. This study is aimed at assessing the optimal method of using AI-based mitotic figure scoring in breast cancer (BC). We utilized whole slide images from a large cohort of BC with extended follow-up comprising a discovery (n = 1715) and a validation (n = 859) set (Nottingham cohort). The Cancer Genome Atlas of breast invasive carcinoma (TCGA-BRCA) cohort (n = 757) was used as an external test set. Employing automated mitosis detection, the mitotic count was assessed using 3 different methods, the mitotic count per tumor area (MCT; calculated by dividing the number of mitotic figures by the total tumor area), the mitotic index (MI; defined as the average number of mitotic figures per 1000 malignant cells), and the mitotic activity index (MAI; defined as the number of mitotic figures in 3 mm2 area within the mitotic hotspot). These automated metrics were evaluated and compared based on their correlation with the well-established visual scoring method of the Nottingham grading system and Ki67 score, clinicopathologic parameters, and patient outcomes. AI-based mitotic scores derived from the 3 methods (MCT, MI, and MAI) were significantly correlated with the clinicopathologic characteristics and patient survival (P < .001). However, the mitotic counts and the derived cutoffs varied significantly between the 3 methods. Only MAI and MCT were positively correlated with the gold standard visual scoring method used in Nottingham grading system (r = 0.8 and r = 0.7, respectively) and Ki67 scores (r = 0.69 and r = 0.55, respectively), and MAI was the only independent predictor of survival (P < .05) in multivariate Cox regression analysis. For clinical applications, the optimum method of scoring mitosis using AI needs to be considered. MAI can provide reliable and reproducible results and can accurately quantify mitotic figures in BC.
Collapse
Affiliation(s)
- Asmaa Ibrahim
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom; Department of Pathology, Faculty of Medicine, Suez Canal University, Egypt
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Noorul Wahab
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Michael S Toss
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom; Histopathology Department, Royal Hallamshire Hospital, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Nehal Atallah
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Ayat G Lashen
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Ayaka Katayama
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom
| | - Simon Graham
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Mohsin Bilal
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - David Snead
- Cellular Pathology, University Hospitals Coventry and Warwickshire NHS Trust, United Kingdom
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, United Kingdom
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, United Kingdom.
| | - Emad Rakha
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, United Kingdom; Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom; Pathology Department, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
3
|
Graham S, Vu QD, Jahanifar M, Weigert M, Schmidt U, Zhang W, Zhang J, Yang S, Xiang J, Wang X, Rumberger JL, Baumann E, Hirsch P, Liu L, Hong C, Aviles-Rivero AI, Jain A, Ahn H, Hong Y, Azzuni H, Xu M, Yaqub M, Blache MC, Piégu B, Vernay B, Scherr T, Böhland M, Löffler K, Li J, Ying W, Wang C, Snead D, Raza SEA, Minhas F, Rajpoot NM. CoNIC Challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting. Med Image Anal 2024; 92:103047. [PMID: 38157647 DOI: 10.1016/j.media.2023.103047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 09/19/2023] [Accepted: 11/29/2023] [Indexed: 01/03/2024]
Abstract
Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.
Collapse
Affiliation(s)
- Simon Graham
- Tissue Image Analytics Centre, University of Warwick, Coventry, United Kingdom; Histofy Ltd, Birmingham, United Kingdom.
| | - Quoc Dang Vu
- Tissue Image Analytics Centre, University of Warwick, Coventry, United Kingdom; Histofy Ltd, Birmingham, United Kingdom
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, University of Warwick, Coventry, United Kingdom
| | - Martin Weigert
- Institute of Bioengineering, School of Life Sciences, EPFL, Lausanne, Switzerland
| | | | - Wenhua Zhang
- The Department of Computer Science, The University of Hong Kong, Hong Kong
| | | | - Sen Yang
- College of Biomedical Engineering, Sichuan University, Chengdu, China
| | - Jinxi Xiang
- Department of Precision Instruments, Tsinghua University, Beijing, China
| | - Xiyue Wang
- College of Computer Science, Sichuan University, Chengdu, China
| | - Josef Lorenz Rumberger
- Max-Delbrueck-Center for Molecular Medicine in the Helmholtz Association, Berlin, Germany; Humboldt University of Berlin, Faculty of Mathematics and Natural Sciences, Berlin, Germany; Charité University Medicine, Berlin, Germany
| | | | - Peter Hirsch
- Max-Delbrueck-Center for Molecular Medicine in the Helmholtz Association, Berlin, Germany; Humboldt University of Berlin, Faculty of Mathematics and Natural Sciences, Berlin, Germany
| | - Lihao Liu
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, United Kingdom
| | - Chenyang Hong
- Department of Computer Science and Engineering, Chinese University of Hong Kong, Hong Kong
| | - Angelica I Aviles-Rivero
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, United Kingdom
| | - Ayushi Jain
- Softsensor.ai, Bridgewater, NJ, United States of America; PRR.ai, TX, United States of America
| | - Heeyoung Ahn
- Department of R&D Center, Arontier Co. Ltd, Seoul, Republic of Korea
| | - Yiyu Hong
- Department of R&D Center, Arontier Co. Ltd, Seoul, Republic of Korea
| | - Hussam Azzuni
- Computer Vision Department, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
| | - Min Xu
- Computer Vision Department, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
| | - Mohammad Yaqub
- Computer Vision Department, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
| | | | - Benoît Piégu
- CNRS, IFCE, INRAE, Université de Tours, PRC, 3780, Nouzilly, France
| | - Bertrand Vernay
- Institut de Génétique et de Biologie Moléculaire et Cellulaire, Illkirch, France; Centre National de la Recherche Scientifique, UMR7104, Illkirch, France; Institut National de la Santé et de la Recherche Médicale, INSERM, U1258, Illkirch, France; Université de Strasbourg, Strasbourg, France
| | - Tim Scherr
- Institute for Automation and Applied Informatics Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany
| | - Moritz Böhland
- Institute for Automation and Applied Informatics Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany
| | - Katharina Löffler
- Institute for Automation and Applied Informatics Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany
| | - Jiachen Li
- School of software engineering, South China University of Technology, Guangzhou, China
| | - Weiqin Ying
- School of software engineering, South China University of Technology, Guangzhou, China
| | - Chixin Wang
- School of software engineering, South China University of Technology, Guangzhou, China
| | - David Snead
- Histofy Ltd, Birmingham, United Kingdom; Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom; Division of Biomedical Sciences, Warwick Medical School, University of Warwick, Coventry, United Kingdom
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, University of Warwick, Coventry, United Kingdom
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, Coventry, United Kingdom
| | - Nasir M Rajpoot
- Tissue Image Analytics Centre, University of Warwick, Coventry, United Kingdom; Histofy Ltd, Birmingham, United Kingdom; Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| |
Collapse
|
4
|
Lu W, Lashen AG, Wahab N, Miligy IM, Jahanifar M, Toss M, Graham S, Bilal M, Bhalerao A, Atallah NM, Makhlouf S, Ibrahim AY, Snead D, Minhas F, Raza SEA, Rakha E, Rajpoot N. AI-based intra-tumor heterogeneity score of Ki67 expression as a prognostic marker for early-stage ER+/HER2- breast cancer. J Pathol Clin Res 2024; 10:e346. [PMID: 37873865 PMCID: PMC10766021 DOI: 10.1002/cjp2.346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 08/11/2023] [Accepted: 09/28/2023] [Indexed: 10/25/2023]
Abstract
Early-stage estrogen receptor positive and human epidermal growth factor receptor negative (ER+/HER2-) luminal breast cancer (BC) is quite heterogeneous and accounts for about 70% of all BCs. Ki67 is a proliferation marker that has a significant prognostic value in luminal BC despite the challenges in its assessment. There is increasing evidence that spatial colocalization, which measures the evenness of different types of cells, is clinically important in several types of cancer. However, reproducible quantification of intra-tumor spatial heterogeneity remains largely unexplored. We propose an automated pipeline for prognostication of luminal BC based on the analysis of spatial distribution of Ki67 expression in tumor cells using a large well-characterized cohort (n = 2,081). The proposed Ki67 colocalization (Ki67CL) score can stratify ER+/HER2- BC patients with high significance in terms of BC-specific survival (p < 0.00001) and distant metastasis-free survival (p = 0.0048). Ki67CL score is shown to be highly significant compared with the standard Ki67 index. In addition, we show that the proposed Ki67CL score can help identify luminal BC patients who can potentially benefit from adjuvant chemotherapy.
Collapse
Affiliation(s)
- Wenqi Lu
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Ayat G Lashen
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
- Department of Pathology, Faculty of MedicineMenoufia UniversityMenoufiaEgypt
| | - Noorul Wahab
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Islam M Miligy
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
- Department of Pathology, Faculty of MedicineMenoufia UniversityMenoufiaEgypt
| | - Mostafa Jahanifar
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Michael Toss
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
| | - Simon Graham
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Mohsin Bilal
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Abhir Bhalerao
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Nehal M Atallah
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
| | - Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
| | - Asmaa Y Ibrahim
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
| | - David Snead
- Department of PathologyUniversity Hospitals Coventry and Warwickshire NHS TrustCoventryUK
| | - Fayyaz Minhas
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Shan E Ahmed Raza
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Emad Rakha
- Academic Unit for Translational Medical Sciences, School of MedicineUniversity of NottinghamNottinghamUK
| | - Nasir Rajpoot
- Tissue Image Analytics (TIA) Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| |
Collapse
|
5
|
Deshpande S, Dawood M, Minhas F, Rajpoot N. SynCLay: Interactive synthesis of histology images from bespoke cellular layouts. Med Image Anal 2024; 91:102995. [PMID: 37898050 DOI: 10.1016/j.media.2023.102995] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 09/27/2023] [Accepted: 10/02/2023] [Indexed: 10/30/2023]
Abstract
Automated synthesis of histology images has several potential applications in computational pathology. However, no existing method can generate realistic tissue images with a bespoke cellular layout or user-defined histology parameters. In this work, we propose a novel framework called SynCLay (Synthesis from Cellular Layouts) that can construct realistic and high-quality histology images from user-defined cellular layouts along with annotated cellular boundaries. Tissue image generation based on bespoke cellular layouts through the proposed framework allows users to generate different histological patterns from arbitrary topological arrangement of different types of cells (e.g., neutrophils, lymphocytes, epithelial cells and others). SynCLay generated synthetic images can be helpful in studying the role of different types of cells present in the tumor microenvironment. Additionally, they can assist in balancing the distribution of cellular counts in tissue images for designing accurate cellular composition predictors by minimizing the effects of data imbalance. We train SynCLay in an adversarial manner and integrate a nuclear segmentation and classification model in its training to refine nuclear structures and generate nuclear masks in conjunction with synthetic images. During inference, we combine the model with another parametric model for generating colon images and associated cellular counts as annotations given the grade of differentiation and cellularities (cell densities) of different cells. We assess the generated images quantitatively using the Frechet Inception Distance and report on feedback from trained pathologists who assigned realism scores to a set of images generated by the framework. The average realism score across all pathologists for synthetic images was as high as that for the real images. Moreover, with the assistance from pathologists, we showcase the ability of the generated images to accurately differentiate between benign and malignant tumors, thus reinforcing their reliability. We demonstrate that the proposed framework can be used to add new cells to a tissue images and alter cellular positions. We also show that augmenting limited real data with the synthetic data generated by our framework can significantly boost prediction performance of the cellular composition prediction task. The implementation of the proposed SynCLay framework is available at https://github.com/Srijay/SynCLay-Framework.
Collapse
Affiliation(s)
- Srijay Deshpande
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, UK.
| | - Muhammad Dawood
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, UK; The Alan Turing Institute, London, UK; Department of Pathology, University Hospitals Coventry & Warwickshire, UK; Histofy Ltd, Birmingham, UK.
| |
Collapse
|
6
|
Wahab N, Toss M, Miligy IM, Jahanifar M, Atallah NM, Lu W, Graham S, Bilal M, Bhalerao A, Lashen AG, Makhlouf S, Ibrahim AY, Snead D, Minhas F, Raza SEA, Rakha E, Rajpoot N. AI-enabled routine H&E image based prognostic marker for early-stage luminal breast cancer. NPJ Precis Oncol 2023; 7:122. [PMID: 37968376 PMCID: PMC10651910 DOI: 10.1038/s41698-023-00472-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 10/24/2023] [Indexed: 11/17/2023] Open
Abstract
Breast cancer (BC) grade is a well-established subjective prognostic indicator of tumour aggressiveness. Tumour heterogeneity and subjective assessment result in high degree of variability among observers in BC grading. Here we propose an objective Haematoxylin & Eosin (H&E) image-based prognostic marker for early-stage luminal/Her2-negative BReAst CancEr that we term as the BRACE marker. The proposed BRACE marker is derived from AI based assessment of heterogeneity in BC at a detailed level using the power of deep learning. The prognostic ability of the marker is validated in two well-annotated cohorts (Cohort-A/Nottingham: n = 2122 and Cohort-B/Coventry: n = 311) on early-stage luminal/HER2-negative BC patients treated with endocrine therapy and with long-term follow-up. The BRACE marker is able to stratify patients for both distant metastasis free survival (p = 0.001, C-index: 0.73) and BC specific survival (p < 0.0001, C-index: 0.84) showing comparable prediction accuracy to Nottingham Prognostic Index and Magee scores, which are both derived from manual histopathological assessment, to identify luminal BC patients that may be likely to benefit from adjuvant chemotherapy.
Collapse
Affiliation(s)
- Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Michael Toss
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Histopathology, Sheffield Teaching Hospitals NHS Trust, Sheffield, UK
| | - Islam M Miligy
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Menoufia University, Shebin El-Koum, Egypt
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Nehal M Atallah
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Menoufia University, Shebin El-Koum, Egypt
| | - Wenqi Lu
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
- Histofy Ltd, Birmingham, UK
| | - Mohsin Bilal
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Ayat G Lashen
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Menoufia University, Shebin El-Koum, Egypt
| | - Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Assiut University, Asyut, Egypt
| | - Asmaa Y Ibrahim
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
| | - David Snead
- Histofy Ltd, Birmingham, UK
- The Alan Turing Institute, London, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Emad Rakha
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK.
- Histofy Ltd, Birmingham, UK.
- The Alan Turing Institute, London, UK.
| |
Collapse
|
7
|
Evans H, Hero E, Minhas F, Wahab N, Dodd K, Sahota H, Ganguly R, Robinson A, Neerudu M, Blessing E, Borkar P, Snead D. Standardized Clinical Annotation of Digital Histopathology Slides at the Point of Diagnosis. Mod Pathol 2023; 36:100297. [PMID: 37544362 DOI: 10.1016/j.modpat.2023.100297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 06/26/2023] [Accepted: 07/25/2023] [Indexed: 08/08/2023]
Abstract
As digital pathology replaces conventional glass slide microscopy as a means of reporting cellular pathology samples, the annotation of digital pathology whole slide images is rapidly becoming part of a pathologist's regular practice. Currently, there is no recognizable organization of these annotations, and as a result, pathologists adopt an arbitrary approach to defining regions of interest, leading to irregularity and inconsistency and limiting the downstream efficient use of this valuable effort. In this study, we propose a Standardized Annotation Reporting Style for digital whole slide images. We formed a list of 167 commonly annotated entities (under 12 specialty subcategories) based on review of Royal College of Pathologists and College of American Pathologists documents, feedback from reporting pathologists in our NHS department, and experience in developing annotation dictionaries for PathLAKE research projects. Each entity was assigned a suitable annotation shape, SNOMED CT (SNOMED International) code, and unique color. Additionally, as an example of how the approach could be expanded to specific tumor types, all lung tumors in the fifth World Health Organization of thoracic tumors 2021 were included. The proposed standardization of annotations increases their utility, making them identifiable at low power and searchable across and between cases. This would aid pathologists reporting and reviewing cases and enable annotations to be used for research. This structured approach could serve as the basis for an industry standard and be easily adopted to ensure maximum functionality and efficiency in the use of annotations made during routine clinical examination of digital slides.
Collapse
Affiliation(s)
- Harriet Evans
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom; Warwick Medical School, University of Warwick, Coventry, United Kingdom.
| | - Emily Hero
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom; Histopathology Department, University Hospitals of Leicester NHS Trust, Leicester, United Kingdom
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, United Kingdom
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, United Kingdom
| | - Katherine Dodd
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| | - Harvir Sahota
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom; Department of Psychiatry, Coventry and Warwickshire Partnership Trust, Coventry, United Kingdom
| | - Ratnadeep Ganguly
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| | - Andrew Robinson
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| | - Manjuvani Neerudu
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| | - Elaine Blessing
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| | - Pallavi Borkar
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom
| | - David Snead
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, United Kingdom; Warwick Medical School, University of Warwick, Coventry, United Kingdom
| |
Collapse
|
8
|
Bilal M, Tsang YW, Ali M, Graham S, Hero E, Wahab N, Dodd K, Sahota H, Wu S, Lu W, Jahanifar M, Robinson A, Azam A, Benes K, Nimir M, Hewitt K, Bhalerao A, Eldaly H, Raza SEA, Gopalakrishnan K, Minhas F, Snead D, Rajpoot N. Development and validation of artificial intelligence-based prescreening of large-bowel biopsies taken in the UK and Portugal: a retrospective cohort study. Lancet Digit Health 2023; 5:e786-e797. [PMID: 37890902 DOI: 10.1016/s2589-7500(23)00148-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 07/10/2023] [Accepted: 07/25/2023] [Indexed: 10/29/2023]
Abstract
BACKGROUND Histopathological examination is a crucial step in the diagnosis and treatment of many major diseases. Aiming to facilitate diagnostic decision making and improve the workload of pathologists, we developed an artificial intelligence (AI)-based prescreening tool that analyses whole-slide images (WSIs) of large-bowel biopsies to identify typical, non-neoplastic, and neoplastic biopsies. METHODS This retrospective cohort study was conducted with an internal development cohort of slides acquired from a hospital in the UK and three external validation cohorts of WSIs acquired from two hospitals in the UK and one clinical laboratory in Portugal. To learn the differential histological patterns from digitised WSIs of large-bowel biopsy slides, our proposed weakly supervised deep-learning model (Colorectal AI Model for Abnormality Detection [CAIMAN]) used slide-level diagnostic labels and no detailed cell or region-level annotations. The method was developed with an internal development cohort of 5054 biopsy slides from 2080 patients that were labelled with corresponding diagnostic categories assigned by pathologists. The three external validation cohorts, with a total of 1536 slides, were used for independent validation of CAIMAN. Each WSI was classified into one of three classes (ie, typical, atypical non-neoplastic, and atypical neoplastic). Prediction scores of image tiles were aggregated into three prediction scores for the whole slide, one for its likelihood of being typical, one for its likelihood of being non-neoplastic, and one for its likelihood of being neoplastic. The assessment of the external validation cohorts was conducted by the trained and frozen CAIMAN model. To evaluate model performance, we calculated area under the convex hull of the receiver operating characteristic curve (AUROC), area under the precision-recall curve, and specificity compared with our previously published iterative draw and rank sampling (IDaRS) algorithm. We also generated heat maps and saliency maps to analyse and visualise the relationship between the WSI diagnostic labels and spatial features of the tissue microenvironment. The main outcome of this study was the ability of CAIMAN to accurately identify typical and atypical WSIs of colon biopsies, which could potentially facilitate automatic removing of typical biopsies from the diagnostic workload in clinics. FINDINGS A randomly selected subset of all large bowel biopsies was obtained between Jan 1, 2012, and Dec 31, 2017. The AI training, validation, and assessments were done between Jan 1, 2021, and Sept 30, 2022. WSIs with diagnostic labels were collected between Jan 1 and Sept 30, 2022. Our analysis showed no statistically significant differences across prediction scores from CAIMAN for typical and atypical classes based on anatomical sites of the biopsy. At 0·99 sensitivity, CAIMAN (specificity 0·5592) was more accurate than an IDaRS-based weakly supervised WSI-classification pipeline (0·4629) in identifying typical and atypical biopsies on cross-validation in the internal development cohort (p<0·0001). At 0·99 sensitivity, CAIMAN was also more accurate than IDaRS for two external validation cohorts (p<0·0001), but not for a third external validation cohort (p=0·10). CAIMAN provided higher specificity than IDaRS at some high-sensitivity thresholds (0·7763 vs 0·6222 for 0·95 sensitivity, 0·7126 vs 0·5407 for 0·97 sensitivity, and 0·5615 vs 0·3970 for 0·99 sensitivity on one of the external validation cohorts) and showed high classification performance in distinguishing between neoplastic biopsies (AUROC 0·9928, 95% CI 0·9927-0·9929), inflammatory biopsies (0·9658, 0·9655-0·9661), and atypical biopsies (0·9789, 0·9786-0·9792). On the three external validation cohorts, CAIMAN had AUROC values of 0·9431 (95% CI 0·9165-0·9697), 0·9576 (0·9568-0·9584), and 0·9636 (0·9615-0·9657) for the detection of atypical biopsies. Saliency maps supported the representation of disease heterogeneity in model predictions and its association with relevant histological features. INTERPRETATION CAIMAN, with its high sensitivity in detecting atypical large-bowel biopsies, might be a promising improvement in clinical workflow efficiency and diagnostic decision making in prescreening of typical colorectal biopsies. FUNDING The Pathology Image Data Lake for Analytics, Knowledge and Education Centre of Excellence; the UK Government's Industrial Strategy Challenge Fund; and Innovate UK on behalf of UK Research and Innovation.
Collapse
Affiliation(s)
- Mohsin Bilal
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Artificial Intelligence and Data Science, National University of Computer and Emerging Sciences, Islamabad, Pakistan
| | - Yee Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Mahmoud Ali
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Histofy, Birmingham, UK
| | - Emily Hero
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Department of Pathology, University Hospitals of Leicester National Health Service Trust, Leicester, UK
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Katherine Dodd
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Harvir Sahota
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Shaobin Wu
- Department of Pathology, East Suffolk and North Essex National Health Service Foundation Trust, Colchester, UK
| | - Wenqi Lu
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Andrew Robinson
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Ayesha Azam
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Ksenija Benes
- Department of Pathology, The Royal Wolverhampton National Health Service Trust, Wolverhampton, UK
| | - Mohammed Nimir
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Katherine Hewitt
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Hesham Eldaly
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Kishore Gopalakrishnan
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - David Snead
- Warwick Medical School, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Histofy, Birmingham, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Histofy, Birmingham, UK; The Alan Turing Institute, London, UK.
| |
Collapse
|
9
|
Makhlouf S, Wahab N, Toss M, Ibrahim A, Lashen AG, Atallah NM, Ghannam S, Jahanifar M, Lu W, Graham S, Mongan NP, Bilal M, Bhalerao A, Snead D, Minhas F, Raza SEA, Rajpoot N, Rakha E. Evaluation of tumour infiltrating lymphocytes in luminal breast cancer using artificial intelligence. Br J Cancer 2023; 129:1747-1758. [PMID: 37777578 PMCID: PMC10667537 DOI: 10.1038/s41416-023-02451-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2023] [Revised: 09/08/2023] [Accepted: 09/20/2023] [Indexed: 10/02/2023] Open
Abstract
BACKGROUND Tumour infiltrating lymphocytes (TILs) are a prognostic parameter in triple-negative and human epidermal growth factor receptor 2 (HER2)-positive breast cancer (BC). However, their role in luminal (oestrogen receptor positive and HER2 negative (ER + /HER2-)) BC remains unclear. In this study, we used artificial intelligence (AI) to assess the prognostic significance of TILs in a large well-characterised cohort of luminal BC. METHODS Supervised deep learning model analysis of Haematoxylin and Eosin (H&E)-stained whole slide images (WSI) was applied to a cohort of 2231 luminal early-stage BC patients with long-term follow-up. Stromal TILs (sTILs) and intratumoural TILs (tTILs) were quantified and their spatial distribution within tumour tissue, as well as the proportion of stroma involved by sTILs were assessed. The association of TILs with clinicopathological parameters and patient outcome was determined. RESULTS A strong positive linear correlation was observed between sTILs and tTILs. High sTILs and tTILs counts, as well as their proximity to stromal and tumour cells (co-occurrence) were associated with poor clinical outcomes and unfavourable clinicopathological parameters including high tumour grade, lymph node metastasis, large tumour size, and young age. AI-based assessment of the proportion of stroma composed of sTILs (as assessed visually in routine practice) was not predictive of patient outcome. tTILs was an independent predictor of worse patient outcome in multivariate Cox Regression analysis. CONCLUSION AI-based detection of TILs counts, and their spatial distribution provides prognostic value in luminal early-stage BC patients. The utilisation of AI algorithms could provide a comprehensive assessment of TILs as a morphological variable in WSIs beyond eyeballing assessment.
Collapse
Affiliation(s)
- Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Assiut University, Assiut, Egypt
| | - Noorul Wahab
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Michael Toss
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Histopathology, Sheffield Teaching Hospitals NHS Trust, Sheffield, UK
| | - Asmaa Ibrahim
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Suez Canal University, Ismailia, Egypt
| | - Ayat G Lashen
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Menoufia University, Menoufia, Egypt
| | - Nehal M Atallah
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Pathology, Faculty of Medicine, Menoufia University, Menoufia, Egypt
| | - Suzan Ghannam
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK
- Department of Histology and cell biology, Faculty of Medicine, Suez Canal University, Ismailia, Egypt
| | | | - Wenqi Lu
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Nigel P Mongan
- Biodiscovery Institute, School of Veterinary Medicine and Sciences, University of Nottingham, Nottingham, UK
- Department of Pharmacology, Weill Cornell Medicine, New York, NY, 10065, USA
| | - Mohsin Bilal
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - David Snead
- University Hospital Coventry and Warwickshire, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK.
| | - Emad Rakha
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK.
- Department of Histopathology, Nottingham University Hospitals NHS Trust, Nottingham, UK.
- Department of Pathology, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
10
|
Eastwood M, Sailem H, Marc ST, Gao X, Offman J, Karteris E, Fernandez AM, Jonigk D, Cookson W, Moffatt M, Popat S, Minhas F, Robertus JL. MesoGraph: Automatic profiling of mesothelioma subtypes from histological images. Cell Rep Med 2023; 4:101226. [PMID: 37816348 PMCID: PMC10591053 DOI: 10.1016/j.xcrm.2023.101226] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 06/08/2023] [Accepted: 09/14/2023] [Indexed: 10/12/2023]
Abstract
Mesothelioma is classified into three histological subtypes, epithelioid, sarcomatoid, and biphasic, according to the relative proportions of epithelioid and sarcomatoid tumor cells present. Current guidelines recommend that the sarcomatoid component of each mesothelioma is quantified, as a higher percentage of sarcomatoid pattern in biphasic mesothelioma shows poorer prognosis. In this work, we develop a dual-task graph neural network (GNN) architecture with ranking loss to learn a model capable of scoring regions of tissue down to cellular resolution. This allows quantitative profiling of a tumor sample according to the aggregate sarcomatoid association score. Tissue is represented by a cell graph with both cell-level morphological and regional features. We use an external multicentric test set from Mesobank, on which we demonstrate the predictive performance of our model. We additionally validate our model predictions through an analysis of the typical morphological features of cells according to their predicted score.
Collapse
Affiliation(s)
- Mark Eastwood
- Tissue Image Analytics Center, University of Warwick, Coventry, UK.
| | - Heba Sailem
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK; Kings College London, London, UK
| | | | - Xiaohong Gao
- Department of Computer Science, University of Middlesex, London, UK
| | - Judith Offman
- Kings College London, London, UK; Wolfson Institute of Population Health, Queen Mary University of London, London, UK
| | - Emmanouil Karteris
- College of Health, Medicine and Life Sciences, Brunel University London, London, UK
| | | | - Danny Jonigk
- German Center for Lung Research (DZL), BREATH, Hanover, Germany; Institute of Pathology, Medical Faculty of RWTH Aachen University, Aachen, Germany
| | - William Cookson
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Miriam Moffatt
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Sanjay Popat
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Center, University of Warwick, Coventry, UK; Warwick Cancer Research Centre, University of Warwick, Coventry, UK
| | | |
Collapse
|
11
|
Atallah NM, Wahab N, Toss MS, Makhlouf S, Ibrahim AY, Lashen AG, Ghannam S, Mongan NP, Jahanifar M, Graham S, Bilal M, Bhalerao A, Ahmed Raza SE, Snead D, Minhas F, Rajpoot N, Rakha E. Deciphering the Morphology of Tumor-Stromal Features in Invasive Breast Cancer Using Artificial Intelligence. Mod Pathol 2023; 36:100254. [PMID: 37380057 DOI: 10.1016/j.modpat.2023.100254] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 06/02/2023] [Accepted: 06/14/2023] [Indexed: 06/30/2023]
Abstract
Tumor-associated stroma in breast cancer (BC) is complex and exhibits a high degree of heterogeneity. To date, no standardized assessment method has been established. Artificial intelligence (AI) could provide an objective morphologic assessment of tumors and stroma, with the potential to identify new features not discernible by visual microscopy. In this study, we used AI to assess the clinical significance of (1) stroma-to-tumor ratio (S:TR) and (2) the spatial arrangement of stromal cells, tumor cell density, and tumor burden in BC. Whole-slide images of a large cohort (n = 1968) of well-characterized luminal BC cases were examined. Region and cell-level annotation was performed, and supervised deep learning models were applied for automated quantification of tumor and stromal features. S:TR was calculated in terms of surface area and cell count ratio, and the S:TR heterogeneity and spatial distribution were also assessed. Tumor cell density and tumor size were used to estimate tumor burden. Cases were divided into discovery (n = 1027) and test (n = 941) sets for validation of the findings. In the whole cohort, the stroma-to-tumor mean surface area ratio was 0.74, and stromal cell density heterogeneity score was high (0.7/1). BC with high S:TR showed features characteristic of good prognosis and longer patient survival in both the discovery and test sets. Heterogeneous spatial distribution of S:TR areas was predictive of worse outcome. Higher tumor burden was associated with aggressive tumor behavior and shorter survival and was an independent predictor of worse outcome (BC-specific survival; hazard ratio: 1.7, P = .03, 95% CI, 1.04-2.83 and distant metastasis-free survival; hazard ratio: 1.64, P = .04, 95% CI, 1.01-2.62) superior to absolute tumor size. The study concludes that AI provides a tool to assess major and subtle morphologic stromal features in BC with prognostic implications. Tumor burden is more prognostically informative than tumor size.
Collapse
Affiliation(s)
- Nehal M Atallah
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Department of Pathology, Faculty of Medicine, Menoufia University, Egypt
| | - Noorul Wahab
- Tissue Image Analytics Centre, University of Warwick, Conventry, UK
| | - Michael S Toss
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Histopathology Department, Royal Hallamshire Hospital, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, UK
| | - Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Department of Pathology, Faculty of Medicine, Assiut University, Egypt
| | - Asmaa Y Ibrahim
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Department of Pathology, Faculty of Medicine, Suez Canal University, Egypt
| | - Ayat G Lashen
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Department of Pathology, Faculty of Medicine, Menoufia University, Egypt
| | - Suzan Ghannam
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Department of Histology and Cell Biology, Faculty of Medicine, Suez Canal University, Egypt
| | - Nigel P Mongan
- Biodiscovery Institute, School of Veterinary Medicine and Sciences, University of Nottingham, Sutton Bonington, UK; Department of Pharmacology, Weill Cornell Medicine, New York
| | | | - Simon Graham
- Tissue Image Analytics Centre, University of Warwick, Conventry, UK
| | - Mohsin Bilal
- Tissue Image Analytics Centre, University of Warwick, Conventry, UK
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, University of Warwick, Conventry, UK
| | | | - David Snead
- Cellular Pathology, University Hospitals Coventry and Warwickshire NHS Trust, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, Conventry, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, Conventry, UK.
| | - Emad Rakha
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham, Nottingham, UK; Department of Pathology, Faculty of Medicine, Menoufia University, Egypt; Pathology Department, Hamad Medical Corporation, Doha, Qatar.
| |
Collapse
|
12
|
Pocock J, Raza SEA, Minhas F, Rajpoot N. WSIC: a Python package and command-line interface for fast whole slide image conversion. Bioinform Adv 2023; 3:vbad122. [PMID: 37720007 PMCID: PMC10502236 DOI: 10.1093/bioadv/vbad122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Revised: 08/09/2023] [Accepted: 09/08/2023] [Indexed: 09/19/2023]
Abstract
Summary Whole slide images (WSIs) are multi-gigapixel images of tissue sections, which are used in digital and computational pathology workflows. WSI datasets are commonly heterogeneous collections of proprietary or niche specialized formats which are challenging to handle. This note describes an open-source Python application for efficiently converting between WSI formats, including common, open, and emerging cloud-friendly formats. WSIC is a software tool that can quickly convert WSI files across various formats. It has a high performance and maintains the resolution metadata of the original images. WSIC is ideal for pre-processing large-scale WSI datasets with different file types. Availability and implementation Source code is available on GitHub at https://github.com/John-P/wsic/ under a permissive licence. WSIC is also available as a package on PyPI at https://pypi.org/project/WSIC/.
Collapse
Affiliation(s)
- Johnathan Pocock
- Department of Computer Science, University of Warwick, West Midlands CV4 7AL, United Kingdom
| | - Shan E Ahmed Raza
- Department of Computer Science, University of Warwick, West Midlands CV4 7AL, United Kingdom
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, West Midlands CV4 7AL, United Kingdom
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, West Midlands CV4 7AL, United Kingdom
| |
Collapse
|
13
|
Graham S, Minhas F, Bilal M, Ali M, Tsang YW, Eastwood M, Wahab N, Jahanifar M, Hero E, Dodd K, Sahota H, Wu S, Lu W, Azam A, Benes K, Nimir M, Hewitt K, Bhalerao A, Robinson A, Eldaly H, Raza SEA, Gopalakrishnan K, Snead D, Rajpoot N. Screening of normal endoscopic large bowel biopsies with interpretable graph learning: a retrospective study. Gut 2023; 72:1709-1721. [PMID: 37173125 PMCID: PMC10423541 DOI: 10.1136/gutjnl-2023-329512] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Accepted: 04/15/2023] [Indexed: 05/15/2023]
Abstract
OBJECTIVE To develop an interpretable artificial intelligence algorithm to rule out normal large bowel endoscopic biopsies, saving pathologist resources and helping with early diagnosis. DESIGN A graph neural network was developed incorporating pathologist domain knowledge to classify 6591 whole-slides images (WSIs) of endoscopic large bowel biopsies from 3291 patients (approximately 54% female, 46% male) as normal or abnormal (non-neoplastic and neoplastic) using clinically driven interpretable features. One UK National Health Service (NHS) site was used for model training and internal validation. External validation was conducted on data from two other NHS sites and one Portuguese site. RESULTS Model training and internal validation were performed on 5054 WSIs of 2080 patients resulting in an area under the curve-receiver operating characteristic (AUC-ROC) of 0.98 (SD=0.004) and AUC-precision-recall (PR) of 0.98 (SD=0.003). The performance of the model, named Interpretable Gland-Graphs using a Neural Aggregator (IGUANA), was consistent in testing over 1537 WSIs of 1211 patients from three independent external datasets with mean AUC-ROC=0.97 (SD=0.007) and AUC-PR=0.97 (SD=0.005). At a high sensitivity threshold of 99%, the proposed model can reduce the number of normal slides to be reviewed by a pathologist by approximately 55%. IGUANA also provides an explainable output highlighting potential abnormalities in a WSI in the form of a heatmap as well as numerical values associating the model prediction with various histological features. CONCLUSION The model achieved consistently high accuracy showing its potential in optimising increasingly scarce pathologist resources. Explainable predictions can guide pathologists in their diagnostic decision-making and help boost their confidence in the algorithm, paving the way for its future clinical adoption.
Collapse
Affiliation(s)
- Simon Graham
- Department of Computer Science, University of Warwick, Coventry, UK
- Histofy Ltd, Birmingham, UK
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Mohsin Bilal
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Mahmoud Ali
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Yee Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Mark Eastwood
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Noorul Wahab
- Department of Computer Science, University of Warwick, Coventry, UK
| | | | - Emily Hero
- Department of Pathology, University Hospitals of Leicester NHS Trust, Leicester, UK
| | - Katherine Dodd
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Harvir Sahota
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Shaobin Wu
- Department of Pathology, East Suffolk and North Essex NHS Foundation Trust, Colchester, UK
| | - Wenqi Lu
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Ayesha Azam
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Ksenija Benes
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
- Department of Pathology, Royal Wolverhampton Hospitals NHS Trust, Wolverhampton, UK
| | - Mohammed Nimir
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Katherine Hewitt
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Abhir Bhalerao
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Andrew Robinson
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Hesham Eldaly
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | | | - Kishore Gopalakrishnan
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - David Snead
- Histofy Ltd, Birmingham, UK
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
- Division of Biomedical Sciences, University of Warwick Warwick Medical School, Coventry, UK
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, Coventry, UK
- Histofy Ltd, Birmingham, UK
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| |
Collapse
|
14
|
Eastwood M, Marc ST, Gao X, Sailem H, Offman J, Karteris E, Fernandez AM, Jonigk D, Cookson W, Moffatt M, Popat S, Minhas F, Robertus JL. Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data. Artif Intell Med 2023; 143:102628. [PMID: 37673586 DOI: 10.1016/j.artmed.2023.102628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 06/30/2023] [Accepted: 07/14/2023] [Indexed: 09/08/2023]
Abstract
Malignant Mesothelioma is a difficult to diagnose and highly lethal cancer usually associated with asbestos exposure. It can be broadly classified into three subtypes: Epithelioid, Sarcomatoid, and a hybrid Biphasic subtype in which significant components of both of the previous subtypes are present. Early diagnosis and identification of the subtype informs treatment and can help improve patient outcome. However, the subtyping of malignant mesothelioma, and specifically the recognition of transitional features from routine histology slides has a high level of inter-observer variability. In this work, we propose an end-to-end multiple instance learning (MIL) approach for malignant mesothelioma subtyping. This uses an adaptive instance-based sampling scheme for training deep convolutional neural networks on bags of image patches that allows learning on a wider range of relevant instances compared to max or top-N based MIL approaches. We also investigate augmenting the instance representation to include aggregate cellular morphology features from cell segmentation. The proposed MIL approach enables identification of malignant mesothelial subtypes of specific tissue regions. From this a continuous characterisation of a sample according to predominance of sarcomatoid vs epithelioid regions is possible, thus avoiding the arbitrary and highly subjective categorisation by currently used subtypes. Instance scoring also enables studying tumor heterogeneity and identifying patterns associated with different subtypes. We have evaluated the proposed method on a dataset of 234 tissue micro-array cores with an AUROC of 0.89±0.05 for this task. The dataset and developed methodology is available for the community at: https://github.com/measty/PINS.
Collapse
Affiliation(s)
- Mark Eastwood
- Tissue Image Analytics Center, University of Warwick, United Kingdom.
| | - Silviu Tudor Marc
- Department of Computer Science, University of Middlesex, United Kingdom
| | - Xiaohong Gao
- Department of Computer Science, University of Middlesex, United Kingdom
| | - Heba Sailem
- Institute of Biomedical Engineering, University of Oxford, United Kingdom; Kings College London, United Kingdom
| | - Judith Offman
- Kings College London, United Kingdom; Wolfson Institute of Population Health, Queen Mary University of London, United Kingdom
| | | | | | - Danny Jonigk
- German Center for Lung Research (DZL), BREATH, Hanover, Germany; Institute of Pathology, Medical Faculty of RWTH Aachen University, Aachen, Germany
| | - William Cookson
- National Heart and Lung Institute, Imperial College London, United Kingdom
| | - Miriam Moffatt
- National Heart and Lung Institute, Imperial College London, United Kingdom
| | - Sanjay Popat
- National Heart and Lung Institute, Imperial College London, United Kingdom
| | - Fayyaz Minhas
- Tissue Image Analytics Center, University of Warwick, United Kingdom
| | - Jan Lukas Robertus
- National Heart and Lung Institute, Imperial College London, United Kingdom
| |
Collapse
|
15
|
Asif A, Rajpoot K, Graham S, Snead D, Minhas F, Rajpoot N. Unleashing the potential of AI for pathology: challenges and recommendations. J Pathol 2023; 260:564-577. [PMID: 37550878 PMCID: PMC10952719 DOI: 10.1002/path.6168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 06/21/2023] [Accepted: 06/22/2023] [Indexed: 08/09/2023]
Abstract
Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Amina Asif
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Kashif Rajpoot
- School of Computer ScienceUniversity of BirminghamBirminghamUK
| | - Simon Graham
- Histofy Ltd, Birmingham Business ParkBirminghamUK
| | - David Snead
- Histofy Ltd, Birmingham Business ParkBirminghamUK
- Department of PathologyUniversity Hospitals Coventry & Warwickshire NHS TrustCoventryUK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
- Cancer Research CentreUniversity of WarwickCoventryUK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
- Histofy Ltd, Birmingham Business ParkBirminghamUK
- Cancer Research CentreUniversity of WarwickCoventryUK
- The Alan Turing InstituteLondonUK
| |
Collapse
|
16
|
Ibrahim A, Toss MS, Makhlouf S, Miligy IM, Minhas F, Rakha EA. Improving mitotic cell counting accuracy and efficiency using phosphohistone-H3 (PHH3) antibody counterstained with haematoxylin and eosin as part of breast cancer grading. Histopathology 2023; 82:393-406. [PMID: 36349500 PMCID: PMC10100421 DOI: 10.1111/his.14837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Revised: 10/08/2022] [Accepted: 11/05/2022] [Indexed: 11/11/2022]
Abstract
BACKGROUND Mitotic count in breast cancer is an important prognostic marker. Unfortunately, substantial inter- and intraobserver variation exists when pathologists manually count mitotic figures. To alleviate this problem, we developed a new technique incorporating both haematoxylin and eosin (H&E) and phosphorylated histone H3 (PHH3), a marker highly specific to mitotic figures, and compared it to visual scoring of mitotic figures using H&E only. METHODS Two full-face sections from 97 cases were cut, one stained with H&E only, and the other was stained with PHH3 and counterstained with H&E (PHH3-H&E). Counting mitoses using PHH3-H&E was compared to traditional mitoses scoring using H&E in terms of reproducibility, scoring time, and the ability to detect mitosis hotspots. We assessed the agreement between manual and image analysis-assisted scoring of mitotic figures using H&E and PHH3-H&E-stained cells. The diagnostic performance of PHH3 in detecting mitotic figures in terms of sensitivity and specificity was measured. Finally, PHH3 replaced the mitosis score in a multivariate analysis to assess its significance. RESULTS Pathologists detected significantly higher mitotic figures using the PHH3-H&E (median ± SD, 20 ± 33) compared with H&E alone (median ± SD, 16 ± 25), P < 0.001. The concordance between pathologists in identifying mitotic figures was highest when using the dual PHH3-H&E technique; in addition, it highlighted mitotic figures at low power, allowing better agreement on choosing the hotspot area (k = 0.842) in comparison with standard H&E (k = 0.625). A better agreement between image analysis-assisted software and the human eye was observed for PHH3-stained mitotic figures. When the mitosis score was replaced with PHH3 in a Cox regression model with other grade components, PHH3 was an independent predictor of survival (hazard ratio [HR] 5.66, 95% confidence interval [CI] 1.92-16.69; P = 0.002), and even showed a more significant association with breast cancer-specific survival (BCSS) than mitosis (HR 3.63, 95% CI 1.49-8.86; P = 0.005) and Ki67 (P = 0.27). CONCLUSION Using PHH3-H&E-stained slides can reliably be used in routine scoring of mitotic figures and integrating both techniques will compensate for each other's limitations and improve diagnostic accuracy, quality, and precision.
Collapse
Affiliation(s)
- Asmaa Ibrahim
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham Biodiscovery Institute, University Park, Nottingham, UK.,Histopathology department, Faculty of Medicine, Suez Canal University, Ismailia, Egypt
| | - Michael S Toss
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham Biodiscovery Institute, University Park, Nottingham, UK
| | - Shorouk Makhlouf
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham Biodiscovery Institute, University Park, Nottingham, UK.,Department of Pathology, Faculty of Medicine, Assiut University, Assiut, Egypt
| | - Islam M Miligy
- Histopathology department, Faculty of Medicine, Menoufia University, Shebin El Kom, Egypt.,Histopathology department, School of Medicine, University of Nottingham, Nottingham, UK
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Emad A Rakha
- Academic Unit for Translational Medical Sciences, School of Medicine, University of Nottingham Biodiscovery Institute, University Park, Nottingham, UK.,Histopathology department, Faculty of Medicine, Menoufia University, Shebin El Kom, Egypt.,Histopathology department, School of Medicine, University of Nottingham, Nottingham, UK
| |
Collapse
|
17
|
Albusayli R, Graham JD, Pathmanathan N, Shaban M, Raza SEA, Minhas F, Armes JE, Rajpoot N. Artificial intelligence-based digital scores of stromal tumour-infiltrating lymphocytes and tumour-associated stroma predict disease-specific survival in triple-negative breast cancer. J Pathol 2023; 260:32-42. [PMID: 36705810 DOI: 10.1002/path.6061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Revised: 01/12/2023] [Accepted: 01/23/2023] [Indexed: 01/28/2023]
Abstract
Triple-negative breast cancer (TNBC) is known to have a relatively poor outcome with variable prognoses, raising the need for more informative risk stratification. We investigated a set of digital, artificial intelligence (AI)-based spatial tumour microenvironment (sTME) features and explored their prognostic value in TNBC. After performing tissue classification on digitised haematoxylin and eosin (H&E) slides of TNBC cases, we employed a deep learning-based algorithm to segment tissue regions into tumour, stroma, and lymphocytes in order to compute quantitative features concerning the spatial relationship of tumour with lymphocytes and stroma. The prognostic value of the digital features was explored using survival analysis with Cox proportional hazard models in a cross-validation setting on two independent international multi-centric TNBC cohorts: The Australian Breast Cancer Tissue Bank (AUBC) cohort (n = 318) and The Cancer Genome Atlas Breast Cancer (TCGA) cohort (n = 111). The proposed digital stromal tumour-infiltrating lymphocytes (Digi-sTILs) score and the digital tumour-associated stroma (Digi-TAS) score were found to carry strong prognostic value for disease-specific survival, with the Digi-sTILs and Digi-TAS scores giving C-index values of 0.65 (p = 0.0189) and 0.60 (p = 0.0437), respectively, on the TCGA cohort as a validation set. Combining the Digi-sTILs feature with the patient's positivity status for axillary lymph nodes yielded a C-index of 0.76 on unseen validation cohorts. We surmise that the proposed digital features could potentially be used for better risk stratification and management of TNBC patients. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Rawan Albusayli
- Tissue Image Analytics Centre, The University of Warwick, Coventry, UK
| | - J Dinny Graham
- The Westmead Institute for Medical Research, The University of Sydney, Sydney, NSW, Australia.,Westmead Breast Cancer Institute, Western Sydney Local Health District, Sydney, NSW, Australia
| | - Nirmala Pathmanathan
- Westmead Breast Cancer Institute, Western Sydney Local Health District, Sydney, NSW, Australia.,Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
| | | | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, The University of Warwick, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, The University of Warwick, Coventry, UK
| | - Jane E Armes
- Pathology Queensland, Queensland Health, Herston, QLD, Australia
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, The University of Warwick, Coventry, UK.,The Alan Turing Institute, London, UK
| |
Collapse
|
18
|
Graham S, Vu QD, Jahanifar M, Raza SEA, Minhas F, Snead D, Rajpoot N. One model is all you need: Multi-task learning enables simultaneous histology image segmentation and classification. Med Image Anal 2023; 83:102685. [PMID: 36410209 DOI: 10.1016/j.media.2022.102685] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 10/20/2022] [Accepted: 11/03/2022] [Indexed: 11/13/2022]
Abstract
The recent surge in performance for image analysis of digitised pathology slides can largely be attributed to the advances in deep learning. Deep models can be used to initially localise various structures in the tissue and hence facilitate the extraction of interpretable features for biomarker discovery. However, these models are typically trained for a single task and therefore scale poorly as we wish to adapt the model for an increasing number of different tasks. Also, supervised deep learning models are very data hungry and therefore rely on large amounts of training data to perform well. In this paper, we present a multi-task learning approach for segmentation and classification of nuclei, glands, lumina and different tissue regions that leverages data from multiple independent data sources. While ensuring that our tasks are aligned by the same tissue type and resolution, we enable meaningful simultaneous prediction with a single network. As a result of feature sharing, we also show that the learned representation can be used to improve the performance of additional tasks via transfer learning, including nuclear classification and signet ring cell detection. As part of this work, we train our developed Cerberus model on a huge amount of data, consisting of over 600 thousand objects for segmentation and 440 thousand patches for classification. We use our approach to process 599 colorectal whole-slide images from TCGA, where we localise 377 million, 900 thousand and 2.1 million nuclei, glands and lumina respectively. We make this resource available to remove a major barrier in the development of explainable models for computational pathology.
Collapse
Affiliation(s)
- Simon Graham
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom; Histofy Ltd, United Kingdom.
| | - Quoc Dang Vu
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom
| | - David Snead
- Histofy Ltd, United Kingdom; Department of Pathology, University Hospitals Coventry & Warwickshire, United Kingdom
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom; Histofy Ltd, United Kingdom; Department of Pathology, University Hospitals Coventry & Warwickshire, United Kingdom
| |
Collapse
|
19
|
Yaseen A, Amin I, Akhter N, Ben-Hur A, Minhas F. Insights into performance evaluation of compound-protein interaction prediction methods. Bioinformatics 2022; 38:ii75-ii81. [PMID: 36124806 DOI: 10.1093/bioinformatics/btac496] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022] Open
Abstract
MOTIVATION Machine-learning-based prediction of compound-protein interactions (CPIs) is important for drug design, screening and repurposing. Despite numerous recent publication with increasing methodological sophistication claiming consistent improvements in predictive accuracy, we have observed a number of fundamental issues in experiment design that produce overoptimistic estimates of model performance. RESULTS We systematically analyze the impact of several factors affecting generalization performance of CPI predictors that are overlooked in existing work: (i) similarity between training and test examples in cross-validation; (ii) synthesizing negative examples in absence of experimentally verified negative examples and (iii) alignment of evaluation protocol and performance metrics with real-world use of CPI predictors in screening large compound libraries. Using both state-of-the-art approaches by other researchers as well as a simple kernel-based baseline, we have found that effective assessment of generalization performance of CPI predictors requires careful control over similarity between training and test examples. We show that, under stringent performance assessment protocols, a simple kernel-based approach can exceed the predictive performance of existing state-of-the-art methods. We also show that random pairing for generating synthetic negative examples for training and performance evaluation results in models with better generalization in comparison to more sophisticated strategies used in existing studies. Our analyses indicate that using proposed experiment design strategies can offer significant improvements for CPI prediction leading to effective target compound screening for drug repurposing and discovery of putative chemical ligands of SARS-CoV-2-Spike and Human-ACE2 proteins. AVAILABILITY AND IMPLEMENTATION Code and supplementary material available at https://github.com/adibayaseen/HKRCPI. SUPPLEMENTARY INFORMATION Supplementary data are available at Bioinformatics online.
Collapse
Affiliation(s)
- Adiba Yaseen
- Department of Computer and Information Sciences (DCIS), Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad 45650, Pakistan
| | - Imran Amin
- National Institute for Biotechnology and Genetic Engineering, Faisalabad 38000, Pakistan
| | - Naeem Akhter
- Department of Computer and Information Sciences (DCIS), Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad 45650, Pakistan
| | - Asa Ben-Hur
- Department of Computer Science, Colorado State University, Fort Collins, CO 80523, USA
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK
| |
Collapse
|
20
|
Ali M, Evans H, Whitney P, Minhas F, Snead DRJ. Using Systemised Nomenclature of Medicine (SNOMED) codes to select digital pathology whole slide images for long-term archiving. J Clin Pathol 2022; 76:349-352. [PMID: 36109157 PMCID: PMC10176345 DOI: 10.1136/jcp-2022-208483] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Accepted: 08/20/2022] [Indexed: 11/04/2022]
Abstract
The archiving of whole slide images represents a hurdle to digital pathology implementation largely because of the amount of data generated. The retention of glass slides is currently recommended for a minimum of 10 years, but it is for individual departments to determine how digital images are archived and for how long. In a retrospective study, we examined the combination of Systemised Nomenclature of Medicine (SNOMED) codes allocated to cases reported between July 2011 and December 2015 and recalled more than 12 months after diagnosis in comparison to non-recalled cases.Our results show that 0.2% of cases are recalled after 12 months, and SNOMED code combinations can be used to identify which cases are likely to be recalled and which are not. This approach could reduce the number of cases archived by 62% and still ensure all cases likely to be recalled remain in the archive.
Collapse
Affiliation(s)
- Mahmoud Ali
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
- Histopathology Department, Cambridge University Hospitals NHS Foundation Trust, Cambridge, Cambridgeshire, UK
| | - Harriet Evans
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Peter Whitney
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, Coventry, West Midlands, UK
| | - David R J Snead
- Histopathology Department, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
- Warick Medical School, University of Warwick, Coventry, UK
| |
Collapse
|
21
|
Lu W, Toss M, Dawood M, Rakha E, Rajpoot N, Minhas F. SlideGraph +: Whole slide image level graphs to predict HER2 status in breast cancer. Med Image Anal 2022; 80:102486. [DOI: 10.1016/j.media.2022.102486] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 04/15/2022] [Accepted: 05/20/2022] [Indexed: 01/18/2023]
|
22
|
Wahab N, Miligy IM, Dodd K, Sahota H, Toss M, Lu W, Jahanifar M, Bilal M, Graham S, Park Y, Hadjigeorghiou G, Bhalerao A, Lashen AG, Ibrahim AY, Katayama A, Ebili HO, Parkin M, Sorell T, Raza SEA, Hero E, Eldaly H, Tsang YW, Gopalakrishnan K, Snead D, Rakha E, Rajpoot N, Minhas F. Semantic annotation for computational pathology: multidisciplinary experience and best practice recommendations. J Pathol Clin Res 2022; 8:116-128. [PMID: 35014198 PMCID: PMC8822374 DOI: 10.1002/cjp2.256] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 11/25/2021] [Accepted: 12/10/2021] [Indexed: 02/06/2023]
Abstract
Recent advances in whole‐slide imaging (WSI) technology have led to the development of a myriad of computer vision and artificial intelligence‐based diagnostic, prognostic, and predictive algorithms. Computational Pathology (CPath) offers an integrated solution to utilise information embedded in pathology WSIs beyond what can be obtained through visual assessment. For automated analysis of WSIs and validation of machine learning (ML) models, annotations at the slide, tissue, and cellular levels are required. The annotation of important visual constructs in pathology images is an important component of CPath projects. Improper annotations can result in algorithms that are hard to interpret and can potentially produce inaccurate and inconsistent results. Despite the crucial role of annotations in CPath projects, there are no well‐defined guidelines or best practices on how annotations should be carried out. In this paper, we address this shortcoming by presenting the experience and best practices acquired during the execution of a large‐scale annotation exercise involving a multidisciplinary team of pathologists, ML experts, and researchers as part of the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a real‐world case study along with examples of different types of annotations, diagnostic algorithm, annotation data dictionary, and annotation constructs. The analyses reported in this work highlight best practice recommendations that can be used as annotation guidelines over the lifecycle of a CPath project.
Collapse
Affiliation(s)
- Noorul Wahab
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Islam M Miligy
- Pathology, University of Nottingham, Nottingham, UK.,Department of Pathology, Faculty of Medicine, Menoufia University, Shebin El-Kom, Egypt
| | - Katherine Dodd
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Harvir Sahota
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Michael Toss
- Pathology, University of Nottingham, Nottingham, UK
| | - Wenqi Lu
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | - Mohsin Bilal
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Young Park
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | - Abhir Bhalerao
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | | | - Ayaka Katayama
- Graduate School of Medicine, Gunma University, Maebashi, Japan
| | | | | | - Tom Sorell
- Department of Politics and International Studies, University of Warwick, Coventry, UK
| | | | - Emily Hero
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK.,Leicester Royal Infirmary, Histopathology, University Hospitals Leicester, Leicester, UK
| | - Hesham Eldaly
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Yee Wah Tsang
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | | | - David Snead
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Emad Rakha
- Pathology, University of Nottingham, Nottingham, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| |
Collapse
|
23
|
Gull S, Minhas F. AMP 0: Species-Specific Prediction of Anti-microbial Peptides Using Zero and Few Shot Learning. IEEE/ACM Trans Comput Biol Bioinform 2022; 19:275-283. [PMID: 32750857 DOI: 10.1109/tcbb.2020.2999399] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Evolution of drug-resistant microbial species is one of the major challenges to global health. Development of new antimicrobial treatments such as antimicrobial peptides needs to be accelerated to combat this threat. However, the discovery of novel antimicrobial peptides is hampered by low-throughput biochemical assays. Computational techniques can be used for rapid screening of promising antimicrobial peptide candidates prior to testing in the wet lab. The vast majority of existing antimicrobial peptide predictors are non-targeted in nature, i.e., they can predict whether a given peptide sequence is antimicrobial, but they are unable to predict whether the sequence can target a particular microbial species. In this work, we have used zero and few shot machine learning to develop a targeted antimicrobial peptide activity predictor called AMP0. The proposed predictor takes the sequence of a peptide and any N/C-termini modifications together with the genomic sequence of a microbial species to generate targeted predictions. Cross-validation results show that the proposed scheme is particularly effective for targeted antimicrobial prediction in comparison to existing approaches and can be used for screening potential antimicrobial peptides in a targeted manner with only a small number of training examples for novel species. AMP0 webserver is available at http://ampzero.pythonanywhere.com.
Collapse
|
24
|
Eastwood M, Marc ST, Gao X, Sailem H, Offman J, Karteris E, Fernandez AM, Jonigk D, Cookson W, Moffatt M, Popat S, Minhas F, Robertus JL. Malignant Mesothelioma Subtyping of Tissue Images via Sampling Driven Multiple Instance Prediction. Artif Intell Med 2022. [DOI: 10.1007/978-3-031-09342-5_25] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
25
|
Bilal M, Raza SEA, Azam A, Graham S, Ilyas M, Cree IA, Snead D, Minhas F, Rajpoot NM. Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: a retrospective study. Lancet Digit Health 2021; 3:e763-e772. [PMID: 34686474 PMCID: PMC8609154 DOI: 10.1016/s2589-7500(21)00180-1] [Citation(s) in RCA: 82] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 07/01/2021] [Accepted: 08/05/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Determining the status of molecular pathways and key mutations in colorectal cancer is crucial for optimal therapeutic decision making. We therefore aimed to develop a novel deep learning pipeline to predict the status of key molecular pathways and mutations from whole-slide images of haematoxylin and eosin-stained colorectal cancer slides as an alternative to current tests. METHODS In this retrospective study, we used 502 diagnostic slides of primary colorectal tumours from 499 patients in The Cancer Genome Atlas colon and rectal cancer (TCGA-CRC-DX) cohort and developed a weakly supervised deep learning framework involving three separate convolutional neural network models. Whole-slide images were divided into equally sized tiles and model 1 (ResNet18) extracted tumour tiles from non-tumour tiles. These tumour tiles were inputted into model 2 (adapted ResNet34), trained by iterative draw and rank sampling to calculate a prediction score for each tile that represented the likelihood of a tile belonging to the molecular labels of high mutation density (vs low mutation density), microsatellite instability (vs microsatellite stability), chromosomal instability (vs genomic stability), CpG island methylator phenotype (CIMP)-high (vs CIMP-low), BRAFmut (vs BRAFWT), TP53mut (vs TP53WT), and KRASWT (vs KRASmut). These scores were used to identify the top-ranked titles from each slide, and model 3 (HoVer-Net) segmented and classified the different types of cell nuclei in these tiles. We calculated the area under the convex hull of the receiver operating characteristic curve (AUROC) as a model performance measure and compared our results with those of previously published methods. FINDINGS Our iterative draw and rank sampling method yielded mean AUROCs for the prediction of hypermutation (0·81 [SD 0·03] vs 0·71), microsatellite instability (0·86 [0·04] vs 0·74), chromosomal instability (0·83 [0·02] vs 0·73), BRAFmut (0·79 [0·01] vs 0·66), and TP53mut (0·73 [0·02] vs 0·64) in the TCGA-CRC-DX cohort that were higher than those from previously published methods, and an AUROC for KRASmut that was similar to previously reported methods (0·60 [SD 0·04] vs 0·60). Mean AUROC for predicting CIMP-high status was 0·79 (SD 0·05). We found high proportions of tumour-infiltrating lymphocytes and necrotic tumour cells to be associated with microsatellite instability, and high proportions of tumour-infiltrating lymphocytes and a low proportion of necrotic tumour cells to be associated with hypermutation. INTERPRETATION After large-scale validation, our proposed algorithm for predicting clinically important mutations and molecular pathways, such as microsatellite instability, in colorectal cancer could be used to stratify patients for targeted therapies with potentially lower costs and quicker turnaround times than sequencing-based or immunohistochemistry-based approaches. FUNDING The UK Medical Research Council.
Collapse
Affiliation(s)
- Mohsin Bilal
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Ayesha Azam
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mohammad Ilyas
- Faculty of Medicine and Health Sciences, University of Nottingham, Nottingham, UK
| | - Ian A Cree
- International Agency for Research on Cancer, Lyon, France
| | - David Snead
- Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Nasir M Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK.
| |
Collapse
|
26
|
Abstract
Quantifying the hemolytic activity of peptides is a crucial step in the discovery of novel therapeutic peptides. Computational methods are attractive in this domain due to their ability to guide wet-lab experimental discovery or screening of peptides based on their hemolytic activity. However, existing methods are unable to accurately model various important aspects of this predictive problem such as the role of N/C-terminal modifications, D- and L- amino acids, etc. In this work, we have developed a novel neural network-based approach called HemoNet for predicting the hemolytic activity of peptides. The proposed method captures the contextual importance of different amino acids in a given peptide sequence using a specialized feature embedding in conjunction with SMILES-based fingerprint representation of N/C-terminal modifications. We have analyzed the predictive performance of the proposed method using stratified cross-validation in comparison with previous methods, non-redundant cross-validation as well as validation on external peptides and clinical antimicrobial peptides. Our analysis shows the proposed approach achieves significantly better predictive performance (AUC-ROC of 88%) in comparison to previous approaches (HemoPI and HemoPred with AUC-ROC of 73%). HemoNet can be a useful tool in the search for novel therapeutic peptides. The python implementation of the proposed method is available at the URL: https://github.com/adibayaseen/HemoNet.
Collapse
Affiliation(s)
- Adiba Yaseen
- Department of Computer and Information Science, Pakistan Institute of Engineering and Applied Science (PIEAS), Islamabad, Pakistan
| | - Sadaf Gull
- Department of Computer and Information Science, Pakistan Institute of Engineering and Applied Science (PIEAS), Islamabad, Pakistan
| | - Naeem Akhtar
- Department of Computer and Information Science, Pakistan Institute of Engineering and Applied Science (PIEAS), Islamabad, Pakistan
| | - Imran Amin
- National Institute for Biotechnology and Genetic Engineering (NIBGE), Faisalabad, Pakistan
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, Coventry, UK
| |
Collapse
|
27
|
Li KK, Jarvis SA, Minhas F. Elementary effects analysis of factors controlling COVID-19 infections in computational simulation reveals the importance of social distancing and mask usage. Comput Biol Med 2021; 134:104369. [PMID: 33915478 PMCID: PMC8019252 DOI: 10.1016/j.compbiomed.2021.104369] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 03/28/2021] [Accepted: 03/28/2021] [Indexed: 12/16/2022]
Abstract
COVID-19 was declared a pandemic by the World Health Organisation (WHO) on March 11th, 2020. With half of the world's countries in lockdown as of April due to this pandemic, monitoring and understanding the spread of the virus and infection rates and how these factors relate to behavioural and societal parameters is crucial for developing control strategies. This paper aims to investigate the effectiveness of masks, social distancing, lockdown and self-isolation for reducing the spread of SARS-CoV-2 infections. Our findings from an agent-based simulation modelling showed that whilst requiring a lockdown is widely believed to be the most efficient method to quickly reduce infection numbers, the practice of social distancing and the usage of surgical masks can potentially be more effective than requiring a lockdown. Our multivariate analysis of simulation results using the Morris Elementary Effects Method suggests that if a sufficient proportion of the population uses surgical masks and follows social distancing regulations, then SARS-CoV-2 infections can be controlled without requiring a lockdown.
Collapse
Affiliation(s)
- Kelvin K.F. Li
- Department of Computer Science, University of Warwick, United Kingdom,Centre of Cyber Logistics, The Chinese University of Hong Kong, Hong Kong,Corresponding author. Centre of Cyber Logistics, The Chinese University of Hong Kong, Hong Kong
| | - Stephen A. Jarvis
- College of Engineering and Physical Sciences, University of Birmingham, United Kingdom
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, United Kingdom
| |
Collapse
|
28
|
Awan R, Benes K, Azam A, Song TH, Shaban M, Verrill C, Tsang YW, Snead D, Minhas F, Rajpoot N. Deep learning based digital cell profiles for risk stratification of urine cytology images. Cytometry A 2021; 99:732-742. [PMID: 33486882 DOI: 10.1002/cyto.a.24313] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2020] [Revised: 12/05/2020] [Accepted: 12/15/2020] [Indexed: 11/06/2022]
Abstract
Urine cytology is a test for the detection of high-grade bladder cancer. In clinical practice, the pathologist would manually scan the sample under the microscope to locate atypical and malignant cells. They would assess the morphology of these cells to make a diagnosis. Accurate identification of atypical and malignant cells in urine cytology is a challenging task and is an essential part of identifying different diagnosis with low-risk and high-risk malignancy. Computer-assisted identification of malignancy in urine cytology can be complementary to the clinicians for treatment management and in providing advice for carrying out further tests. In this study, we presented a method for identifying atypical and malignant cells followed by their profiling to predict the risk of diagnosis automatically. For cell detection and classification, we employed two different deep learning-based approaches. Based on the best performing network predictions at the cell level, we identified low-risk and high-risk cases using the count of atypical cells and the total count of atypical and malignant cells. The area under the receiver operating characteristic (ROC) curve shows that a total count of atypical and malignant cells is comparably better at diagnosis as compared to the count of malignant cells only. We obtained area under the ROC curve with the count of malignant cells and the total count of atypical and malignant cells as 0.81 and 0.83, respectively. Our experiments also demonstrate that the digital risk could be a better predictor of the final histopathology-based diagnosis. We also analyzed the variability in annotations at both cell and whole slide image level and also explored the possible inherent rationales behind this variability.
Collapse
Affiliation(s)
- Ruqayya Awan
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Ksenija Benes
- The Royal Wolverhampton NHS Trust, Wolverhampton, UK
| | - Ayesha Azam
- Department of Computer Science, University of Warwick, Coventry, UK.,Department of Pathology, University Hospitals Coventry and Warwickshire, Coventry, UK
| | - Tzu-Hsi Song
- Department of Computer Science, University of Warwick, Coventry, UK.,Laboratory of Quantitative Cellular Imaging, Worcester Polytechnic Institute, Worcester, Massachusetts, USA
| | - Muhammad Shaban
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Clare Verrill
- Nuffield Department of Surgical Sciences and Oxford NIHR Biomedical Research Centre, University of Oxford, Oxford, UK
| | - Yee Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire, Coventry, UK
| | - David Snead
- Department of Pathology, University Hospitals Coventry and Warwickshire, Coventry, UK
| | - Fayyaz Minhas
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, Coventry, UK.,Department of Pathology, University Hospitals Coventry and Warwickshire, Coventry, UK.,The Alan Turing Institute, London, UK
| |
Collapse
|
29
|
Gull S, Shamim N, Minhas F. AMAP: Hierarchical multi-label prediction of biologically active and antimicrobial peptides. Comput Biol Med 2019; 107:172-181. [DOI: 10.1016/j.compbiomed.2019.02.018] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2018] [Revised: 02/17/2019] [Accepted: 02/20/2019] [Indexed: 12/12/2022]
|
30
|
Elahi MM, Minhas F, Elahi MO, Moreira-Gonzalez A, Jackson IT. Incompletely excised basal cell cancer: the re-excised specimen. European Journal of Plastic Surgery 2003. [DOI: 10.1007/s00238-003-0538-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
31
|
Ge K, Minhas F, Duhadaway J, Mao NC, Wilson D, Buccafusca R, Sakamuro D, Nelson P, Malkowicz SB, Tomaszewski J, Prendergast GC. Loss of heterozygosity and tumor suppressor activity of Bin1 in prostate carcinoma. Int J Cancer 2000; 86:155-61. [PMID: 10738240 DOI: 10.1002/(sici)1097-0215(20000415)86:2<155::aid-ijc2>3.0.co;2-m] [Citation(s) in RCA: 69] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The genetic events underlying the development of prostate cancer are poorly defined. c-Myc is often activated in tumors that have progressed to metastatic status, so events that promote this process may be important. Bin1 is a nucleocytoplasmic adaptor protein with features of a tumor suppressor that was identified through its ability to interact with and inhibit malignant transformation by c-Myc. We investigated a role for Bin1 loss or inactivation in prostate cancer because the human Bin1 gene is located at chromosome 2q14 within a region that is frequently deleted in metastatic prostate cancer but where no tumor suppressor candidate has been located. A novel polymorphic microsatellite marker located within intron 5 of the human Bin1 gene was used to demonstrate loss of heterozygosity and coding alteration in 40% of informative cases of prostate neoplasia examined. RNA and immunohistochemical analyses indicated that Bin1 was expressed in most primary tumors, even at slightly elevated levels relative to benign tissues, but that it was frequently missing or inactivated by aberrant splicing in metastatic tumors and androgen-independent tumor cell lines. Ectopic expression of Bin1 suppressed the growth of prostate cancer lines in vitro. Our findings support the candidacy of Bin1 as the chromosome 2q prostate tumor suppressor gene.
Collapse
Affiliation(s)
- K Ge
- The Wistar Institute, Philadelphia, Pennsylvania, USA
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|