1
|
Makhlouf Y, Singh VK, Craig S, McArdle A, French D, Loughrey MB, Oliver N, Acevedo JB, O’Reilly P, James JA, Maxwell P, Salto-Tellez M. True-T - Improving T-cell response quantification with holistic artificial intelligence based prediction in immunohistochemistry images. Comput Struct Biotechnol J 2024; 23:174-185. [PMID: 38146436 PMCID: PMC10749253 DOI: 10.1016/j.csbj.2023.11.048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 11/24/2023] [Accepted: 11/24/2023] [Indexed: 12/27/2023] Open
Abstract
The immune response associated with oncogenesis and potential oncological ther- apeutic interventions has dominated the field of cancer research over the last decade. T-cell lymphocytes in the tumor microenvironment are a crucial aspect of cancer's adaptive immunity, and the quantification of T-cells in specific can- cer types has been suggested as a potential diagnostic aid. However, this is cur- rently not part of routine diagnostics. To address this challenge, we present a new method called True-T, which employs artificial intelligence-based techniques to quantify T-cells in colorectal cancer (CRC) using immunohistochemistry (IHC) images. True-T analyses the chromogenic tissue hybridization signal of three widely recognized T-cell markers (CD3, CD4, and CD8). Our method employs a pipeline consisting of three stages: T-cell segmentation, density estimation from the segmented mask, and prediction of individual five-year survival rates. In the first stage, we utilize the U-Net method, where a pre-trained ResNet-34 is em- ployed as an encoder to extract clinically relevant T-cell features. The segmenta- tion model is trained and evaluated individually, demonstrating its generalization in detecting the CD3, CD4, and CD8 biomarkers in IHC images. In the second stage, the density of T-cells is estimated using the predicted mask, which serves as a crucial indicator for patient survival statistics in the third stage. This ap- proach was developed and tested in 1041 patients from four reference diagnostic institutions, ensuring broad applicability. The clinical effectiveness of True-T is demonstrated in stages II-IV CRC by offering valuable prognostic information that surpasses previous quantitative gold standards, opening possibilities for po- tential clinical applications. Finally, to evaluate the robustness and broader ap- plicability of our approach without additional training, we assessed the universal accuracy of the CD3 component of the True-T algorithm across 13 distinct solid tumors.
Collapse
Affiliation(s)
- Yasmine Makhlouf
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Vivek Kumar Singh
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Stephanie Craig
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Aoife McArdle
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Dominique French
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Maurice B. Loughrey
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
- Cellular Pathology, Belfast Health and Social Care Trust, Belfast City Hospital, Lisburn Road, Belfast BT9 7AB, UK
| | - Nicola Oliver
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Juvenal Baena Acevedo
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | | | - Jacqueline A. James
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
- Regional Molecular Diagnostic Service, Belfast Health and Social Care Trust, Belfast BT9 7AE, UK
| | - Perry Maxwell
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
| | - Manuel Salto-Tellez
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston, Centre for Cancer Research, Queen’s University Belfast, Belfast BT9 7AE, UK
- Sonrai Analytics, Belfast BT9 7AE, UK
- Regional Molecular Diagnostic Service, Belfast Health and Social Care Trust, Belfast BT9 7AE, UK
- Integrated Pathology Unit, Institute of Cancer Research and Royal Marsden Hospital, London SW7 3RP, UK
| |
Collapse
|
2
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
3
|
Haghofer A, Fuchs-Baumgartinger A, Lipnik K, Klopfleisch R, Aubreville M, Scharinger J, Weissenböck H, Winkler SM, Bertram CA. Histological classification of canine and feline lymphoma using a modular approach based on deep learning and advanced image processing. Sci Rep 2023; 13:19436. [PMID: 37945699 PMCID: PMC10636139 DOI: 10.1038/s41598-023-46607-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 11/02/2023] [Indexed: 11/12/2023] Open
Abstract
Histopathological examination of tissue samples is essential for identifying tumor malignancy and the diagnosis of different types of tumor. In the case of lymphoma classification, nuclear size of the neoplastic lymphocytes is one of the key features to differentiate the different subtypes. Based on the combination of artificial intelligence and advanced image processing, we provide a workflow for the classification of lymphoma with regards to their nuclear size (small, intermediate, and large). As the baseline for our workflow testing, we use a Unet++ model trained on histological images of canine lymphoma with individually labeled nuclei. As an alternative to the Unet++, we also used a publicly available pre-trained and unmodified instance segmentation model called Stardist to demonstrate that our modular classification workflow can be combined with different types of segmentation models if they can provide proper nuclei segmentation. Subsequent to nuclear segmentation, we optimize algorithmic parameters for accurate classification of nuclear size using a newly derived reference size and final image classification based on a pathologists-derived ground truth. Our image classification module achieves a classification accuracy of up to 92% on canine lymphoma data. Compared to the accuracy ranging from 66.67 to 84% achieved using measurements provided by three individual pathologists, our algorithm provides a higher accuracy level and reproducible results. Our workflow also demonstrates a high transferability to feline lymphoma, as shown by its accuracy of up to 84.21%, even though our workflow was not optimized for feline lymphoma images. By determining the nuclear size distribution in tumor areas, our workflow can assist pathologists in subtyping lymphoma based on the nuclei size and potentially improve reproducibility. Our proposed approach is modular and comprehensible, thus allowing adaptation for specific tasks and increasing the users' trust in computer-assisted image classification.
Collapse
Affiliation(s)
- Andreas Haghofer
- Bioinformatics Research Group, University of Applied Sciences Upper Austria, Softwarepark 11-13, 4232, Hagenberg, Austria.
- Department of Computer Science, Johannes Kepler University, Altenberger Straße 69, 4040, Linz, Austria.
| | - Andrea Fuchs-Baumgartinger
- Institute of Pathology, University of Veterinary Medicine Vienna, Veterinärplatz 1, 1210, Vienna, Austria
| | - Karoline Lipnik
- Institute of Pathology, University of Veterinary Medicine Vienna, Veterinärplatz 1, 1210, Vienna, Austria
| | - Robert Klopfleisch
- Institute of Veterinary Pathology, Freie Univerisität Berlin, Robert-von-Ostertag-Str. 15, 14163, Berlin, Germany
| | - Marc Aubreville
- Technische Hochschule Ingolstadt, Esplanade 10, 85049, Ingolstadt, Germany
| | - Josef Scharinger
- Institute of Computational Perception, Johannes Kepler University, Altenberger Straße 69, 4040, Linz, Austria
| | - Herbert Weissenböck
- Institute of Pathology, University of Veterinary Medicine Vienna, Veterinärplatz 1, 1210, Vienna, Austria
| | - Stephan M Winkler
- Bioinformatics Research Group, University of Applied Sciences Upper Austria, Softwarepark 11-13, 4232, Hagenberg, Austria
- Department of Computer Science, Johannes Kepler University, Altenberger Straße 69, 4040, Linz, Austria
| | - Christof A Bertram
- Institute of Pathology, University of Veterinary Medicine Vienna, Veterinärplatz 1, 1210, Vienna, Austria
| |
Collapse
|
4
|
Prezja F, Äyrämö S, Pölönen I, Ojala T, Lahtinen S, Ruusuvuori P, Kuopio T. Improved accuracy in colorectal cancer tissue decomposition through refinement of established deep learning solutions. Sci Rep 2023; 13:15879. [PMID: 37741820 PMCID: PMC10517936 DOI: 10.1038/s41598-023-42357-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 09/08/2023] [Indexed: 09/25/2023] Open
Abstract
Hematoxylin and eosin-stained biopsy slides are regularly available for colorectal cancer patients. These slides are often not used to define objective biomarkers for patient stratification and treatment selection. Standard biomarkers often pertain to costly and slow genetic tests. However, recent work has shown that relevant biomarkers can be extracted from these images using convolutional neural networks (CNNs). The CNN-based biomarkers predicted colorectal cancer patient outcomes comparably to gold standards. Extracting CNN-biomarkers is fast, automatic, and of minimal cost. CNN-based biomarkers rely on the ability of CNNs to recognize distinct tissue types from microscope whole slide images. The quality of these biomarkers (coined 'Deep Stroma') depends on the accuracy of CNNs in decomposing all relevant tissue classes. Improving tissue decomposition accuracy is essential for improving the prognostic potential of CNN-biomarkers. In this study, we implemented a novel training strategy to refine an established CNN model, which then surpassed all previous solutions . We obtained a 95.6% average accuracy in the external test set and 99.5% in the internal test set. Our approach reduced errors in biomarker-relevant classes, such as Lymphocytes, and was the first to include interpretability methods. These methods were used to better apprehend our model's limitations and capabilities.
Collapse
Affiliation(s)
- Fabi Prezja
- Faculty of Information Technology, University of Jyväskylä, Jyväskylä, 40014, Finland.
- Digital Health Intelligence Laboratory, University of Jyväskylä, Jyväskylä, 40014, Finland.
| | - Sami Äyrämö
- Faculty of Information Technology, University of Jyväskylä, Jyväskylä, 40014, Finland
- Digital Health Intelligence Laboratory, University of Jyväskylä, Jyväskylä, 40014, Finland
| | - Ilkka Pölönen
- Faculty of Information Technology, University of Jyväskylä, Jyväskylä, 40014, Finland
- Spectral Imaging Laboratory, University of Jyväskylä, Jyväskylä, 40014, Finland
| | - Timo Ojala
- Faculty of Information Technology, University of Jyväskylä, Jyväskylä, 40014, Finland
- Digital Health Intelligence Laboratory, University of Jyväskylä, Jyväskylä, 40014, Finland
| | - Suvi Lahtinen
- Faculty of Information Technology, University of Jyväskylä, Jyväskylä, 40014, Finland
- Department of Biological and Environmental Science, Faculty of Mathematics and Science, University of Jyväskylä, Jyväskylä, 40014, Finland
| | - Pekka Ruusuvuori
- Institute of Biomedicine, Cancer Research Unit, University of Turku, Turku, 20014, Finland
- FICAN West Cancer Centre, Turku University Hospital, Turku, 20521, Finland
| | - Teijo Kuopio
- Department of Education and Research, Hospital Nova of Central Finland, Jyväskylä, 40620, Finland
- Department of Biological and Environmental Science, University of Jyväskylä, Jyväskylä, 40014, Finland
- Department of Pathology, Hospital Nova of Central Finland, Jyväskylä, 40620, Finland
| |
Collapse
|