1
|
Tafavvoghi M, Bongo LA, Shvetsov N, Busund LTR, Møllersen K. Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review. J Pathol Inform 2024; 15:100363. [PMID: 38405160 PMCID: PMC10884505 DOI: 10.1016/j.jpi.2024.100363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 01/23/2024] [Indexed: 02/27/2024] Open
Abstract
Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.
Collapse
Affiliation(s)
- Masoud Tafavvoghi
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| | - Lars Ailo Bongo
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | - Nikita Shvetsov
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | | | - Kajsa Møllersen
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
2
|
Gonzalez R, Saha A, Campbell CJ, Nejat P, Lokker C, Norgan AP. Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities. J Pathol Inform 2024; 15:100347. [PMID: 38162950 PMCID: PMC10755052 DOI: 10.1016/j.jpi.2023.100347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 10/06/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024] Open
Abstract
This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.
Collapse
Affiliation(s)
- Ricardo Gonzalez
- DeGroote School of Business, McMaster University, Hamilton, Ontario, Canada
- Division of Computational Pathology and Artificial Intelligence, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| | - Ashirbani Saha
- Department of Oncology, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- Escarpment Cancer Research Institute, McMaster University and Hamilton Health Sciences, Hamilton, Ontario, Canada
| | - Clinton J.V. Campbell
- William Osler Health System, Brampton, Ontario, Canada
- Department of Pathology and Molecular Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Peyman Nejat
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Cynthia Lokker
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Andrew P. Norgan
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
3
|
Paverd H, Zormpas-Petridis K, Clayton H, Burge S, Crispin-Ortuzar M. Radiology and multi-scale data integration for precision oncology. NPJ Precis Oncol 2024; 8:158. [PMID: 39060351 PMCID: PMC11282284 DOI: 10.1038/s41698-024-00656-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Accepted: 07/15/2024] [Indexed: 07/28/2024] Open
Abstract
In this Perspective paper we explore the potential of integrating radiological imaging with other data types, a critical yet underdeveloped area in comparison to the fusion of other multi-omic data. Radiological images provide a comprehensive, three-dimensional view of cancer, capturing features that would be missed by biopsies or other data modalities. This paper explores the complexities and challenges of incorporating medical imaging into data integration models, in the context of precision oncology. We present the different categories of imaging-omics integration and discuss recent progress, highlighting the opportunities that arise from bringing together spatial data on different scales.
Collapse
Affiliation(s)
- Hania Paverd
- Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
- Department of Oncology, University of Cambridge, Cambridge, UK
- Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, UK
| | | | - Hannah Clayton
- Department of Oncology, University of Cambridge, Cambridge, UK
- Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, UK
| | - Sarah Burge
- Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, UK
| | - Mireia Crispin-Ortuzar
- Department of Oncology, University of Cambridge, Cambridge, UK.
- Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge, UK.
| |
Collapse
|
4
|
Chen J, Larsson L, Swarbrick A, Lundeberg J. Spatial landscapes of cancers: insights and opportunities. Nat Rev Clin Oncol 2024:10.1038/s41571-024-00926-7. [PMID: 39043872 DOI: 10.1038/s41571-024-00926-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/28/2024] [Indexed: 07/25/2024]
Abstract
Solid tumours comprise many different cell types organized in spatially structured arrangements, with substantial intratumour and intertumour heterogeneity. Advances in spatial profiling technologies over the past decade hold promise to capture the complexity of these cellular architectures to build a holistic view of the intricate molecular mechanisms that shape the tumour ecosystem. Some of these mechanisms act at the cellular scale and are controlled by cell-autonomous programmes or communication between nearby cells, whereas other mechanisms result from coordinated efforts between large networks of cells and extracellular molecules organized into tissues and organs. In this Review we provide insights into the application of single-cell and spatial profiling tools, with a focus on spatially resolved transcriptomic tools developed to understand the cellular architecture of the tumour microenvironment and identify opportunities to use them to improve clinical management of cancers.
Collapse
Affiliation(s)
- Julia Chen
- Cancer Ecosystems Program, Garvan Institute of Medical Research, Sydney, New South Wales, Australia
- School of Clinical Medicine, Faculty of Medicine and Health, University of New South Wales, Sydney, New South Wales, Australia
- Department of Medical Oncology, St George Hospital, Sydney, New South Wales, Australia
| | - Ludvig Larsson
- Department of Gene Technology, KTH Royal Institute of Technology, Science for Life Laboratory, Stockholm, Sweden
| | - Alexander Swarbrick
- Cancer Ecosystems Program, Garvan Institute of Medical Research, Sydney, New South Wales, Australia.
- School of Clinical Medicine, Faculty of Medicine and Health, University of New South Wales, Sydney, New South Wales, Australia.
| | - Joakim Lundeberg
- Department of Gene Technology, KTH Royal Institute of Technology, Science for Life Laboratory, Stockholm, Sweden.
| |
Collapse
|
5
|
Hoang DT, Dinstag G, Shulman ED, Hermida LC, Ben-Zvi DS, Elis E, Caley K, Sammut SJ, Sinha S, Sinha N, Dampier CH, Stossel C, Patil T, Rajan A, Lassoued W, Strauss J, Bailey S, Allen C, Redman J, Beker T, Jiang P, Golan T, Wilkinson S, Sowalsky AG, Pine SR, Caldas C, Gulley JL, Aldape K, Aharonov R, Stone EA, Ruppin E. A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics. NATURE CANCER 2024:10.1038/s43018-024-00793-2. [PMID: 38961276 DOI: 10.1038/s43018-024-00793-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 06/06/2024] [Indexed: 07/05/2024]
Abstract
Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an indirect two-step approach consisting of (1) DeepPT, a deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response to targeted and immune therapies from the inferred expression values. We show that DeepPT successfully predicts transcriptomics in all 16 The Cancer Genome Atlas cohorts tested and generalizes well to two independent datasets. ENLIGHT-DeepPT successfully predicts true responders in five independent patient cohorts involving four different treatments spanning six cancer types, with an overall odds ratio of 2.28 and a 39.5% increased response rate among predicted responders versus the baseline rate. Notably, its prediction accuracy, obtained without any training on the treatment data, is comparable to that achieved by directly predicting the response from the images, which requires specific training on the treatment evaluation cohorts.
Collapse
Affiliation(s)
- Danh-Tai Hoang
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia.
| | | | - Eldad D Shulman
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Leandro C Hermida
- Department of Immunology, University of Pittsburgh, Pittsburgh, PA, USA
- Tumor Microenvironment Center, UPMC Hillman Cancer Center, University of Pittsburgh, Pittsburgh, PA, USA
| | | | | | - Katherine Caley
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia
| | - Stephen-John Sammut
- Breast Cancer Now Toby Robins Research Centre, Institute of Cancer Research, London, UK
- The Royal Marsden Hospital NHS Foundation Trust, London, UK
| | - Sanju Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Neelam Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Christopher H Dampier
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Chani Stossel
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Tejas Patil
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Arun Rajan
- Thoracic and GI Malignancies Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Wiem Lassoued
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Julius Strauss
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Shania Bailey
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Clint Allen
- Surgical Oncology Program, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Jason Redman
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Peng Jiang
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Talia Golan
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Scott Wilkinson
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Adam G Sowalsky
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Sharon R Pine
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Carlos Caldas
- School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - James L Gulley
- Genitourinary Malignancy Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Kenneth Aldape
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Eric A Stone
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia.
| | - Eytan Ruppin
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA.
| |
Collapse
|
6
|
Hoang DT, Shulman ED, Turakulov R, Abdullaev Z, Singh O, Campagnolo EM, Lalchungnunga H, Stone EA, Nasrallah MP, Ruppin E, Aldape K. Prediction of DNA methylation-based tumor types from histopathology in central nervous system tumors with deep learning. Nat Med 2024; 30:1952-1961. [PMID: 38760587 DOI: 10.1038/s41591-024-02995-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 04/11/2024] [Indexed: 05/19/2024]
Abstract
Precision in the diagnosis of diverse central nervous system (CNS) tumor types is crucial for optimal treatment. DNA methylation profiles, which capture the methylation status of thousands of individual CpG sites, are state-of-the-art data-driven means to enhance diagnostic accuracy but are also time consuming and not widely available. Here, to address these limitations, we developed Deep lEarning from histoPathoLOgy and methYlation (DEPLOY), a deep learning model that classifies CNS tumors to ten major categories from histopathology. DEPLOY integrates three distinct components: the first classifies CNS tumors directly from slide images ('direct model'), the second initially generates predictions for DNA methylation beta values, which are subsequently used for tumor classification ('indirect model'), and the third classifies tumor types directly from routinely available patient demographics. First, we find that DEPLOY accurately predicts beta values from histopathology images. Second, using a ten-class model trained on an internal dataset of 1,796 patients, we predict the tumor categories in three independent external test datasets including 2,156 patients, achieving an overall accuracy of 95% and balanced accuracy of 91% on samples that are predicted with high confidence. These results showcase the potential future use of DEPLOY to assist pathologists in diagnosing CNS tumors within a clinically relevant short time frame.
Collapse
Affiliation(s)
- Danh-Tai Hoang
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia
| | - Eldad D Shulman
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Rust Turakulov
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Zied Abdullaev
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Omkar Singh
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Emma M Campagnolo
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - H Lalchungnunga
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Eric A Stone
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia
| | - MacLean P Nasrallah
- Division of Neuropathology, Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Eytan Ruppin
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA.
| | - Kenneth Aldape
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA.
| |
Collapse
|
7
|
Levenson RM, Singh Y, Rieck B, Hathaway QA, Farrelly C, Rozenblit J, Prasanna P, Erickson B, Choudhary A, Carlsson G, Sarkar D. Advancing Precision Medicine: Algebraic Topology and Differential Geometry in Radiology and Computational Pathology. J Transl Med 2024; 104:102060. [PMID: 38626875 DOI: 10.1016/j.labinv.2024.102060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 04/08/2024] [Accepted: 04/10/2024] [Indexed: 05/19/2024] Open
Abstract
Precision medicine aims to provide personalized care based on individual patient characteristics, rather than guideline-directed therapies for groups of diseases or patient demographics. Images-both radiology- and pathology-derived-are a major source of information on presence, type, and status of disease. Exploring the mathematical relationship of pixels in medical imaging ("radiomics") and cellular-scale structures in digital pathology slides ("pathomics") offers powerful tools for extracting both qualitative and, increasingly, quantitative data. These analytical approaches, however, may be significantly enhanced by applying additional methods arising from fields of mathematics such as differential geometry and algebraic topology that remain underexplored in this context. Geometry's strength lies in its ability to provide precise local measurements, such as curvature, that can be crucial for identifying abnormalities at multiple spatial levels. These measurements can augment the quantitative features extracted in conventional radiomics, leading to more nuanced diagnostics. By contrast, topology serves as a robust shape descriptor, capturing essential features such as connected components and holes. The field of topological data analysis was initially founded to explore the shape of data, with functional network connectivity in the brain being a prominent example. Increasingly, its tools are now being used to explore organizational patterns of physical structures in medical images and digitized pathology slides. By leveraging tools from both differential geometry and algebraic topology, researchers and clinicians may be able to obtain a more comprehensive, multi-layered understanding of medical images and contribute to precision medicine's armamentarium.
Collapse
Affiliation(s)
- Richard M Levenson
- Department of Pathology and Laboratory Medicine, University of California Davis, Davis, California.
| | - Yashbir Singh
- Department of Radiology, Mayo Clinic, Rochester, Minnesota.
| | - Bastian Rieck
- Helmholtz Munich and Technical University of Munich, Munich, Germany
| | - Quincy A Hathaway
- Department of Medical Education, West Virginia University, Morgantown, West Virginia
| | | | | | - Prateek Prasanna
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | | | | | - Gunnar Carlsson
- Department of Mathematics, Stanford University, Stanford, California
| | - Deepa Sarkar
- Institute of Genomic Health, Ichan school of Medicine, Mount Sinai, New York
| |
Collapse
|
8
|
Rossi M, Radisky DC. Multiplex Digital Spatial Profiling in Breast Cancer Research: State-of-the-Art Technologies and Applications across the Translational Science Spectrum. Cancers (Basel) 2024; 16:1615. [PMID: 38730568 PMCID: PMC11083340 DOI: 10.3390/cancers16091615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Revised: 04/17/2024] [Accepted: 04/21/2024] [Indexed: 05/13/2024] Open
Abstract
While RNA sequencing and multi-omic approaches have significantly advanced cancer diagnosis and treatment, their limitation in preserving critical spatial information has been a notable drawback. This spatial context is essential for understanding cellular interactions and tissue dynamics. Multiplex digital spatial profiling (MDSP) technologies overcome this limitation by enabling the simultaneous analysis of transcriptome and proteome data within the intact spatial architecture of tissues. In breast cancer research, MDSP has emerged as a promising tool, revealing complex biological questions related to disease evolution, identifying biomarkers, and discovering drug targets. This review highlights the potential of MDSP to revolutionize clinical applications, ranging from risk assessment and diagnostics to prognostics, patient monitoring, and the customization of treatment strategies, including clinical trial guidance. We discuss the major MDSP techniques, their applications in breast cancer research, and their integration in clinical practice, addressing both their potential and current limitations. Emphasizing the strategic use of MDSP in risk stratification for women with benign breast disease, we also highlight its transformative potential in reshaping the landscape of breast cancer research and treatment.
Collapse
Affiliation(s)
| | - Derek C. Radisky
- Department of Cancer Biology, Mayo Clinic, Jacksonville, FL 32224, USA;
| |
Collapse
|
9
|
Haag F, Hertel A, Tharmaseelan H, Kuru M, Haselmann V, Brochhausen C, Schönberg SO, Froelich MF. Imaging-based characterization of tumoral heterogeneity for personalized cancer treatment. ROFO-FORTSCHR RONTG 2024; 196:262-272. [PMID: 37944935 DOI: 10.1055/a-2175-4622] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2023]
Abstract
With personalized tumor therapy, understanding and addressing the heterogeneity of malignant tumors is becoming increasingly important. Heterogeneity can be found within one lesion (intralesional) and between several tumor lesions emerging from one primary tumor (interlesional). The heterogeneous tumor cells may show a different response to treatment due to their biology, which in turn influences the outcome of the affected patients and the choice of therapeutic agents. Therefore, both intra- and interlesional heterogeneity should be addressed at the diagnostic stage. While genetic and biological heterogeneity are important parameters in molecular tumor characterization and in histopathology, they are not yet addressed routinely in medical imaging. This article summarizes the recently established markers for tumor heterogeneity in imaging as well as heterogeneous/mixed response to therapy. Furthermore, a look at emerging markers is given. The ultimate goal of this overview is to provide comprehensive understanding of tumor heterogeneity and its implications for radiology and for communication with interdisciplinary teams in oncology. KEY POINTS:: · Tumor heterogeneity can be described within one lesion (intralesional) or between several lesions (interlesional).. · The heterogeneous biology of tumor cells can lead to a mixed therapeutic response and should be addressed in diagnostics and the therapeutic regime.. · Quantitative image diagnostics can be enhanced using AI, improved histopathological methods, and liquid profiling in the future..
Collapse
Affiliation(s)
- Florian Haag
- Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim, Germany
| | - Alexander Hertel
- Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim, Germany
| | - Hishan Tharmaseelan
- Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim, Germany
| | - Mustafa Kuru
- Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim, Germany
| | - Verena Haselmann
- Institute of Clinical Chemistry, Medical Faculty Mannheim of the University of Heidelberg, University Hospital Mannheim, Germany
| | - Christoph Brochhausen
- Institute of Pathology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany
| | - Stefan O Schönberg
- Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim, Germany
| | - Matthias F Froelich
- Institute of Clinical Radiology and Nuclear Medicine, University Medical Centre Mannheim, Germany
| |
Collapse
|
10
|
Feng X, Shu W, Li M, Li J, Xu J, He M. Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview. J Transl Med 2024; 22:131. [PMID: 38310237 PMCID: PMC10837897 DOI: 10.1186/s12967-024-04915-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/20/2024] [Indexed: 02/05/2024] Open
Abstract
The capability to gather heterogeneous data, alongside the increasing power of artificial intelligence to examine it, leading a revolution in harnessing multimodal data in the life sciences. However, most approaches are limited to unimodal data, leaving integrated approaches across modalities relatively underdeveloped in computational pathology. Pathogenomics, as an invasive method to integrate advanced molecular diagnostics from genomic data, morphological information from histopathological imaging, and codified clinical data enable the discovery of new multimodal cancer biomarkers to propel the field of precision oncology in the coming decade. In this perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods in pathogenomics. It includes correlation between the pathological and genomic profile of cancer, fusion of histology, and genomics profile of cancer. We also present challenges, opportunities, and avenues for future work.
Collapse
Affiliation(s)
- Xiaobing Feng
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Wen Shu
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Mingya Li
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Junyu Li
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Junyao Xu
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Min He
- College of Electrical and Information Engineering, Hunan University, Changsha, China.
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China.
| |
Collapse
|
11
|
Yan S, Guo Y, Lin L, Zhang W. Breaks for Precision Medicine in Cancer: Development and Prospects of Spatiotemporal Transcriptomics. Cancer Biother Radiopharm 2024; 39:35-45. [PMID: 38181185 DOI: 10.1089/cbr.2023.0116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2024] Open
Abstract
With the development of the social economy and the deepening understanding of cancer, cancer has become a significant cause of death, threatening human health. Although researchers have made rapid progress in cancer treatment strategies in recent years, the overall survival of cancer patients is still not optimistic. Therefore, it is essential to reveal the spatial pattern of gene expression, spatial heterogeneity of cell populations, microenvironment interactions, and other aspects of cancer. Spatiotemporal transcriptomics can help analyze the mechanism of cancer occurrence and development, greatly help precise cancer treatment, and improve clinical prognosis. Here, we review the integration strategies of single-cell RNA sequencing and spatial transcriptomics data, summarize the recent advances in spatiotemporal transcriptomics in cancer studies, and discuss the combined application of spatial multiomics, which provides new directions and strategies for the precise treatment and clinical prognosis of cancer.
Collapse
Affiliation(s)
- Shiqi Yan
- Department of Laboratory Medicine, The Third Xiangya Hospital, Central South University, Changsha, Hunan, People's Republic of China
- Department of Laboratory Medicine, Xiangya School of Medicine, Central South University, Changsha, Hunan, People's Republic of China
| | - Yilin Guo
- Department of Laboratory Medicine, The Third Xiangya Hospital, Central South University, Changsha, Hunan, People's Republic of China
- Department of Laboratory Medicine, Xiangya School of Medicine, Central South University, Changsha, Hunan, People's Republic of China
| | - Lizhong Lin
- Department of Clinical Laboratory, The First People's Hospital of Changde City, Changde, Hunan, People's Republic of China
| | - Wenling Zhang
- Department of Laboratory Medicine, The Third Xiangya Hospital, Central South University, Changsha, Hunan, People's Republic of China
- Department of Laboratory Medicine, Xiangya School of Medicine, Central South University, Changsha, Hunan, People's Republic of China
| |
Collapse
|
12
|
Kloosterman A, Baars I, Högberg B. An error correction strategy for image reconstruction by DNA sequencing microscopy. NATURE COMPUTATIONAL SCIENCE 2024; 4:119-127. [PMID: 38253805 PMCID: PMC10899105 DOI: 10.1038/s43588-023-00589-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 12/13/2023] [Indexed: 01/24/2024]
Abstract
By pairing adjacent molecules in situ and then mapping these pairs, DNA microscopy could substantially reduce the workload in spatial omics methods by directly inferring geometry from sequencing data alone. However, experimental artifacts can lead to errors in the adjacency data, which distort the spatial reconstruction. Here we describe a method to correct two such errors: spurious crosslinks formed between any two nodes, and fused nodes that are formed out of multiple molecules. We build on the principle that spatially close molecules should be connected and show that these errors violate this principle, allowing for their detection and correction. Our method corrects errors in simulated data, even in the presence of up to 20% errors, and proves to be more efficient at removing errors from experimental data than a read count filter. Integrating this method in DNA microscopy will substantially improve the accuracy of spatial reconstructions with lower data loss.
Collapse
Affiliation(s)
- Alexander Kloosterman
- Department of Medical Biochemistry and Biophysics, Karolinska Institutet, Stockholm, Sweden
| | - Igor Baars
- Department of Medical Biochemistry and Biophysics, Karolinska Institutet, Stockholm, Sweden
| | - Björn Högberg
- Department of Medical Biochemistry and Biophysics, Karolinska Institutet, Stockholm, Sweden.
| |
Collapse
|
13
|
Song S, Mohsin E, Zhang R, Kuznetsov A, Shen L, Grossman RL, Weber CR, Khan AA. ATAT: Automated Tissue Alignment and Traversal in Spatial Transcriptomics with Self-Supervised Learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.08.570839. [PMID: 38106010 PMCID: PMC10723486 DOI: 10.1101/2023.12.08.570839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2023]
Abstract
Spatial transcriptomics (ST) has enhanced RNA analysis in tissue biopsies, but interpreting these data is challenging without expert input. We present Automated Tissue Alignment and Traversal (ATAT), a novel computational framework designed to enhance ST analysis in the context of multiple and complex tissue architectures and morphologies, such as those found in biopsies of the gastrointestinal tract. ATAT utilizes self-supervised contrastive learning on hematoxylin and eosin (H&E) stained images to automate the alignment and traversal of ST data. This approach addresses a critical gap in current ST analysis methodologies, which rely heavily on manual annotation and pathologist expertise to delineate regions of interest for accurate gene expression modeling. Our framework not only streamlines the alignment of multiple ST samples, but also demonstrates robustness in modeling gene expression transitions across specific regions. Additionally, we highlight the ability of ATAT to traverse complex tissue topologies in real-world cases from various individuals and conditions. Our method successfully elucidates differences in immune infiltration patterns across the intestinal wall, enabling the modeling of transcriptional changes across histological layers. We show that ATAT achieves comparable performance to the state-of-the-art method, while alleviating the burden of manual annotation and enabling alignment of tissue samples with complex morphologies.
Collapse
Affiliation(s)
- Steven Song
- Department of Computer Science, University of Chicago, IL 60637, USA
- Interdisciplinary Scientist Training Program, University of Chicago, Chicago, IL 60637, USA
| | - Emaan Mohsin
- Department of Pathology, University of Chicago, Chicago, IL 60637, USA
| | - Renyu Zhang
- Department of Computer Science, University of Chicago, IL 60637, USA
| | - Andrey Kuznetsov
- Department of Pathology, University of Chicago, Chicago, IL 60637, USA
| | - Le Shen
- Department of Pathology, University of Chicago, Chicago, IL 60637, USA
| | - Robert L. Grossman
- Department of Computer Science, University of Chicago, IL 60637, USA
- Department of Medicine, University of Chicago, Chicago, IL 60637, USA
| | | | - Aly A. Khan
- Department of Pathology, University of Chicago, Chicago, IL 60637, USA
- Committee on Immunology, University of Chicago, Chicago, IL 60637, USA
- Institute for Population and Precision Health, University of Chicago, Chicago, IL 60637, USA
- Department of Family Medicine, University of Chicago, Chicago, IL 60637, USA
| |
Collapse
|
14
|
Davis S, Scott C, Oetjen J, Charles PD, Kessler BM, Ansorge O, Fischer R. Deep topographic proteomics of a human brain tumour. Nat Commun 2023; 14:7710. [PMID: 38001067 PMCID: PMC10673928 DOI: 10.1038/s41467-023-43520-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 11/13/2023] [Indexed: 11/26/2023] Open
Abstract
The spatial organisation of cellular protein expression profiles within tissue determines cellular function and is key to understanding disease pathology. To define molecular phenotypes in the spatial context of tissue, there is a need for unbiased, quantitative technology capable of mapping proteomes within tissue structures. Here, we present a workflow for spatially-resolved, quantitative proteomics of tissue that generates maps of protein abundance across tissue slices derived from a human atypical teratoid-rhabdoid tumour at three spatial resolutions, the highest being 40 µm, to reveal distinct abundance patterns of thousands of proteins. We employ spatially-aware algorithms that do not require prior knowledge of the fine tissue structure to detect proteins and pathways with spatial abundance patterns and correlate proteins in the context of tissue heterogeneity and cellular features such as extracellular matrix or proximity to blood vessels. We identify PYGL, ASPH and CD45 as spatial markers for tumour boundary and reveal immune response-driven, spatially-organised protein networks of the extracellular tumour matrix. Overall, we demonstrate spatially-aware deep proteo-phenotyping of tissue heterogeneity, to re-define understanding tissue biology and pathology at the molecular level.
Collapse
Affiliation(s)
- Simon Davis
- Target Discovery Institute, Centre for Medicines Discovery, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK
- Chinese Academy for Medical Sciences Oxford Institute, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK
| | - Connor Scott
- Academic Unit of Neuropathology, Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, OX3 9DU, UK
| | - Janina Oetjen
- Bruker Daltonics GmbH & Co. KG, Fahrenheitstraße 4, 28359, Bremen, Germany
| | - Philip D Charles
- Target Discovery Institute, Centre for Medicines Discovery, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK
- Big Data Institute, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK
| | - Benedikt M Kessler
- Target Discovery Institute, Centre for Medicines Discovery, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK
- Chinese Academy for Medical Sciences Oxford Institute, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK
| | - Olaf Ansorge
- Academic Unit of Neuropathology, Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, OX3 9DU, UK
| | - Roman Fischer
- Target Discovery Institute, Centre for Medicines Discovery, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK.
- Chinese Academy for Medical Sciences Oxford Institute, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, UK.
| |
Collapse
|
15
|
Berl A, Shir-az O, Genish I, Biran H, Mann D, Singh A, Wise J, Kravtsov V, Kidron D, Golberg A, Vitkin E, Yakhini Z, Shalom A. Exploring multisite heterogeneity of human basal cell carcinoma proteome and transcriptome. PLoS One 2023; 18:e0293744. [PMID: 37948379 PMCID: PMC10637653 DOI: 10.1371/journal.pone.0293744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Accepted: 10/18/2023] [Indexed: 11/12/2023] Open
Abstract
Basal cell carcinoma (BCC) is the most common type of skin cancer. Due to multiple, potential underlying molecular tumor aberrations, clinical treatment protocols are not well-defined. This study presents multisite molecular heterogeneity profiles of human BCC based on RNA and proteome profiling. Three areas from lesions excised from 9 patients were analyzed. The focus was gene expression profiles based on proteome and RNA measurements of intra-tumor heterogeneity from the same patient and inter-tumor heterogeneity in nodular, infiltrative, and superficial BCC tumor subtypes from different patients. We observed significant overlap in intra- and inter-tumor variability of proteome and RNA expression profiles, showing significant multisite heterogeneity of protein expression in the BCC tumors. Inter-subtype analysis has also identified unique proteins for each BCC subtype. This profiling leads to a deeper understanding of BCC molecular heterogeneity and potentially contributes to developing new sampling tools for personalized diagnostics therapeutic approaches to BCC.
Collapse
Affiliation(s)
- Ariel Berl
- Department of Plastic Surgery, Meir Medical Center, Kfar Sava, Israel, Affiliated with the Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Ofir Shir-az
- Department of Plastic Surgery, Meir Medical Center, Kfar Sava, Israel, Affiliated with the Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Ilai Genish
- Efi Arazi School of Computer Science, Reichman University, Herzliya, Israel
| | - Hadas Biran
- Department of Computer Science, Technion - Israel Institute of Technology, Haifa, Israel
| | - Din Mann
- Department of Plastic Surgery, Meir Medical Center, Kfar Sava, Israel, Affiliated with the Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Amrita Singh
- Department of Environmental Studies, Porter School of Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Julia Wise
- Department of Environmental Studies, Porter School of Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Vladimir Kravtsov
- Department of Pathology, Meir Medical Center, Kfar Sava, Israel, Affiliated with the Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Debora Kidron
- Department of Pathology, Meir Medical Center, Kfar Sava, Israel, Affiliated with the Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Alexander Golberg
- Department of Environmental Studies, Porter School of Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Edward Vitkin
- Efi Arazi School of Computer Science, Reichman University, Herzliya, Israel
| | - Zohar Yakhini
- Efi Arazi School of Computer Science, Reichman University, Herzliya, Israel
- Department of Computer Science, Technion - Israel Institute of Technology, Haifa, Israel
| | - Avshalom Shalom
- Department of Plastic Surgery, Meir Medical Center, Kfar Sava, Israel, Affiliated with the Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|
16
|
Fatemi MY, Lu Y, Sharma C, Feng E, Azher ZL, Diallo AB, Srinivasan G, Rosner GM, Pointer KB, Christensen BC, Salas LA, Tsongalis GJ, Palisoul SM, Perreard L, Kolling FW, Vaickus LJ, Levy JJ. Feasibility of Inferring Spatial Transcriptomics from Single-Cell Histological Patterns for Studying Colon Cancer Tumor Heterogeneity. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.10.09.23296701. [PMID: 37873186 PMCID: PMC10593064 DOI: 10.1101/2023.10.09.23296701] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
Background Spatial transcriptomics involves studying the spatial organization of gene expression within tissues, offering insights into the molecular diversity of tumors. While spatial gene expression is commonly amalgamated from 1-10 cells across 50-micron spots, recent methods have demonstrated the capability to disaggregate this information at subspot resolution by leveraging both expression and histological patterns. However, elucidating such information from histology alone presents a significant challenge but if solved can better permit spatial molecular analysis at cellular resolution for instances where Visium data is not available, reducing study costs. This study explores integrating single-cell histological and transcriptomic data to infer spatial mRNA expression patterns in whole slide images collected from a cohort of stage pT3 colorectal cancer patients. A cell graph neural network algorithm was developed to align histological information extracted from detected cells with single cell RNA patterns through optimal transport methods, facilitating the analysis of cellular groupings and gene relationships. This approach leveraged spot-level expression as an intermediary to co-map histological and transcriptomic information at the single-cell level. Results Our study demonstrated that single-cell transcriptional heterogeneity within a spot could be predicted from histological markers extracted from cells detected within a spot. Furthermore, our model exhibited proficiency in delineating overarching gene expression patterns across whole-slide images. This approach compared favorably to traditional patch-based computer vision methods as well as other methods which did not incorporate single cell expression during the model fitting procedures. Topological nuances of single-cell expression within a Visium spot were preserved using the developed methodology. Conclusion This innovative approach augments the resolution of spatial molecular assays utilizing histology as a sole input through synergistic co-mapping of histological and transcriptomic datasets at the single-cell level, anchored by spatial transcriptomics. While initial results are promising, they warrant rigorous validation. This includes collaborating with pathologists for precise spatial identification of distinct cell types and utilizing sophisticated assays, such as Xenium, to attain deeper subcellular insights.
Collapse
|
17
|
Fatemi MY, Lu Y, Diallo AB, Srinivasan G, Azher ZL, Christensen BC, Salas LA, Tsongalis GJ, Palisoul SM, Perreard L, Kolling FW, Vaickus LJ, Levy JJ. The Overlooked Role of Specimen Preparation in Bolstering Deep Learning-Enhanced Spatial Transcriptomics Workflows. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.10.09.23296700. [PMID: 37873287 PMCID: PMC10593052 DOI: 10.1101/2023.10.09.23296700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
The application of deep learning methods to spatial transcriptomics has shown promise in unraveling the complex relationships between gene expression patterns and tissue architecture as they pertain to various pathological conditions. Deep learning methods that can infer gene expression patterns directly from tissue histomorphology can expand the capability to discern spatial molecular markers within tissue slides. However, current methods utilizing these techniques are plagued by substantial variability in tissue preparation and characteristics, which can hinder the broader adoption of these tools. Furthermore, training deep learning models using spatial transcriptomics on small study cohorts remains a costly endeavor. Necessitating novel tissue preparation processes enhance assay reliability, resolution, and scalability. This study investigated the impact of an enhanced specimen processing workflow for facilitating a deep learning-based spatial transcriptomics assessment. The enhanced workflow leveraged the flexibility of the Visium CytAssist assay to permit automated H&E staining (e.g., Leica Bond) of tissue slides, whole-slide imaging at 40x-resolution, and multiplexing of tissue sections from multiple patients within individual capture areas for spatial transcriptomics profiling. Using a cohort of thirteen pT3 stage colorectal cancer (CRC) patients, we compared the efficacy of deep learning models trained on slide prepared using an enhanced workflow as compared to the traditional workflow which leverages manual tissue staining and standard imaging of tissue slides. Leveraging Inceptionv3 neural networks, we aimed to predict gene expression patterns across matched serial tissue sections, each stemming from a distinct workflow but aligned based on persistent histological structures. Findings indicate that the enhanced workflow considerably outperformed the traditional spatial transcriptomics workflow. Gene expression profiles predicted from enhanced tissue slides also yielded expression patterns more topologically consistent with the ground truth. This led to enhanced statistical precision in pinpointing biomarkers associated with distinct spatial structures. These insights can potentially elevate diagnostic and prognostic biomarker detection by broadening the range of spatial molecular markers linked to metastasis and recurrence. Future endeavors will further explore these findings to enrich our comprehension of various diseases and uncover molecular pathways with greater nuance. Combining deep learning with spatial transcriptomics provides a compelling avenue to enrich our understanding of tumor biology and improve clinical outcomes. For results of the highest fidelity, however, effective specimen processing is crucial, and fostering collaboration between histotechnicians, pathologists, and genomics specialists is essential to herald this new era in spatial transcriptomics-driven cancer research.
Collapse
|
18
|
Seal S, Bitler BG, Ghosh D. SMASH: Scalable Method for Analyzing Spatial Heterogeneity of genes in spatial transcriptomics data. PLoS Genet 2023; 19:e1010983. [PMID: 37862362 PMCID: PMC10619839 DOI: 10.1371/journal.pgen.1010983] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 11/01/2023] [Accepted: 09/19/2023] [Indexed: 10/22/2023] Open
Abstract
In high-throughput spatial transcriptomics (ST) studies, it is of great interest to identify the genes whose level of expression in a tissue covaries with the spatial location of cells/spots. Such genes, also known as spatially variable genes (SVGs), can be crucial to the biological understanding of both structural and functional characteristics of complex tissues. Existing methods for detecting SVGs either suffer from huge computational demand or significantly lack statistical power. We propose a non-parametric method termed SMASH that achieves a balance between the above two problems. We compare SMASH with other existing methods in varying simulation scenarios demonstrating its superior statistical power and robustness. We apply the method to four ST datasets from different platforms uncovering interesting biological insights.
Collapse
Affiliation(s)
- Souvik Seal
- Department of Public Health Sciences, School of Medicine, Medical University of South Carolina, Charleston, South Carolina, United States of America
| | - Benjamin G. Bitler
- Department of Obstetrics and Gynecology, School of Medicine, University of Colorado Denver Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Debashis Ghosh
- Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Denver Anschutz Medical Campus, Aurora, Colorado, United States of America
| |
Collapse
|
19
|
Dent A, Faust K, Lam K, Alhangari N, Leon AJ, Tsang Q, Kamil ZS, Gao A, Pal P, Lheureux S, Oza A, Diamandis P. HAVOC: Small-scale histomic mapping of cancer biodiversity across large tissue distances using deep neural networks. SCIENCE ADVANCES 2023; 9:eadg1894. [PMID: 37774029 PMCID: PMC10541015 DOI: 10.1126/sciadv.adg1894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Accepted: 08/28/2023] [Indexed: 10/01/2023]
Abstract
Intratumoral heterogeneity can wreak havoc on current precision medicine strategies because of challenges in sufficient sampling of geographically separated areas of biodiversity distributed across centimeter-scale tumor distances. To address this gap, we developed a deep learning pipeline that leverages histomorphologic fingerprints of tissue to create "Histomic Atlases of Variation Of Cancers" (HAVOC). Using a number of objective molecular readouts, we demonstrate that HAVOC can define regional cancer boundaries with distinct biology. Using larger tumor specimens, we show that HAVOC can map biodiversity even across multiple tissue sections. By guiding profiling of 19 partitions across six high-grade gliomas, HAVOC revealed that distinct differentiation states can often coexist and be regionally distributed within these tumors. Last, to highlight generalizability, we benchmark HAVOC on additional tumor types. Together, we establish HAVOC as a versatile tool to generate small-scale maps of tissue heterogeneity and guide regional deployment of molecular resources to relevant biodiverse niches.
Collapse
Affiliation(s)
- Anglin Dent
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
| | - Kevin Faust
- Department of Computer Science, University of Toronto, 40 St. George Street, Toronto, ON M5S 2E4, Canada
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - K. H. Brian Lam
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Narges Alhangari
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
| | - Alberto J. Leon
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Queenie Tsang
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Zaid Saeed Kamil
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
| | - Andrew Gao
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
| | - Prodipto Pal
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
| | - Stephanie Lheureux
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Amit Oza
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Phedias Diamandis
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
- Department of Medical Biophysics, University of Toronto, 101 College St, Toronto, ON M5G 1L7, Canada
| |
Collapse
|
20
|
Hoang DT, Dinstag G, Hermida LC, Ben-Zvi DS, Elis E, Caley K, Sammut SJ, Sinha S, Sinha N, Dampier CH, Stossel C, Patil T, Rajan A, Lassoued W, Strauss J, Bailey S, Allen C, Redman J, Beker T, Jiang P, Golan T, Wilkinson S, Sowalsky AG, Pine SR, Caldas C, Gulley JL, Aldape K, Aharonov R, Stone EA, Ruppin E. Prediction of cancer treatment response from histopathology images through imputed transcriptomics. RESEARCH SQUARE 2023:rs.3.rs-3193270. [PMID: 37790315 PMCID: PMC10543028 DOI: 10.21203/rs.3.rs-3193270/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin (H&E)-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an approach for predicting response to multiple targeted and immunotherapies from H&E-slides. In difference from existing approaches that aim to predict treatment response directly from the slides, ENLIGHT-DeepPT is an indirect two-step approach consisting of (1) DeepPT, a new deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response based on the DeepPT inferred expression values. DeepPT successfully predicts transcriptomics in all 16 TCGA cohorts tested and generalizes well to two independent datasets. Our key contribution is showing that ENLIGHT-DeepPT successfully predicts true responders in five independent patients' cohorts involving four different treatments spanning six cancer types with an overall odds ratio of 2.44, increasing the baseline response rate by 43.47% among predicted responders, without the need for any treatment data for training. Furthermore, its prediction accuracy on these datasets is comparable to a supervised approach predicting the response directly from the images, which needs to be trained and tested on the same cohort. ENLIGHT-DeepPT future application could provide clinicians with rapid treatment recommendations to an array of different therapies and importantly, may contribute to advancing precision oncology in developing countries.
Collapse
Affiliation(s)
- Danh-Tai Hoang
- Biological Data Science Institute, College of Science, Australian National University, Canberra, ACT, Australia
| | | | - Leandro C. Hermida
- Department of Immunology, University of Pittsburgh, Pittsburgh, PA, USA
- Tumor Microenvironment Center, UPMC Hillman Cancer Center, University of Pittsburgh, Pittsburgh, PA, USA
| | | | | | - Katherine Caley
- Biological Data Science Institute, College of Science, Australian National University, Canberra, ACT, Australia
| | - Stephen-John Sammut
- Breast Cancer Now Toby Robins Research Centre, The Institute of Cancer Research, London, United Kingdom
- The Royal Marsden Hospital NHS Foundation Trust, London, United Kingdom
- Cancer Research UK Cambridge Institute, University of Cambridge, Li Ka Shing Centre, Cambridge, UK
| | - Sanju Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Neelam Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Christopher H. Dampier
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Chani Stossel
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Tejas Patil
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Arun Rajan
- Thoracic and GI Malignancies Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Wiem Lassoued
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Julius Strauss
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Shania Bailey
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Clint Allen
- Surgical Oncology Program, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Jason Redman
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Peng Jiang
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Talia Golan
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Scott Wilkinson
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Adam G. Sowalsky
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Sharon R. Pine
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Carlos Caldas
- Cancer Research UK Cambridge Institute, University of Cambridge, Li Ka Shing Centre, Cambridge, UK
| | - James L. Gulley
- Genitourinary Malignancy Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Kenneth Aldape
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Eric A. Stone
- Biological Data Science Institute, College of Science, Australian National University, Canberra, ACT, Australia
| | - Eytan Ruppin
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| |
Collapse
|
21
|
Du J, An ZJ, Huang ZF, Yang YC, Zhang MH, Fu XH, Shi WY, Hou J. Novel insights from spatial transcriptome analysis in solid tumors. Int J Biol Sci 2023; 19:4778-4792. [PMID: 37781515 PMCID: PMC10539699 DOI: 10.7150/ijbs.83098] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 08/03/2023] [Indexed: 10/03/2023] Open
Abstract
Since its first application in 2016, spatial transcriptomics has become a rapidly evolving technology in recent years. Spatial transcriptomics enables transcriptomic data to be acquired from intact tissue sections and provides spatial distribution information and remedies the disadvantage of single-cell RNA sequencing (scRNA-seq), whose data lack spatially resolved information. Presently, spatial transcriptomics has been widely applied to various tissue types, especially for the study of tumor heterogeneity. In this review, we provide a summary of the research progress in utilizing spatial transcriptomics to investigate tumor heterogeneity and the microenvironment with a focus on solid tumors. We summarize the research breakthroughs in various fields and perspectives due to the application of spatial transcriptomics, including cell clustering and interaction, cellular metabolism, gene expression, immune cell programs and combination with other techniques. As a combination of multiple transcriptomics, single-cell multiomics shows its superiority and validity in single-cell analysis. We also discuss the application prospect of single-cell multiomics, and we believe that with the progress of data integration from various transcriptomics, a multilayered subcellular landscape will be revealed.
Collapse
Affiliation(s)
- Jun Du
- Department of Hematology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, 160 Pujiang Road, Shanghai, 200127, China
| | - Zhi-Jie An
- School of Medicine, Shanghai Jiao Tong University, Shanghai, 200025, China
| | - Zou-Fang Huang
- Ganzhou Key Laboratory of Hematology, Department of Hematology, The First Affiliated Hospital of Gannan Medical University, Ganzhou, Jiangxi, 341000, China
| | - Yu-Chen Yang
- School of Medicine, Shanghai Jiao Tong University, Shanghai, 200025, China
| | - Ming-Hui Zhang
- School of Medicine, Shanghai Jiao Tong University, Shanghai, 200025, China
| | - Xue-Hang Fu
- Department of Hematology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, 160 Pujiang Road, Shanghai, 200127, China
| | - Wei-Yang Shi
- Ministry of Education Key Laboratory of Marine Genetics and Breeding, College of Marine Life Sciences, Ocean University of China, Qingdao 266003, China
| | - Jian Hou
- Department of Hematology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, 160 Pujiang Road, Shanghai, 200127, China
| |
Collapse
|
22
|
Halawani R, Buchert M, Chen YPP. Deep learning exploration of single-cell and spatially resolved cancer transcriptomics to unravel tumour heterogeneity. Comput Biol Med 2023; 164:107274. [PMID: 37506451 DOI: 10.1016/j.compbiomed.2023.107274] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2023] [Revised: 07/03/2023] [Accepted: 07/16/2023] [Indexed: 07/30/2023]
Abstract
Tumour heterogeneity is one of the critical confounding aspects in decoding tumour growth. Malignant cells display variations in their gene transcription profiles and mutation spectra even when originating from a single progenitor cell. Single-cell and spatial transcriptomics sequencing have recently emerged as key technologies for unravelling tumour heterogeneity. Single-cell sequencing promotes individual cell-type identification through transcriptome-wide gene expression measurements of each cell. Spatial transcriptomics facilitates identification of cell-cell interactions and the structural organization of heterogeneous cells within a tumour tissue through associating spatial RNA abundance of cells at distinct spots in the tissue section. However, extracting features and analyzing single-cell and spatial transcriptomics data poses challenges. Single-cell transcriptome data is extremely noisy and its sparse nature and dropouts can lead to misinterpretation of gene expression and the misclassification of cell types. Deep learning predictive power can overcome data challenges, provide high-resolution analysis and enhance precision oncology applications that involve early cancer prognosis, diagnosis, patient survival estimation and anti-cancer therapy planning. In this paper, we provide a background to and review of the recent progress of deep learning frameworks to investigate tumour heterogeneity using both single-cell and spatial transcriptomics data types.
Collapse
Affiliation(s)
- Raid Halawani
- Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia
| | - Michael Buchert
- School of Cancer Medicine, La Trobe University, Melbourne, Victoria, Australia; Olivia Newton-John Cancer Research Institute, Melbourne, Victoria, Australia
| | - Yi-Ping Phoebe Chen
- Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia.
| |
Collapse
|
23
|
Davri A, Birbas E, Kanavos T, Ntritsos G, Giannakeas N, Tzallas AT, Batistatou A. Deep Learning for Lung Cancer Diagnosis, Prognosis and Prediction Using Histological and Cytological Images: A Systematic Review. Cancers (Basel) 2023; 15:3981. [PMID: 37568797 PMCID: PMC10417369 DOI: 10.3390/cancers15153981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 07/27/2023] [Accepted: 08/03/2023] [Indexed: 08/13/2023] Open
Abstract
Lung cancer is one of the deadliest cancers worldwide, with a high incidence rate, especially in tobacco smokers. Lung cancer accurate diagnosis is based on distinct histological patterns combined with molecular data for personalized treatment. Precise lung cancer classification from a single H&E slide can be challenging for a pathologist, requiring most of the time additional histochemical and special immunohistochemical stains for the final pathology report. According to WHO, small biopsy and cytology specimens are the available materials for about 70% of lung cancer patients with advanced-stage unresectable disease. Thus, the limited available diagnostic material necessitates its optimal management and processing for the completion of diagnosis and predictive testing according to the published guidelines. During the new era of Digital Pathology, Deep Learning offers the potential for lung cancer interpretation to assist pathologists' routine practice. Herein, we systematically review the current Artificial Intelligence-based approaches using histological and cytological images of lung cancer. Most of the published literature centered on the distinction between lung adenocarcinoma, lung squamous cell carcinoma, and small cell lung carcinoma, reflecting the realistic pathologist's routine. Furthermore, several studies developed algorithms for lung adenocarcinoma predominant architectural pattern determination, prognosis prediction, mutational status characterization, and PD-L1 expression status estimation.
Collapse
Affiliation(s)
- Athena Davri
- Department of Pathology, Faculty of Medicine, School of Health Sciences, University of Ioannina, 45500 Ioannina, Greece;
| | - Effrosyni Birbas
- Faculty of Medicine, School of Health Sciences, University of Ioannina, 45110 Ioannina, Greece; (E.B.); (T.K.)
| | - Theofilos Kanavos
- Faculty of Medicine, School of Health Sciences, University of Ioannina, 45110 Ioannina, Greece; (E.B.); (T.K.)
| | - Georgios Ntritsos
- Department of Hygiene and Epidemiology, Faculty of Medicine, School of Health Sciences, University of Ioannina, 45110 Ioannina, Greece;
- Department of Informatics and Telecommunications, University of Ioannina, 47100 Arta, Greece;
| | - Nikolaos Giannakeas
- Department of Informatics and Telecommunications, University of Ioannina, 47100 Arta, Greece;
| | - Alexandros T. Tzallas
- Department of Informatics and Telecommunications, University of Ioannina, 47100 Arta, Greece;
| | - Anna Batistatou
- Department of Pathology, Faculty of Medicine, School of Health Sciences, University of Ioannina, 45500 Ioannina, Greece;
| |
Collapse
|
24
|
Lee M. Recent Advancements in Deep Learning Using Whole Slide Imaging for Cancer Prognosis. Bioengineering (Basel) 2023; 10:897. [PMID: 37627783 PMCID: PMC10451210 DOI: 10.3390/bioengineering10080897] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 07/21/2023] [Accepted: 07/27/2023] [Indexed: 08/26/2023] Open
Abstract
This review furnishes an exhaustive analysis of the latest advancements in deep learning techniques applied to whole slide images (WSIs) in the context of cancer prognosis, focusing specifically on publications from 2019 through 2023. The swiftly maturing field of deep learning, in combination with the burgeoning availability of WSIs, manifests significant potential in revolutionizing the predictive modeling of cancer prognosis. In light of the swift evolution and profound complexity of the field, it is essential to systematically review contemporary methodologies and critically appraise their ramifications. This review elucidates the prevailing landscape of this intersection, cataloging major developments, evaluating their strengths and weaknesses, and providing discerning insights into prospective directions. In this paper, a comprehensive overview of the field aims to be presented, which can serve as a critical resource for researchers and clinicians, ultimately enhancing the quality of cancer care outcomes. This review's findings accentuate the need for ongoing scrutiny of recent studies in this rapidly progressing field to discern patterns, understand breakthroughs, and navigate future research trajectories.
Collapse
Affiliation(s)
- Minhyeok Lee
- School of Electrical and Electronics Engineering, Chung-Ang University, Seoul 06974, Republic of Korea
| |
Collapse
|
25
|
Park H, Jo SH, Lee RH, Macks CP, Ku T, Park J, Lee CW, Hur JK, Sohn CH. Spatial Transcriptomics: Technical Aspects of Recent Developments and Their Applications in Neuroscience and Cancer Research. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2023; 10:e2206939. [PMID: 37026425 PMCID: PMC10238226 DOI: 10.1002/advs.202206939] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 03/10/2023] [Indexed: 06/04/2023]
Abstract
Spatial transcriptomics is a newly emerging field that enables high-throughput investigation of the spatial localization of transcripts and related analyses in various applications for biological systems. By transitioning from conventional biological studies to "in situ" biology, spatial transcriptomics can provide transcriptome-scale spatial information. Currently, the ability to simultaneously characterize gene expression profiles of cells and relevant cellular environment is a paradigm shift for biological studies. In this review, recent progress in spatial transcriptomics and its applications in neuroscience and cancer studies are highlighted. Technical aspects of existing technologies and future directions of new developments (as of March 2023), computational analysis of spatial transcriptome data, application notes in neuroscience and cancer studies, and discussions regarding future directions of spatial multi-omics and their expanding roles in biomedical applications are emphasized.
Collapse
Affiliation(s)
- Han‐Eol Park
- Center for NanomedicineInstitute for Basic ScienceYonsei UniversitySeoul03722Republic of Korea
- Graduate Program in Nanobiomedical EngineeringAdvanced Science InstituteYonsei UniversitySeoul03722Republic of Korea
- School of Biological SciencesSeoul National UniversitySeoul08826Republic of Korea
| | - Song Hyun Jo
- Graduate School of Medical Science and EngineeringKorea Advanced Institute of Science and Technology (KAIST)Daejeon34141Republic of Korea
| | - Rosalind H. Lee
- School of Life SciencesGwangju Institute of Science and Technology (GIST)Gwangju61005Republic of Korea
| | - Christian P. Macks
- Center for NanomedicineInstitute for Basic ScienceYonsei UniversitySeoul03722Republic of Korea
- Graduate Program in Nanobiomedical EngineeringAdvanced Science InstituteYonsei UniversitySeoul03722Republic of Korea
| | - Taeyun Ku
- Graduate School of Medical Science and EngineeringKorea Advanced Institute of Science and Technology (KAIST)Daejeon34141Republic of Korea
| | - Jihwan Park
- School of Life SciencesGwangju Institute of Science and Technology (GIST)Gwangju61005Republic of Korea
| | - Chung Whan Lee
- Department of ChemistryGachon UniversitySeongnamGyeonggi‐do13120Republic of Korea
| | - Junho K. Hur
- Department of GeneticsCollege of MedicineHanyang UniversitySeoul04763Republic of Korea
| | - Chang Ho Sohn
- Center for NanomedicineInstitute for Basic ScienceYonsei UniversitySeoul03722Republic of Korea
- Graduate Program in Nanobiomedical EngineeringAdvanced Science InstituteYonsei UniversitySeoul03722Republic of Korea
| |
Collapse
|
26
|
Nikfar M, Mi H, Gong C, Kimko H, Popel AS. Quantifying Intratumoral Heterogeneity and Immunoarchitecture Generated In-Silico by a Spatial Quantitative Systems Pharmacology Model. Cancers (Basel) 2023; 15:2750. [PMID: 37345087 DOI: 10.3390/cancers15102750] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 05/05/2023] [Accepted: 05/11/2023] [Indexed: 06/23/2023] Open
Abstract
Spatial heterogeneity is a hallmark of cancer. Tumor heterogeneity can vary with time and location. The tumor microenvironment (TME) encompasses various cell types and their interactions that impart response to therapies. Therefore, a quantitative evaluation of tumor heterogeneity is crucial for the development of effective treatments. Different approaches, such as multiregional sequencing, spatial transcriptomics, analysis of autopsy samples, and longitudinal analysis of biopsy samples, can be used to analyze the intratumoral heterogeneity (ITH) and temporal evolution and to reveal the mechanisms of therapeutic response. However, because of the limitations of these data and the uncertainty associated with the time points of sample collection, having a complete understanding of intratumoral heterogeneity role is challenging. Here, we used a hybrid model that integrates a whole-patient compartmental quantitative-systems-pharmacology (QSP) model with a spatial agent-based model (ABM) describing the TME; we applied four spatial metrics to quantify model-simulated intratumoral heterogeneity and classified the TME immunoarchitecture for representative cases of effective and ineffective anti-PD-1 therapy. The four metrics, adopted from computational digital pathology, included mixing score, average neighbor frequency, Shannon's entropy and area under the curve (AUC) of the G-cross function. A fifth non-spatial metric was used to supplement the analysis, which was the ratio of the number of cancer cells to immune cells. These metrics were utilized to classify the TME as "cold", "compartmentalized" and "mixed", which were related to treatment efficacy. The trends in these metrics for effective and ineffective treatments are in qualitative agreement with the clinical literature, indicating that compartmentalized immunoarchitecture is likely to result in more efficacious treatment outcomes.
Collapse
Affiliation(s)
- Mehdi Nikfar
- Department of Biomedical Engineering, School of Medicine, Johns Hopkins University, Baltimore, MD 21205, USA
| | - Haoyang Mi
- Department of Biomedical Engineering, School of Medicine, Johns Hopkins University, Baltimore, MD 21205, USA
| | - Chang Gong
- Clinical Pharmacology & Quantitative Pharmacology, AstraZeneca, Waltham, MA 02451, USA
| | - Holly Kimko
- Clinical Pharmacology & Quantitative Pharmacology, AstraZeneca, Gaithersburg, MD 20878, USA
| | - Aleksander S Popel
- Department of Biomedical Engineering, School of Medicine, Johns Hopkins University, Baltimore, MD 21205, USA
- Sidney Kimmel Comprehensive Cancer Center, Department of Oncology, Johns Hopkins University, Baltimore, MD 21231, USA
| |
Collapse
|
27
|
Sikaroudi M, Hosseini M, Gonzalez R, Rahnamayan S, Tizhoosh HR. Generalization of vision pre-trained models for histopathology. Sci Rep 2023; 13:6065. [PMID: 37055519 PMCID: PMC10102232 DOI: 10.1038/s41598-023-33348-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Accepted: 04/12/2023] [Indexed: 04/15/2023] Open
Abstract
Out-of-distribution (OOD) generalization, especially for medical setups, is a key challenge in modern machine learning which has only recently received much attention. We investigate how different convolutional pre-trained models perform on OOD test data-that is data from domains that have not been seen during training-on histopathology repositories attributed to different trial sites. Different trial site repositories, pre-trained models, and image transformations are examined as specific aspects of pre-trained models. A comparison is also performed among models trained entirely from scratch (i.e., without pre-training) and models already pre-trained. The OOD performance of pre-trained models on natural images, i.e., (1) vanilla pre-trained ImageNet, (2) semi-supervised learning (SSL), and (3) semi-weakly-supervised learning (SWSL) models pre-trained on IG-1B-Targeted are examined in this study. In addition, the performance of a histopathology model (i.e., KimiaNet) trained on the most comprehensive histopathology dataset, i.e., TCGA, has also been studied. Although the performance of SSL and SWSL pre-trained models are conducive to better OOD performance in comparison to the vanilla ImageNet pre-trained model, the histopathology pre-trained model is still the best in overall. In terms of top-1 accuracy, we demonstrate that diversifying the images in the training using reasonable image transformations is effective to avoid learning shortcuts when the distribution shift is significant. In addition, XAI techniques-which aim to achieve high-quality human-understandable explanations of AI decisions-are leveraged for further investigations.
Collapse
Affiliation(s)
| | | | - Ricardo Gonzalez
- Kimia Lab, University of Waterloo, Waterloo, ON, Canada
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Shahryar Rahnamayan
- Kimia Lab, University of Waterloo, Waterloo, ON, Canada
- Engineering Department, Brock University, St. Catharines, ON, Canada
| | - H R Tizhoosh
- Kimia Lab, University of Waterloo, Waterloo, ON, Canada.
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA.
- Rhazes Lab, Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
28
|
Mandair D, Reis-Filho JS, Ashworth A. Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology. NPJ Breast Cancer 2023; 9:21. [PMID: 37024522 PMCID: PMC10079681 DOI: 10.1038/s41523-023-00518-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Accepted: 02/27/2023] [Indexed: 04/08/2023] Open
Abstract
Breast cancer remains a highly prevalent disease with considerable inter- and intra-tumoral heterogeneity complicating prognostication and treatment decisions. The utilization and depth of genomic, transcriptomic and proteomic data for cancer has exploded over recent times and the addition of spatial context to this information, by understanding the correlating morphologic and spatial patterns of cells in tissue samples, has created an exciting frontier of research, histo-genomics. At the same time, deep learning (DL), a class of machine learning algorithms employing artificial neural networks, has rapidly progressed in the last decade with a confluence of technical developments - including the advent of modern graphic processing units (GPU), allowing efficient implementation of increasingly complex architectures at scale; advances in the theoretical and practical design of network architectures; and access to larger datasets for training - all leading to sweeping advances in image classification and object detection. In this review, we examine recent developments in the application of DL in breast cancer histology with particular emphasis of those producing biologic insights or novel biomarkers, spanning the extraction of genomic information to the use of stroma to predict cancer recurrence, with the aim of suggesting avenues for further advancing this exciting field.
Collapse
Affiliation(s)
- Divneet Mandair
- UCSF Helen Diller Family Comprehensive Cancer Center, San Francisco, CA, 94158, USA
| | | | - Alan Ashworth
- UCSF Helen Diller Family Comprehensive Cancer Center, San Francisco, CA, 94158, USA.
| |
Collapse
|
29
|
Fatemi M, Feng E, Sharma C, Azher Z, Goel T, Ramwala O, Palisoul SM, Barney RE, Perreard L, Kolling FW, Salas LA, Christensen BC, Tsongalis GJ, Vaickus LJ, Levy JJ. Inferring spatial transcriptomics markers from whole slide images to characterize metastasis-related spatial heterogeneity of colorectal tumors: A pilot study. J Pathol Inform 2023; 14:100308. [PMID: 37114077 PMCID: PMC10127126 DOI: 10.1016/j.jpi.2023.100308] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 03/23/2023] [Accepted: 03/24/2023] [Indexed: 03/31/2023] Open
Abstract
Over 150 000 Americans are diagnosed with colorectal cancer (CRC) every year, and annually over 50 000 individuals will die from CRC, necessitating improvements in screening, prognostication, disease management, and therapeutic options. Tumor metastasis is the primary factor related to the risk of recurrence and mortality. Yet, screening for nodal and distant metastasis is costly, and invasive and incomplete resection may hamper adequate assessment. Signatures of the tumor-immune microenvironment (TIME) at the primary site can provide valuable insights into the aggressiveness of the tumor and the effectiveness of various treatment options. Spatially resolved transcriptomics technologies offer an unprecedented characterization of TIME through high multiplexing, yet their scope is constrained by cost. Meanwhile, it has long been suspected that histological, cytological, and macroarchitectural tissue characteristics correlate well with molecular information (e.g., gene expression). Thus, a method for predicting transcriptomics data through inference of RNA patterns from whole slide images (WSI) is a key step in studying metastasis at scale. In this work, we collected tissue from 4 stage-III (pT3) matched colorectal cancer patients for spatial transcriptomics profiling. The Visium spatial transcriptomics (ST) assay was used to measure transcript abundance for 17 943 genes at up to 5000 55-micron (i.e., 1-10 cells) spots per patient sampled in a honeycomb pattern, co-registered with hematoxylin and eosin (H&E) stained WSI. The Visium ST assay can measure expression at these spots through tissue permeabilization of mRNAs, which are captured through spatially (i.e., x-y positional coordinates) barcoded, gene specific oligo probes. WSI subimages were extracted around each co-registered Visium spot and were used to predict the expression at these spots using machine learning models. We prototyped and compared several convolutional, transformer, and graph convolutional neural networks to predict spatial RNA patterns at the Visium spots under the hypothesis that the transformer- and graph-based approaches better capture relevant spatial tissue architecture. We further analyzed the model's ability to recapitulate spatial autocorrelation statistics using SPARK and SpatialDE. Overall, the results indicate that the transformer- and graph-based approaches were unable to outperform the convolutional neural network architecture, though they exhibited optimal performance for relevant disease-associated genes. Initial findings suggest that different neural networks that operate on different scales are relevant for capturing distinct disease pathways (e.g., epithelial to mesenchymal transition). We add further evidence that deep learning models can accurately predict gene expression in whole slide images and comment on understudied factors which may increase its external applicability (e.g., tissue context). Our preliminary work will motivate further investigation of inference for molecular patterns from whole slide images as metastasis predictors and in other applications.
Collapse
Affiliation(s)
- Michael Fatemi
- Department of Computer Science, University of Virginia, Charlottesville, VA, USA
| | - Eric Feng
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, USA
| | - Cyril Sharma
- Department of Computer Science, Purdue University, West Lafayette, IN, USA
| | - Zarif Azher
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, USA
| | - Tarushii Goel
- Department of Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Ojas Ramwala
- Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA, USA
| | - Scott M. Palisoul
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
| | - Rachael E. Barney
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
| | | | | | - Lucas A. Salas
- Department of Epidemiology, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Department of Molecular and Systems Biology, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Integrative Neuroscience at Dartmouth (IND) graduate program, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
| | - Brock C. Christensen
- Department of Epidemiology, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Department of Molecular and Systems Biology, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Department of Community and Family Medicine, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
| | - Gregory J. Tsongalis
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
| | - Louis J. Vaickus
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
| | - Joshua J. Levy
- Emerging Diagnostic and Investigative Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Health, Lebanon, NH, USA
- Department of Epidemiology, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
- Department of Dermatology, Dartmouth Health, Lebanon, NH, USA
- Program in Quantitative Biomedical Sciences, Dartmouth College Geisel School of Medicine, Hanover, NH, USA
| |
Collapse
|
30
|
Alsaafin A, Safarpoor A, Sikaroudi M, Hipp JD, Tizhoosh HR. Learning to predict RNA sequence expressions from whole slide images with applications for search and classification. Commun Biol 2023; 6:304. [PMID: 36949169 PMCID: PMC10033650 DOI: 10.1038/s42003-023-04583-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 02/13/2023] [Indexed: 03/24/2023] Open
Abstract
Deep learning methods are widely applied in digital pathology to address clinical challenges such as prognosis and diagnosis. As one of the most recent applications, deep models have also been used to extract molecular features from whole slide images. Although molecular tests carry rich information, they are often expensive, time-consuming, and require additional tissue to sample. In this paper, we propose tRNAsformer, an attention-based topology that can learn both to predict the bulk RNA-seq from an image and represent the whole slide image of a glass slide simultaneously. The tRNAsformer uses multiple instance learning to solve a weakly supervised problem while the pixel-level annotation is not available for an image. We conducted several experiments and achieved better performance and faster convergence in comparison to the state-of-the-art algorithms. The proposed tRNAsformer can assist as a computational pathology tool to facilitate a new generation of search and classification methods by combining the tissue morphology and the molecular fingerprint of the biopsy samples.
Collapse
Affiliation(s)
- Areej Alsaafin
- Rhazes Lab, Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
- Kimia Lab, University of Waterloo, Waterloo, ON, Canada
| | | | | | - Jason D Hipp
- Division of Computational Pathology and AI, Mayo Clinic, Rochester, MN, USA
| | - H R Tizhoosh
- Rhazes Lab, Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA.
- Kimia Lab, University of Waterloo, Waterloo, ON, Canada.
| |
Collapse
|
31
|
From patterns to patients: Advances in clinical machine learning for cancer diagnosis, prognosis, and treatment. Cell 2023; 186:1772-1791. [PMID: 36905928 DOI: 10.1016/j.cell.2023.01.035] [Citation(s) in RCA: 76] [Impact Index Per Article: 76.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 01/10/2023] [Accepted: 01/26/2023] [Indexed: 03/12/2023]
Abstract
Machine learning (ML) is increasingly used in clinical oncology to diagnose cancers, predict patient outcomes, and inform treatment planning. Here, we review recent applications of ML across the clinical oncology workflow. We review how these techniques are applied to medical imaging and to molecular data obtained from liquid and solid tumor biopsies for cancer diagnosis, prognosis, and treatment design. We discuss key considerations in developing ML for the distinct challenges posed by imaging and molecular data. Finally, we examine ML models approved for cancer-related patient usage by regulatory agencies and discuss approaches to improve the clinical usefulness of ML.
Collapse
|
32
|
Yang D, Wu Y, Wan Z, Xu Z, Li W, Yuan P, Shang Q, Peng J, Tao L, Chen Q, Dan H, Xu H. HISMD: A Novel Immune Subtyping System for HNSCC. J Dent Res 2023; 102:270-279. [PMID: 36333876 DOI: 10.1177/00220345221134605] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Abstract
Immune subtyping is an important way to reveal immune heterogeneity, which may contribute to the diversity of the progression and treatment in head and neck squamous cell carcinoma (HNSCC). However, reported immune subtypes mainly focus on levels of immune infiltration and are mostly based on a mono-omics profile. This study aimed to identify a comprehensive immune subtype for HNSCC via multi-omics clustering and build a novel subtype prediction system for clinical application. Data were obtained from The Cancer Genome Atlas database and our independent multicenter cohort. Multi-omics clustering was performed to identify 3 clusters of 499 patients in The Cancer Genome Atlas based on immune-related gene expression and somatic mutations. The immune characteristics and biological features of the obtained clusters were revealed by bioinformatics, and 3 immune subtypes were identified: 1) adaptive immune activation subtype predominantly enriched in T cells, 2) innate immune activation subtype predominantly enriched in macrophages, and 3) immune desert subtype. Subsequently, the clinical implications of each subtype were analyzed per clinical epidemiology. We found that adaptive immune activation showed better survival outcomes and had a similar response to chemotherapy with innate immune activation, whereas immune desert might be relatively resistant to chemotherapy. Moreover, a subtype prediction system was developed by deep learning with whole slide images and named HISMD: HNSCC Immune Subtypes via Multi-omics and Deep Learning. We endowed HISMD with interpretability through image-based key feature extraction. The clinical implications, biological significances, and predictive stability of HISMD were successfully verified by using our independent multicenter cohort data set. In summary, this study revealed the immune heterogeneity of HNSCC and obtained a novel, highly accurate, and interpretable immune subtyping prediction system. For clinical implementation in the future, additional validation and utility studies are warranted.
Collapse
Affiliation(s)
- D Yang
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - Y Wu
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - Z Wan
- Department of Pathology, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - Z Xu
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - W Li
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - P Yuan
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - Q Shang
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - J Peng
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - L Tao
- College of Mathematics, Sichuan University, Chengdu, China
| | - Q Chen
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China.,Key Laboratory of Oral Biomedical Research of Zhejiang Province, Affiliated Stomatology Hospital, Zhejiang University School of Stomatology, Hangzhou, China
| | - H Dan
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - H Xu
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| |
Collapse
|
33
|
Petinrin OO, Saeed F, Toseef M, Liu Z, Basurra S, Muyide IO, Li X, Lin Q, Wong KC. Machine Learning in Metastatic Cancer Research: Potentials, Possibilities, and Prospects. Comput Struct Biotechnol J 2023; 21:2454-2470. [PMID: 37077177 PMCID: PMC10106342 DOI: 10.1016/j.csbj.2023.03.046] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Revised: 03/26/2023] [Accepted: 03/27/2023] [Indexed: 03/31/2023] Open
Abstract
Cancer has received extensive recognition for its high mortality rate, with metastatic cancer being the top cause of cancer-related deaths. Metastatic cancer involves the spread of the primary tumor to other body organs. As much as the early detection of cancer is essential, the timely detection of metastasis, the identification of biomarkers, and treatment choice are valuable for improving the quality of life for metastatic cancer patients. This study reviews the existing studies on classical machine learning (ML) and deep learning (DL) in metastatic cancer research. Since the majority of metastatic cancer research data are collected in the formats of PET/CT and MRI image data, deep learning techniques are heavily involved. However, its black-box nature and expensive computational cost are notable concerns. Furthermore, existing models could be overestimated for their generality due to the non-diverse population in clinical trial datasets. Therefore, research gaps are itemized; follow-up studies should be carried out on metastatic cancer using machine learning and deep learning tools with data in a symmetric manner.
Collapse
|
34
|
Patkulkar P, Subbalakshmi AR, Jolly MK, Sinharay S. Mapping Spatiotemporal Heterogeneity in Tumor Profiles by Integrating High-Throughput Imaging and Omics Analysis. ACS OMEGA 2023; 8:6126-6138. [PMID: 36844580 PMCID: PMC9948167 DOI: 10.1021/acsomega.2c06659] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/16/2022] [Accepted: 01/05/2023] [Indexed: 05/14/2023]
Abstract
Intratumoral heterogeneity associates with more aggressive disease progression and worse patient outcomes. Understanding the reasons enabling the emergence of such heterogeneity remains incomplete, which restricts our ability to manage it from a therapeutic perspective. Technological advancements such as high-throughput molecular imaging, single-cell omics, and spatial transcriptomics allow recording of patterns of spatiotemporal heterogeneity in a longitudinal manner, thus offering insights into the multiscale dynamics of its evolution. Here, we review the latest technological trends and biological insights from molecular diagnostics as well as spatial transcriptomics, both of which have witnessed burgeoning growth in the recent past in terms of mapping heterogeneity within tumor cell types as well as the stromal constitution. We also discuss ongoing challenges, indicating possible ways to integrate insights across these methods to have a systems-level spatiotemporal map of heterogeneity in each tumor and a more systematic investigation of the implications of heterogeneity for patient outcomes.
Collapse
|
35
|
Levy-Jurgenson A, Tekpli X, Kristensen VN, Yakhini Z. Analysis of Spatial Molecular Data. Methods Mol Biol 2023; 2614:349-356. [PMID: 36587134 DOI: 10.1007/978-1-0716-2914-7_20] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
Abstract
Digital analysis of pathology whole-slide images has been recently gaining interest in the context of cancer diagnosis and treatment. In particular, deep learning methods have demonstrated significant potential in supporting pathology analysis, recently detecting molecular traits never before recognized in pathology H&E whole-slide images (WSIs). Alongside these advancements in the digital analysis of WSIs, it is becoming increasingly evident that both spatial and overall tumor heterogeneity may be significant determinants of cancer prognosis and treatment outcome. In this chapter, we describe methods that aim to support these two elements. We describe both an end-to-end deep learning pipeline for producing limited spatial transcriptomics from WSIs with associated bulk gene expression data, as well as an algorithm for quantifying spatial tumor heterogeneity based on the results of this pipeline.
Collapse
Affiliation(s)
- Alona Levy-Jurgenson
- Department of Computer Science, Technion - Israel Institute of Technology, Haifa, Israel
| | - Xavier Tekpli
- Department of Medical Genetics, Institute of Clinical Medicine, University of Oslo and Oslo University Hospital, Oslo, Norway
- Department of Cancer Genetics, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway
| | - Vessela N Kristensen
- Department of Medical Genetics, Institute of Clinical Medicine, University of Oslo and Oslo University Hospital, Oslo, Norway
- Department of Cancer Genetics, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway
- Division of Medicine, Department of Clinical Molecular Biology and Laboratory Science (EpiGen), Akershus University Hospital, Lørenskog, Norway
| | - Zohar Yakhini
- Department of Computer Science, Technion - Israel Institute of Technology, Haifa, Israel.
- Interdisciplinary Center, Arazi School of Computer Science, Herzliya, Israel.
| |
Collapse
|
36
|
McCarthy PM, Valdera FA, Smolinsky TR, Adams AM, O’Shea AE, Thomas KK, Van Decar S, Carpenter EL, Tiwari A, Myers JW, Hale DF, Vreeland TJ, Peoples GE, Stojadinovic A, Clifton GT. Tumor infiltrating lymphocytes as an endpoint in cancer vaccine trials. Front Immunol 2023; 14:1090533. [PMID: 36960052 PMCID: PMC10029975 DOI: 10.3389/fimmu.2023.1090533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Accepted: 02/14/2023] [Indexed: 03/09/2023] Open
Abstract
Checkpoint inhibitors have invigorated cancer immunotherapy research, including cancer vaccination. Classic early phase trial design and endpoints used in developing chemotherapy are not suited for evaluating all forms of cancer treatment. Peripheral T cell response dynamics have demonstrated inconsistency in assessing the efficacy of cancer vaccination. Tumor infiltrating lymphocytes (TILs), reflect the local tumor microenvironment and may prove a superior endpoint in cancer vaccination trials. Cancer vaccines may also promote success in combination immunotherapy treatment of weakly immunogenic tumors. This review explores the impact of TILs as an endpoint for cancer vaccination in multiple malignancies, summarizes the current literature regarding TILs analysis, and discusses the challenges of providing validity and a standardized implementation of this approach.
Collapse
Affiliation(s)
- Patrick M. McCarthy
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Franklin A. Valdera
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Todd R. Smolinsky
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
- *Correspondence: Todd R. Smolinsky, ; Elizabeth L. Carpenter,
| | - Alexandra M. Adams
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Anne E. O’Shea
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Katryna K. Thomas
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Spencer Van Decar
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Elizabeth L. Carpenter
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
- *Correspondence: Todd R. Smolinsky, ; Elizabeth L. Carpenter,
| | - Ankur Tiwari
- Department of Surgery, University of Texas Health Science Center, San Antonio, TX, United States
| | - John W. Myers
- Department of Surgery, Madigan Army Medical Center, Ft. Lewis, WA, United States
| | - Diane F. Hale
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | - Timothy J. Vreeland
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| | | | | | - Guy T. Clifton
- Department of Surgery, Brooke Army Medical Center, Ft. Sam Houston, TX, United States
| |
Collapse
|
37
|
Jung M, Lee JA, Yoo SY, Bae JM, Kang GH, Kim JH. Intratumoral spatial heterogeneity of tumor-infiltrating lymphocytes is a significant factor for precisely stratifying prognostic immune subgroups of microsatellite instability-high colorectal carcinomas. Mod Pathol 2022; 35:2011-2022. [PMID: 35869301 DOI: 10.1038/s41379-022-01137-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 06/20/2022] [Accepted: 06/28/2022] [Indexed: 02/07/2023]
Abstract
Although the density of tumor-infiltrating lymphocytes (TILs) is known to be linked to prognosis in various cancers, the prognostic impact and immunologic significance of the spatial heterogeneity of TILs have been rarely investigated. In this study, CD3+ and CD8+ TILs were quantified in independent cohorts (discovery, n = 73; and external validation, n = 93) of colorectal carcinomas (CRCs) with microsatellite instability-high (MSI-H) utilizing whole-slide image analysis of CD3/CD8 immunohistochemistry. The Shannon and Simpson indices, which measure intratumoral patch-to-patch evenness of TIL densities, were used to quantitatively assess the spatial heterogeneity of TILs in each case. To uncover immune-related gene expression signatures of spatial heterogeneity-based TIL subgroups of MSI-H CRCs, representative cases were subjected to GeoMx digital spatial profiler (DSP) analysis. As expected, a low density of TILs was significantly associated with poor disease-free survival (DFS) in MSI-H CRCs. The TIL-low tumors were further classified into two subgroups based on the spatial heterogeneity of TILs: TIL-low/heterogeneity-high and TIL-low/heterogeneity-low subgroups. In both discovery and validation cohorts, the TIL-low/heterogeneity-high, TIL-low/heterogeneity-low, and TIL-high subgroups were significantly associated with poor, intermediate, and good DFS, respectively. In the DSP analysis, the TIL-low/heterogeneity-high subgroup showed higher spatial diversity in the expression of immune-related genes than that of the TIL-low/heterogeneity-low subgroup and exhibited upregulation of genes related to immune checkpoints, chemokine/cytokine receptors, and myeloid cells. TIL-low/heterogeneity-high tumors were also enriched with gene sets related to good response to immune checkpoint inhibitor therapy. In conclusion, TIL-low MSI-H CRCs are prognostically heterogeneous and can be divided into prognostically and immunologically distinct subgroups by considering the spatial heterogeneity of TILs. Our data suggest that intratumoral spatial heterogeneity of TILs can be used as a key element for clinically relevant immunologic subtyping of tumors.
Collapse
Affiliation(s)
- Minsun Jung
- Department of Pathology, Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Ji Ae Lee
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea.,Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Seung-Yeon Yoo
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea.,Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea.,Seegene Medical Foundation, Seoul, Republic of Korea
| | - Jeong Mo Bae
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea.,Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Gyeong Hoon Kang
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea.,Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jung Ho Kim
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea. .,Cancer Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
38
|
Rodrigues A, Nogueira C, Marinho LC, Velozo G, Sousa J, Silva PG, Tavora F. Computer-assisted tumor grading, validation of PD-L1 scoring, and quantification of CD8-positive immune cell density in urothelial carcinoma, a visual guide for pathologists using QuPath. SURGICAL AND EXPERIMENTAL PATHOLOGY 2022. [DOI: 10.1186/s42047-022-00112-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
Abstract
Background
Advances in digital imaging in pathology and the new capacity to scan high-quality images have change the way to practice and research in surgical pathology. QuPath is an open-source pathology software that offers a reproducible way to analyze quantified variables. We aimed to present the functionality of biomarker scoring using QuPath and provide a guide for the validation of pathologic grading using a series of cases of urothelial carcinomas.
Methods
Tissue microarrays of urothelial carcinomas were constructed and scanned. The images stained with HE, CD8 and PD-L1 immunohistochemistry were imported into QuPath and dearrayed. Training images were used to build a grade classifier and applied to all cases. Quantification of CD8 and PD-L1 was undertaken for each core using cytoplasmic and membrane color segmentation and output measurement and compared with pathologists semi-quantitative assessments.
Results
There was a good correlation between tumor grade by the pathologist and by QuPath software (Kappa agreement 0.73). For low-grade carcinomas (by the report and pathologist), the concordance was not as high. Of the 32 low-grade tumors, 22 were correctly classified as low-grade, but 11 (34%) were diagnosed as high-grade, with the high-grade to the low-grade ratio in these misclassified cases ranging from 0.41 to 0.58. The median ratio for bona fide high-grade carcinomas was 0.59. Some of the reasons the authors list as potential mimickers for high-grade cases are fulguration artifact, nuclear hyperchromasia, folded tissues, and inconsistency in staining. The correlation analysis between the software and the pathologist showed that the CD8 marker showed a moderate (r = 0.595) and statistically significant (p < 0.001) correlation. The internal consistency of this parameter showed an index of 0.470. The correlation analysis between the software and the pathologist showed that the PDL1 marker showed a robust (r = 0.834) and significant (p < 0.001) correlation. The internal consistency of this parameter showed a CCI of 0.851.
Conclusions
We were able to demonstrate the utility of QuPath in identifying and scoring tumor cells and IHC quantification of two biomarkers. The protocol we present uses a free open-source platform to help researchers deal with imaging and data processing in the surgical pathology field.
Collapse
|
39
|
Kosaraju S, Park J, Lee H, Yang JW, Kang M. Deep learning-based framework for slide-based histopathological image analysis. Sci Rep 2022; 12:19075. [PMID: 36351997 PMCID: PMC9646838 DOI: 10.1038/s41598-022-23166-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2022] [Accepted: 10/26/2022] [Indexed: 11/11/2022] Open
Abstract
Digital pathology coupled with advanced machine learning (e.g., deep learning) has been changing the paradigm of whole-slide histopathological images (WSIs) analysis. Major applications in digital pathology using machine learning include automatic cancer classification, survival analysis, and subtyping from pathological images. While most pathological image analyses are based on patch-wise processing due to the extremely large size of histopathology images, there are several applications that predict a single clinical outcome or perform pathological diagnosis per slide (e.g., cancer classification, survival analysis). However, current slide-based analyses are task-dependent, and a general framework of slide-based analysis in WSI has been seldom investigated. We propose a novel slide-based histopathology analysis framework that creates a WSI representation map, called HipoMap, that can be applied to any slide-based problems, coupled with convolutional neural networks. HipoMap converts a WSI of various shapes and sizes to structured image-type representation. Our proposed HipoMap outperformed existing methods in intensive experiments with various settings and datasets. HipoMap showed the Area Under the Curve (AUC) of 0.96±0.026 (5% improved) in the experiments for lung cancer classification, and c-index of 0.787±0.013 (3.5% improved) and coefficient of determination ([Formula: see text]) of 0.978±0.032 (24% improved) in survival analysis and survival prediction with TCGA lung cancer data respectively, as a general framework of slide-based analysis with a flexible capability. The results showed significant improvement comparing to the current state-of-the-art methods on each task. We further discussed experimental results of HipoMap as pathological viewpoints and verified the performance using publicly available TCGA datasets. A Python package is available at https://pypi.org/project/hipomap , and the package can be easily installed using Python PIP. The open-source codes in Python are available at: https://github.com/datax-lab/HipoMap .
Collapse
Affiliation(s)
- Sai Kosaraju
- grid.272362.00000 0001 0806 6926Department of Computer Science, University of Nevada, Las Vegas, Las Vegas, NV 89154 USA
| | - Jeongyeon Park
- grid.412859.30000 0004 0533 4202Department of Computer Science, Sun Moon University, Asan, 336708 South Korea
| | - Hyun Lee
- grid.412859.30000 0004 0533 4202Department of Computer Science, Sun Moon University, Asan, 336708 South Korea
| | - Jung Wook Yang
- grid.256681.e0000 0001 0661 1492Department of Pathology, Gyeongsang National University Hospital, Gyeongsang National University College of Medicine, Jinju, South Korea
| | - Mingon Kang
- grid.272362.00000 0001 0806 6926Department of Computer Science, University of Nevada, Las Vegas, Las Vegas, NV 89154 USA
| |
Collapse
|
40
|
Lee SH, Jang HJ. Deep learning-based prediction of molecular cancer biomarkers from tissue slides: A new tool for precision oncology. Clin Mol Hepatol 2022; 28:754-772. [PMID: 35443570 PMCID: PMC9597228 DOI: 10.3350/cmh.2021.0394] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 04/17/2022] [Indexed: 02/06/2023] Open
Abstract
Molecular tests are necessary to stratify cancer patients for targeted therapy. However, high cost and technical barriers limit the application of these tests, hindering optimal treatment. Recently, deep learning (DL) has been applied to predict molecular test results from digitized images of tissue slides. Furthermore, treatment response and prognosis can be predicted from tissue slides using DL. In this review, we summarized DL-based studies regarding the prediction of genetic mutation, microsatellite instability, tumor mutational burden, molecular subtypes, gene expression, treatment response, and prognosis directly from hematoxylin- and eosin-stained tissue slides. Although performance needs to be improved, these studies clearly demonstrated the feasibility of DL-based prediction of key molecular features in cancer tissues. With the accumulation of data and technical advances, the performance of the DL system could be improved in the near future. Therefore, we expect that DL could provide cost- and time-effective alternative tools for patient stratification in the era of precision oncology.
Collapse
Affiliation(s)
- Sung Hak Lee
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Hyun-Jong Jang
- Catholic Big Data Integration Center, Department of Physiology, College of Medicine, The Catholic University of Korea, Seoul, Korea,Corresponding author : Hyun-Jong Jang Department of Physiology, College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Korea Tel: +82-2-2258-7274, Fax: +82-2-532-9575, E-mail:
| |
Collapse
|
41
|
Vitkin E, Singh A, Wise J, Ben-Elazar S, Yakhini Z, Golberg A. Nondestructive protein sampling with electroporation facilitates profiling of spatial differential protein expression in breast tumors in vivo. Sci Rep 2022; 12:15835. [PMID: 36151122 PMCID: PMC9508265 DOI: 10.1038/s41598-022-19984-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2022] [Accepted: 09/07/2022] [Indexed: 11/26/2022] Open
Abstract
Excision tissue biopsy, while central to cancer treatment and precision medicine, presents risks to the patient and does not provide a sufficiently broad and faithful representation of the heterogeneity of solid tumors. Here we introduce e-biopsy—a novel concept for molecular profiling of solid tumors using molecular sampling with electroporation. As e-biopsy provides access to the molecular composition of a solid tumor by permeabilization of the cell membrane, it facilitates tumor diagnostics without tissue resection. Furthermore, thanks to its non tissue destructive characteristics, e-biopsy enables probing the solid tumor multiple times in several distinct locations in the same procedure, thereby enabling the spatial profiling of tumor molecular heterogeneity.We demonstrate e-biopsy in vivo, using the 4T1 breast cancer model in mice to assess its performance, as well as the inferred spatial differential protein expression. In particular, we show that proteomic profiles obtained via e-biopsy in vivo distinguish the tumors from healthy breast tissue and reflect spatial tumor differential protein expression. E-biopsy provides a completely new molecular sampling modality for solid tumors molecular cartography, providing information that potentially enables more rapid and sensitive detection at lesser risk, as well as more precise personalized medicine.
Collapse
Affiliation(s)
- Edward Vitkin
- School of Computer Science, Reichman University (IDC Herzliya), Herzliya, Israel
| | - Amrita Singh
- Porter School of Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Julia Wise
- Porter School of Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Shay Ben-Elazar
- School of Computer Science, Reichman University (IDC Herzliya), Herzliya, Israel
| | - Zohar Yakhini
- School of Computer Science, Reichman University (IDC Herzliya), Herzliya, Israel. .,Computer Science Faculty, Technion, Haifa, Israel.
| | - Alexander Golberg
- Porter School of Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
42
|
Akhoundova D, Rubin MA. Clinical application of advanced multi-omics tumor profiling: Shaping precision oncology of the future. Cancer Cell 2022; 40:920-938. [PMID: 36055231 DOI: 10.1016/j.ccell.2022.08.011] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Revised: 05/22/2022] [Accepted: 08/11/2022] [Indexed: 12/17/2022]
Abstract
Next-generation DNA sequencing technology has dramatically advanced clinical oncology through the identification of therapeutic targets and molecular biomarkers, leading to the personalization of cancer treatment with significantly improved outcomes for many common and rare tumor entities. More recent developments in advanced tumor profiling now enable dissection of tumor molecular architecture and the functional phenotype at cellular and subcellular resolution. Clinical translation of high-resolution tumor profiling and integration of multi-omics data into precision treatment, however, pose significant challenges at the level of prospective validation and clinical implementation. In this review, we summarize the latest advances in multi-omics tumor profiling, focusing on spatial genomics and chromatin organization, spatial transcriptomics and proteomics, liquid biopsy, and ex vivo modeling of drug response. We analyze the current stages of translational validation of these technologies and discuss future perspectives for their integration into precision treatment.
Collapse
Affiliation(s)
- Dilara Akhoundova
- Department for BioMedical Research, University of Bern, 3008 Bern, Switzerland; Department of Medical Oncology, Inselspital, University Hospital of Bern, 3010 Bern, Switzerland
| | - Mark A Rubin
- Department for BioMedical Research, University of Bern, 3008 Bern, Switzerland; Bern Center for Precision Medicine, Inselspital, University Hospital of Bern, 3008 Bern, Switzerland.
| |
Collapse
|
43
|
Shmatko A, Ghaffari Laleh N, Gerstung M, Kather JN. Artificial intelligence in histopathology: enhancing cancer research and clinical oncology. NATURE CANCER 2022; 3:1026-1038. [PMID: 36138135 DOI: 10.1038/s43018-022-00436-4] [Citation(s) in RCA: 112] [Impact Index Per Article: 56.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Accepted: 08/03/2022] [Indexed: 06/16/2023]
Abstract
Artificial intelligence (AI) methods have multiplied our capabilities to extract quantitative information from digital histopathology images. AI is expected to reduce workload for human experts, improve the objectivity and consistency of pathology reports, and have a clinical impact by extracting hidden information from routinely available data. Here, we describe how AI can be used to predict cancer outcome, treatment response, genetic alterations and gene expression from digitized histopathology slides. We summarize the underlying technologies and emerging approaches, noting limitations, including the need for data sharing and standards. Finally, we discuss the broader implications of AI in cancer research and oncology.
Collapse
Affiliation(s)
- Artem Shmatko
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | | | - Moritz Gerstung
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
- Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany.
| |
Collapse
|
44
|
Understanding Breast Cancers through Spatial and High-Resolution Visualization Using Imaging Technologies. Cancers (Basel) 2022; 14:cancers14174080. [PMID: 36077616 PMCID: PMC9454728 DOI: 10.3390/cancers14174080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 08/12/2022] [Accepted: 08/18/2022] [Indexed: 11/17/2022] Open
Abstract
Breast cancer is the most common cancer affecting women worldwide. Although many analyses and treatments have traditionally targeted the breast cancer cells themselves, recent studies have focused on investigating entire cancer tissues, including breast cancer cells. To understand the structure of breast cancer tissues, including breast cancer cells, it is necessary to investigate the three-dimensional location of the cells and/or proteins comprising the tissues and to clarify the relationship between the three-dimensional structure and malignant transformation or metastasis of breast cancers. In this review, we aim to summarize the methods for analyzing the three-dimensional structure of breast cancer tissue, paying particular attention to the recent technological advances in the combination of the tissue-clearing method and optical three-dimensional imaging. We also aimed to identify the latest methods for exploring the relationship between the three-dimensional cell arrangement in breast cancer tissues and the gene expression of each cell. Finally, we aimed to describe the three-dimensional imaging features of breast cancer tissues using noninvasive photoacoustic imaging methods.
Collapse
|
45
|
Foroughi pour A, White BS, Park J, Sheridan TB, Chuang JH. Deep learning features encode interpretable morphologies within histological images. Sci Rep 2022; 12:9428. [PMID: 35676395 PMCID: PMC9177767 DOI: 10.1038/s41598-022-13541-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2022] [Accepted: 05/25/2022] [Indexed: 12/12/2022] Open
Abstract
Convolutional neural networks (CNNs) are revolutionizing digital pathology by enabling machine learning-based classification of a variety of phenotypes from hematoxylin and eosin (H&E) whole slide images (WSIs), but the interpretation of CNNs remains difficult. Most studies have considered interpretability in a post hoc fashion, e.g. by presenting example regions with strongly predicted class labels. However, such an approach does not explain the biological features that contribute to correct predictions. To address this problem, here we investigate the interpretability of H&E-derived CNN features (the feature weights in the final layer of a transfer-learning-based architecture). While many studies have incorporated CNN features into predictive models, there has been little empirical study of their properties. We show such features can be construed as abstract morphological genes (“mones”) with strong independent associations to biological phenotypes. Many mones are specific to individual cancer types, while others are found in multiple cancers especially from related tissue types. We also observe that mone-mone correlations are strong and robustly preserved across related cancers. Importantly, linear mone-based classifiers can very accurately separate 38 distinct classes (19 tumor types and their adjacent normals, AUC = \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$97.1\% \pm 2.8\%$$\end{document}97.1%±2.8% for each class prediction), and linear classifiers are also highly effective for universal tumor detection (AUC = \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$99.2\% \pm 0.12\%$$\end{document}99.2%±0.12%). This linearity provides evidence that individual mones or correlated mone clusters may be associated with interpretable histopathological features or other patient characteristics. In particular, the statistical similarity of mones to gene expression values allows integrative mone analysis via expression-based bioinformatics approaches. We observe strong correlations between individual mones and individual gene expression values, notably mones associated with collagen gene expression in ovarian cancer. Mone-expression comparisons also indicate that immunoglobulin expression can be identified using mones in colon adenocarcinoma and that immune activity can be identified across multiple cancer types, and we verify these findings by expert histopathological review. Our work demonstrates that mones provide a morphological H&E decomposition that can be effectively associated with diverse phenotypes, analogous to the interpretability of transcription via gene expression values. Our work also demonstrates mones can be interpreted without using a classifier as a proxy.
Collapse
|
46
|
Li Y, Stanojevic S, Garmire LX. Emerging Artificial Intelligence Applications in Spatial Transcriptomics Analysis. Comput Struct Biotechnol J 2022; 20:2895-2908. [PMID: 35765645 PMCID: PMC9201012 DOI: 10.1016/j.csbj.2022.05.056] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 05/28/2022] [Accepted: 05/28/2022] [Indexed: 11/19/2022] Open
Abstract
Spatial transcriptomics (ST) has advanced significantly in the last few years. Such advancement comes with the urgent need for novel computational methods to handle the unique challenges of ST data analysis. Many artificial intelligence (AI) methods have been developed to utilize various machine learning and deep learning techniques for computational ST analysis. This review provides a comprehensive and up-to-date survey of current AI methods for ST analysis.
Collapse
Affiliation(s)
- Yijun Li
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA
| | - Stefan Stanojevic
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Lana X. Garmire
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
- Corresponding author.
| |
Collapse
|
47
|
Gao D, Ning J, Liu G, Sun S, Dang X. SpatialMap: Spatial Mapping of Unmeasured Gene Expression Profiles in Spatial Transcriptomic Data Using Generalized Linear Spatial Models. Front Genet 2022; 13:893522. [PMID: 35692845 PMCID: PMC9181802 DOI: 10.3389/fgene.2022.893522] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 04/19/2022] [Indexed: 12/02/2022] Open
Abstract
Recent advances in various single-cell RNA sequencing (scRNA-seq) technologies have enabled profiling the gene expression level with the whole transcriptome at a single-cell resolution. However, it lacks the spatial context of tissues. The image-based transcriptomics in situ studies (e.g., MERFISH and seqFISH) maintain the cell spatial context at individual cell levels but can only measure a limited number of genes or transcripts (up to roughly 1,000 genes). Therefore, integrating scRNA-seq data and image-based transcriptomics data can potentially gain the complementary benefits of both. Here, we develop a computational method, SpatialMap, to bridge the gap, which primarily facilitates spatial mapping of unmeasured gene profiles in spatial transcriptomic data via integrating with scRNA-seq data from the same tissue. SpatialMap directly models the count nature of spatial gene expression data through generalized linear spatial models, which accounts for the spatial correlation among spatial locations using conditional autoregressive (CAR) prior. With a newly developed computationally efficient penalized quasi-likelihood (PQL)-based algorithm, SpatialMap can scale up to performing large-scale spatial mapping analysis. Finally, we applied the SpatialMap to four publicly available tissue-paired studies (i.e., scRNA-seq studies and image-based transcriptomics studies). The results demonstrate that the proposed method can accurately predict unmeasured gene expression profiles across various spatial and scRNA-seq dataset pairs of different species and technologies.
Collapse
Affiliation(s)
- Dalong Gao
- First Department of Orthopedics, The Second Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China
- Sports Medicine and Joint Surgery, Xianyang Central Hospital, Xianyang, China
| | - Jin Ning
- School of Public Health, Xi’an Jiaotong University, Xi’an, China
| | - Gang Liu
- Sports Medicine and Joint Surgery, Xianyang Central Hospital, Xianyang, China
| | - Shiquan Sun
- School of Public Health, Xi’an Jiaotong University, Xi’an, China
- Key Laboratory of Trace Elements and Endemic Disease, Xi’an Jiaotong University, Xi’an, China
- *Correspondence: Shiquan Sun, ; Xiaoqian Dang,
| | - Xiaoqian Dang
- First Department of Orthopedics, The Second Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China
- *Correspondence: Shiquan Sun, ; Xiaoqian Dang,
| |
Collapse
|
48
|
Miller BF, Huang F, Atta L, Sahoo A, Fan J. Reference-free cell type deconvolution of multi-cellular pixel-resolution spatially resolved transcriptomics data. Nat Commun 2022; 13:2339. [PMID: 35487922 PMCID: PMC9055051 DOI: 10.1038/s41467-022-30033-z] [Citation(s) in RCA: 54] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2021] [Accepted: 04/12/2022] [Indexed: 12/12/2022] Open
Abstract
Recent technological advancements have enabled spatially resolved transcriptomic profiling but at multi-cellular pixel resolution, thereby hindering the identification of cell-type-specific spatial patterns and gene expression variation. To address this challenge, we develop STdeconvolve as a reference-free approach to deconvolve underlying cell types comprising such multi-cellular pixel resolution spatial transcriptomics (ST) datasets. Using simulated as well as real ST datasets from diverse spatial transcriptomics technologies comprising a variety of spatial resolutions such as Spatial Transcriptomics, 10X Visium, DBiT-seq, and Slide-seq, we show that STdeconvolve can effectively recover cell-type transcriptional profiles and their proportional representation within pixels without reliance on external single-cell transcriptomics references. STdeconvolve provides comparable performance to existing reference-based methods when suitable single-cell references are available, as well as potentially superior performance when suitable single-cell references are not available. STdeconvolve is available as an open-source R software package with the source code available at https://github.com/JEFworks-Lab/STdeconvolve. Identifying cell-type-specific spatial patterns in ST data is critical for understanding tissue organization but current methods rely on external references. Here the authors develop a reference-free method to effectively recover cell-type transcriptional profiles and proportions.
Collapse
Affiliation(s)
- Brendan F Miller
- Center for Computational Biology, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21211, United States.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Feiyang Huang
- Center for Computational Biology, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21211, United States.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Lyla Atta
- Center for Computational Biology, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21211, United States.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Arpan Sahoo
- Center for Computational Biology, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21211, United States.,Department of Computer Science, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Jean Fan
- Center for Computational Biology, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21211, United States. .,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21218, United States. .,Department of Computer Science, Johns Hopkins University, Baltimore, MD, 21218, United States.
| |
Collapse
|
49
|
He B, Bukhari S, Fox E, Abid A, Shen J, Kawas C, Corrada M, Montine T, Zou J. AI-enabled in silico immunohistochemical characterization for Alzheimer's disease. CELL REPORTS METHODS 2022; 2:100191. [PMID: 35497493 PMCID: PMC9046239 DOI: 10.1016/j.crmeth.2022.100191] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 12/23/2021] [Accepted: 03/04/2022] [Indexed: 01/07/2023]
Abstract
We develop a deep learning approach, in silico immunohistochemistry (IHC), which takes routinely collected histochemical-stained samples as input and computationally generates virtual IHC slide images. We apply in silico IHC to Alzheimer's disease samples, where several hallmark changes are conventionally identified using IHC staining across many regions of the brain. In silico IHC computationally identifies neurofibrillary tangles, β-amyloid plaques, and neuritic plaques at a high spatial resolution directly from the histochemical images, with areas under the receiver operating characteristic curve of between 0.88 and 0.92. In silico IHC learns to identify subtle cellular morphologies associated with these lesions and can generate in silico IHC slides that capture key features of the actual IHC.
Collapse
Affiliation(s)
- Bryan He
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA
| | - Syed Bukhari
- Department of Pathology, Stanford University, Stanford, CA 94305, USA
| | - Edward Fox
- Department of Pathology, Stanford University, Stanford, CA 94305, USA
| | - Abubakar Abid
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA
| | - Jeanne Shen
- Department of Pathology, Stanford University, Stanford, CA 94305, USA
| | - Claudia Kawas
- Department of Neurology, University of California, Irvine, Irvine, CA 92697, USA
| | - Maria Corrada
- Department of Neurology, University of California, Irvine, Irvine, CA 92697, USA
| | - Thomas Montine
- Department of Pathology, Stanford University, Stanford, CA 94305, USA
| | - James Zou
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA
- Department of Pathology, Stanford University, Stanford, CA 94305, USA
- Department of Biomedical Data Science, Stanford University, Stanford, CA 94305, USA
- Chan-Zuckerberg Biohub, San Francisco, CA 94158, USA
| |
Collapse
|
50
|
Cifci D, Foersch S, Kather JN. Artificial intelligence to identify genetic alterations in conventional histopathology. J Pathol 2022; 257:430-444. [PMID: 35342954 DOI: 10.1002/path.5898] [Citation(s) in RCA: 43] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Revised: 03/09/2022] [Accepted: 03/23/2022] [Indexed: 11/10/2022]
Abstract
Precision oncology relies on the identification of targetable molecular alterations in tumor tissues. In many tumor types, a limited set of molecular tests is currently part of standard diagnostic workflows. However, universal testing for all targetable alterations, especially rare ones, is limited by the cost and availability of molecular assays. From 2017 to 2021, multiple studies have shown that artificial intelligence (AI) methods can predict the probability of specific genetic alterations directly from conventional hematoxylin and eosin (H&E) tissue slides. Although these methods are currently less accurate than gold-standard testing (e.g. immunohistochemistry, polymerase chain reaction or next-generation sequencing), they could be used as pre-screening tools to reduce the workload of genetic analyses. In this systematic literature review, we summarize the state of the art in predicting molecular alterations from H&E using AI. We found that AI methods perform reasonably well across multiple tumor types, although few algorithms have been broadly validated. In addition, we found that genetic alterations in FGFR, IDH, PIK3CA, BRAF, TP53 and DNA repair pathways are predictable from H&E in multiple tumor types, while many other genetic alterations have rarely been investigated or were only poorly predictable. Finally, we discuss the next steps for the implementation of AI-based surrogate tests in diagnostic workflows. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Didem Cifci
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Sebastian Foersch
- Institute of Pathology, University Medical Center of the Johannes Gutenberg-University, Mainz, Germany
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.,Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.,Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
| |
Collapse
|