1
|
Tafavvoghi M, Bongo LA, Shvetsov N, Busund LTR, Møllersen K. Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review. J Pathol Inform 2024; 15:100363. [PMID: 38405160 PMCID: PMC10884505 DOI: 10.1016/j.jpi.2024.100363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 01/23/2024] [Indexed: 02/27/2024] Open
Abstract
Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.
Collapse
Affiliation(s)
- Masoud Tafavvoghi
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| | - Lars Ailo Bongo
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | - Nikita Shvetsov
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | | | - Kajsa Møllersen
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
2
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
3
|
Janitri V, ArulJothi KN, Ravi Mythili VM, Singh SK, Prasher P, Gupta G, Dua K, Hanumanthappa R, Karthikeyan K, Anand K. The roles of patient-derived xenograft models and artificial intelligence toward precision medicine. MedComm (Beijing) 2024; 5:e745. [PMID: 39329017 PMCID: PMC11424683 DOI: 10.1002/mco2.745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2024] [Revised: 08/22/2024] [Accepted: 08/22/2024] [Indexed: 09/28/2024] Open
Abstract
Patient-derived xenografts (PDX) involve transplanting patient cells or tissues into immunodeficient mice, offering superior disease models compared with cell line xenografts and genetically engineered mice. In contrast to traditional cell-line xenografts and genetically engineered mice, PDX models harbor the molecular and biologic features from the original patient tumor and are generationally stable. This high fidelity makes PDX models particularly suitable for preclinical and coclinical drug testing, therefore better predicting therapeutic efficacy. Although PDX models are becoming more useful, the several factors influencing their reliability and predictive power are not well understood. Several existing studies have looked into the possibility that PDX models could be important in enhancing our knowledge with regard to tumor genetics, biomarker discovery, and personalized medicine; however, a number of problems still need to be addressed, such as the high cost and time-consuming processes involved, together with the variability in tumor take rates. This review addresses these gaps by detailing the methodologies to generate PDX models, their application in cancer research, and their advantages over other models. Further, it elaborates on how artificial intelligence and machine learning were incorporated into PDX studies to fast-track therapeutic evaluation. This review is an overview of the progress that has been done so far in using PDX models for cancer research and shows their potential to be further improved in improving our understanding of oncogenesis.
Collapse
Affiliation(s)
| | - Kandasamy Nagarajan ArulJothi
- Department of Genetic Engineering, College of Engineering and TechnologySRM Institute of Science and TechnologyChengalpattuTamil NaduIndia
| | - Vijay Murali Ravi Mythili
- Department of Genetic Engineering, College of Engineering and TechnologySRM Institute of Science and TechnologyChengalpattuTamil NaduIndia
| | - Sachin Kumar Singh
- School of Pharmaceutical SciencesLovely Professional UniversityPhagwaraPunjabIndia
| | - Parteek Prasher
- Department of ChemistryUniversity of Petroleum & Energy Studies, Energy AcresDehradunIndia
| | - Gaurav Gupta
- Centre for Research Impact & Outcome, Chitkara College of PharmacyChitkara UniversityRajpuraPunjabIndia
| | - Kamal Dua
- Faculty of Health, Australian Research Center in Complementary and Integrative, MedicineUniversity of Technology SydneyUltimoNSWAustralia
- Discipline of Pharmacy, Graduate School of HealthUniversity of Technology SydneyUltimoNSWAustralia
| | - Rakshith Hanumanthappa
- JSS Banashankari Arts, Commerce, and SK Gubbi Science CollegeKarnatak UniversityDharwadKarnatakaIndia
| | - Karthikeyan Karthikeyan
- Centre of Excellence in PCB Design and Analysis, Department of Electronics and Communication EngineeringM. Kumarasamy College of EngineeringKarurTamil NaduIndia
| | - Krishnan Anand
- Department of Chemical Pathology, School of Pathology, Office of the Dean, Faculty of Health SciencesUniversity of the Free StateBloemfonteinSouth Africa
| |
Collapse
|
4
|
Jiang B, Bao L, He S, Chen X, Jin Z, Ye Y. Deep learning applications in breast cancer histopathological imaging: diagnosis, treatment, and prognosis. Breast Cancer Res 2024; 26:137. [PMID: 39304962 DOI: 10.1186/s13058-024-01895-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2024] [Accepted: 09/16/2024] [Indexed: 09/22/2024] Open
Abstract
Breast cancer is the most common malignant tumor among women worldwide and remains one of the leading causes of death among women. Its incidence and mortality rates are continuously rising. In recent years, with the rapid advancement of deep learning (DL) technology, DL has demonstrated significant potential in breast cancer diagnosis, prognosis evaluation, and treatment response prediction. This paper reviews relevant research progress and applies DL models to image enhancement, segmentation, and classification based on large-scale datasets from TCGA and multiple centers. We employed foundational models such as ResNet50, Transformer, and Hover-net to investigate the performance of DL models in breast cancer diagnosis, treatment, and prognosis prediction. The results indicate that DL techniques have significantly improved diagnostic accuracy and efficiency, particularly in predicting breast cancer metastasis and clinical prognosis. Furthermore, the study emphasizes the crucial role of robust databases in developing highly generalizable models. Future research will focus on addressing challenges related to data management, model interpretability, and regulatory compliance, ultimately aiming to provide more precise clinical treatment and prognostic evaluation programs for breast cancer patients.
Collapse
Affiliation(s)
- Bitao Jiang
- Department of Hematology and Oncology, Beilun District People's Hospital, Ningbo, 315800, China.
- Department of Hematology and Oncology, Beilun Branch of the First Affiliated Hospital of Zhejiang University, Ningbo, 315800, China.
| | - Lingling Bao
- Department of Hematology and Oncology, Beilun District People's Hospital, Ningbo, 315800, China
- Department of Hematology and Oncology, Beilun Branch of the First Affiliated Hospital of Zhejiang University, Ningbo, 315800, China
| | - Songqin He
- Department of Oncology, The 906th Hospital of the Joint Logistics Force of the Chinese People's Liberation Army, Ningbo, 315100, China
| | - Xiao Chen
- Department of Oncology, The 906th Hospital of the Joint Logistics Force of the Chinese People's Liberation Army, Ningbo, 315100, China
| | - Zhihui Jin
- Department of Hematology and Oncology, Beilun District People's Hospital, Ningbo, 315800, China
- Department of Hematology and Oncology, Beilun Branch of the First Affiliated Hospital of Zhejiang University, Ningbo, 315800, China
| | - Yingquan Ye
- Department of Oncology, The 906th Hospital of the Joint Logistics Force of the Chinese People's Liberation Army, Ningbo, 315100, China.
| |
Collapse
|
5
|
Hoang DT, Dinstag G, Shulman ED, Hermida LC, Ben-Zvi DS, Elis E, Caley K, Sammut SJ, Sinha S, Sinha N, Dampier CH, Stossel C, Patil T, Rajan A, Lassoued W, Strauss J, Bailey S, Allen C, Redman J, Beker T, Jiang P, Golan T, Wilkinson S, Sowalsky AG, Pine SR, Caldas C, Gulley JL, Aldape K, Aharonov R, Stone EA, Ruppin E. A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics. NATURE CANCER 2024; 5:1305-1317. [PMID: 38961276 DOI: 10.1038/s43018-024-00793-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 06/06/2024] [Indexed: 07/05/2024]
Abstract
Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an indirect two-step approach consisting of (1) DeepPT, a deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response to targeted and immune therapies from the inferred expression values. We show that DeepPT successfully predicts transcriptomics in all 16 The Cancer Genome Atlas cohorts tested and generalizes well to two independent datasets. ENLIGHT-DeepPT successfully predicts true responders in five independent patient cohorts involving four different treatments spanning six cancer types, with an overall odds ratio of 2.28 and a 39.5% increased response rate among predicted responders versus the baseline rate. Notably, its prediction accuracy, obtained without any training on the treatment data, is comparable to that achieved by directly predicting the response from the images, which requires specific training on the treatment evaluation cohorts.
Collapse
Affiliation(s)
- Danh-Tai Hoang
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia.
| | | | - Eldad D Shulman
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Leandro C Hermida
- Department of Immunology, University of Pittsburgh, Pittsburgh, PA, USA
- Tumor Microenvironment Center, UPMC Hillman Cancer Center, University of Pittsburgh, Pittsburgh, PA, USA
| | | | | | - Katherine Caley
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia
| | - Stephen-John Sammut
- Breast Cancer Now Toby Robins Research Centre, Institute of Cancer Research, London, UK
- The Royal Marsden Hospital NHS Foundation Trust, London, UK
| | - Sanju Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Neelam Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Christopher H Dampier
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Chani Stossel
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Tejas Patil
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Arun Rajan
- Thoracic and GI Malignancies Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Wiem Lassoued
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Julius Strauss
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Shania Bailey
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Clint Allen
- Surgical Oncology Program, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Jason Redman
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Peng Jiang
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Talia Golan
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Scott Wilkinson
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Adam G Sowalsky
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Sharon R Pine
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Carlos Caldas
- School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - James L Gulley
- Genitourinary Malignancy Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Kenneth Aldape
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Eric A Stone
- Biological Data Science Institute, College of Science, Australian National University, Canberra, Australian Capital Territory, Australia.
| | - Eytan Ruppin
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA.
| |
Collapse
|
6
|
Kumar RMR, Joghee S. Enhancing breast cancer treatment through pharmacogenomics: A narrative review. Clin Chim Acta 2024; 562:119893. [PMID: 39068964 DOI: 10.1016/j.cca.2024.119893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2024] [Revised: 07/24/2024] [Accepted: 07/25/2024] [Indexed: 07/30/2024]
Abstract
Pharmacogenomics has become integral to personalised medicine in breast cancer, utilising genetic insights to customize treatment strategies and enhance patient outcomes. Understanding how genetic variations influence drug metabolism, response, and toxicity is crucial for guiding treatment selection and dosing regimens. Genetic polymorphisms in drug-metabolizing enzymes and transporters significantly impact pharmacokinetic variability, influencing the efficacy and safety of chemotherapy agents and targeted therapies. Biomarkers associated with the hormone receptor status of breast cancer and mutations serve as key determinants of treatment response, aiding in the selection of therapies. Despite substantial progress in understanding the pharmacogenomic landscape of breast cancer, efforts to identify novel genetic markers and refine treatment optimisation strategies are required. Genome-wide association studies and advanced sequencing technologies hold promise for uncovering genetic determinants of drug response variability and elucidating complex pharmacogenomic interactions. The future of pharmacogenomics in breast cancer lies in real-time treatment monitoring, the discovery of additional predictive markers, and the seamless integration of pharmacogenomic data into clinical decision-making processes. However, translating pharmacogenomic discoveries into routine clinical practice requires collaborative efforts among stakeholders to address implementation challenges and ensure equitable access to genetic testing. By embracing pharmacogenomics, clinicians can tailor treatment approaches to individual patients, maximizing therapeutic benefits while minimizing adverse effects. This review discusses the integration of pharmacogenomics in breast cancer treatment, highlighting the significance of understanding genetic influences on treatment response and toxicity, and the potential of advanced technologies in refining treatment strategies.
Collapse
Affiliation(s)
- Ram Mohan Ram Kumar
- Department of Pharmaceutical Biotechnology, JSS College of Pharmacy, JSS Academy of Higher Education & Research, Mysuru, Karnataka, India.
| | - Suresh Joghee
- Department of Pharmacognosy, JSS College of Pharmacy, JSS Academy of Higher Education & Research, Mysuru, Karnataka, India
| |
Collapse
|
7
|
Calderaro J, Žigutytė L, Truhn D, Jaffe A, Kather JN. Artificial intelligence in liver cancer - new tools for research and patient management. Nat Rev Gastroenterol Hepatol 2024; 21:585-599. [PMID: 38627537 DOI: 10.1038/s41575-024-00919-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 03/11/2024] [Indexed: 07/31/2024]
Abstract
Liver cancer has high incidence and mortality globally. Artificial intelligence (AI) has advanced rapidly, influencing cancer care. AI systems are already approved for clinical use in some tumour types (for example, colorectal cancer screening). Crucially, research demonstrates that AI can analyse histopathology, radiology and natural language in liver cancer, and can replace manual tasks and access hidden information in routinely available clinical data. However, for liver cancer, few of these applications have translated into large-scale clinical trials or clinically approved products. Here, we advocate for the incorporation of AI in all stages of liver cancer management. We present a taxonomy of AI approaches in liver cancer, highlighting areas with academic and commercial potential, and outline a policy for AI-based liver cancer management, including interdisciplinary training of researchers, clinicians and patients. The potential of AI in liver cancer is immense, but effort is required to ensure that AI can fulfil expectations.
Collapse
Affiliation(s)
- Julien Calderaro
- Département de Pathologie, Assistance Publique Hôpitaux de Paris, Groupe Hospitalier Henri Mondor, Créteil, France
- Institut Mondor de Recherche Biomédicale, MINT-HEP Mondor Integrative Hepatology, Université Paris Est Créteil, Créteil, France
| | - Laura Žigutytė
- Else Kroener Fresenius Center for Digital Health (EKFZ), Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
| | - Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Ariel Jaffe
- Mayo Clinic, Rochester, MN, USA
- Department of Internal Medicine, Section of Digestive Diseases, Yale School of Medicine, New Haven, CT, USA
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health (EKFZ), Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Medical Oncology, National Center for Tumour Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
8
|
Chakrabarty N, Mahajan A. Imaging Analytics using Artificial Intelligence in Oncology: A Comprehensive Review. Clin Oncol (R Coll Radiol) 2024; 36:498-513. [PMID: 37806795 DOI: 10.1016/j.clon.2023.09.013] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 08/09/2023] [Accepted: 09/21/2023] [Indexed: 10/10/2023]
Abstract
The present era has seen a surge in artificial intelligence-related research in oncology, mainly using deep learning, because of powerful computer hardware, improved algorithms and the availability of large amounts of data from open-source domains and the use of transfer learning. Here we discuss the multifaceted role of deep learning in cancer care, ranging from risk stratification, the screening and diagnosis of cancer, to the prediction of genomic mutations, treatment response and survival outcome prediction, through the use of convolutional neural networks. Another role of artificial intelligence is in the generation of automated radiology reports, which is a boon in high-volume centres to minimise report turnaround time. Although a validated and deployable deep-learning model for clinical use is still in its infancy, there is ongoing research to overcome the barriers for its universal implementation and we also delve into this aspect. We also briefly describe the role of radiomics in oncoimaging. Artificial intelligence can provide answers pertaining to cancer management at baseline imaging, saving cost and time. Imaging biobanks, which are repositories of anonymised images, are also briefly described. We also discuss the commercialisation and ethical issues pertaining to artificial intelligence. The latest generation generalist artificial intelligence model is also briefly described at the end of the article. We believe this article will not only enrich knowledge, but also promote research acumen in the minds of readers to take oncoimaging to another level using artificial intelligence and also work towards clinical translation of such research.
Collapse
Affiliation(s)
- N Chakrabarty
- Department of Radiodiagnosis, Advanced Centre for Treatment, Research and Education in Cancer, Tata Memorial Centre, Homi Bhabha National Institute (HBNI), Parel, Mumbai, Maharashtra, India.
| | - A Mahajan
- The Clatterbridge Cancer Centre NHS Foundation Trust, Liverpool, UK.
| |
Collapse
|
9
|
Cui Y, Li Y, Miedema JR, Edmiston SN, Farag SW, Marron JS, Thomas NE. Region of Interest Detection in Melanocytic Skin Tumor Whole Slide Images-Nevus and Melanoma. Cancers (Basel) 2024; 16:2616. [PMID: 39123344 PMCID: PMC11311050 DOI: 10.3390/cancers16152616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2024] [Revised: 07/17/2024] [Accepted: 07/19/2024] [Indexed: 08/12/2024] Open
Abstract
Automated region of interest detection in histopathological image analysis is a challenging and important topic with tremendous potential impact on clinical practice. The deep learning methods used in computational pathology may help us to reduce costs and increase the speed and accuracy of cancer diagnosis. We started with the UNC Melanocytic Tumor Dataset cohort which contains 160 hematoxylin and eosin whole slide images of primary melanoma (86) and nevi (74). We randomly assigned 80% (134) as a training set and built an in-house deep learning method to allow for classification, at the slide level, of nevi and melanoma. The proposed method performed well on the other 20% (26) test dataset; the accuracy of the slide classification task was 92.3% and our model also performed well in terms of predicting the region of interest annotated by the pathologists, showing excellent performance of our model on melanocytic skin tumors. Even though we tested the experiments on a skin tumor dataset, our work could also be extended to other medical image detection problems to benefit the clinical evaluation and diagnosis of different tumors.
Collapse
Affiliation(s)
- Yi Cui
- Department of Economics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
| | - Yao Li
- Department of Statistics & Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (Y.L.); (J.S.M.)
| | - Jayson R. Miedema
- Department of Pathology and Laboratory Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
| | - Sharon N. Edmiston
- Lineberger Comprehensive Cancer Center, UNC School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
| | - Sherif W. Farag
- Eshelman School of Pharmacy, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
| | - James Stephen Marron
- Department of Statistics & Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (Y.L.); (J.S.M.)
- Lineberger Comprehensive Cancer Center, UNC School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Nancy E. Thomas
- Lineberger Comprehensive Cancer Center, UNC School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
- Department of Dermatology, UNC School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
10
|
Asadi-Aghbolaghi M, Darbandsari A, Zhang A, Contreras-Sanz A, Boschman J, Ahmadvand P, Köbel M, Farnell D, Huntsman DG, Churg A, Black PC, Wang G, Gilks CB, Farahani H, Bashashati A. Learning generalizable AI models for multi-center histopathology image classification. NPJ Precis Oncol 2024; 8:151. [PMID: 39030380 PMCID: PMC11271637 DOI: 10.1038/s41698-024-00652-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 07/11/2024] [Indexed: 07/21/2024] Open
Abstract
Investigation of histopathology slides by pathologists is an indispensable component of the routine diagnosis of cancer. Artificial intelligence (AI) has the potential to enhance diagnostic accuracy, improve efficiency, and patient outcomes in clinical pathology. However, variations in tissue preparation, staining protocols, and histopathology slide digitization could result in over-fitting of deep learning models when trained on the data from only one center, thereby underscoring the necessity to generalize deep learning networks for multi-center use. Several techniques, including the use of grayscale images, color normalization techniques, and Adversarial Domain Adaptation (ADA) have been suggested to generalize deep learning algorithms, but there are limitations to their effectiveness and discriminability. Convolutional Neural Networks (CNNs) exhibit higher sensitivity to variations in the amplitude spectrum, whereas humans predominantly rely on phase-related components for object recognition. As such, we propose Adversarial fourIer-based Domain Adaptation (AIDA) which applies the advantages of a Fourier transform in adversarial domain adaptation. We conducted a comprehensive examination of subtype classification tasks in four cancers, incorporating cases from multiple medical centers. Specifically, the datasets included multi-center data for 1113 ovarian cancer cases, 247 pleural cancer cases, 422 bladder cancer cases, and 482 breast cancer cases. Our proposed approach significantly improved performance, achieving superior classification results in the target domain, surpassing the baseline, color augmentation and normalization techniques, and ADA. Furthermore, extensive pathologist reviews suggested that our proposed approach, AIDA, successfully identifies known histotype-specific features. This superior performance highlights AIDA's potential in addressing generalization challenges in deep learning models for multi-center histopathology datasets.
Collapse
Affiliation(s)
| | - Amirali Darbandsari
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Allen Zhang
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | | | - Jeffrey Boschman
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Pouya Ahmadvand
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Martin Köbel
- Department of Pathology and Laboratory Medicine, University of Calgary, Calgary, AB, Canada
| | - David Farnell
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | - David G Huntsman
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- BC Cancer Research Institute, Vancouver, BC, Canada
| | - Andrew Churg
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | - Peter C Black
- Department of Urologic Sciences, University of British Columbia, Vancouver, BC, Canada
| | - Gang Wang
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| | - C Blake Gilks
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | - Hossein Farahani
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Ali Bashashati
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.
| |
Collapse
|
11
|
Sohrabei S, Moghaddasi H, Hosseini A, Ehsanzadeh SJ. Investigating the effects of artificial intelligence on the personalization of breast cancer management: a systematic study. BMC Cancer 2024; 24:852. [PMID: 39026174 PMCID: PMC11256548 DOI: 10.1186/s12885-024-12575-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 06/27/2024] [Indexed: 07/20/2024] Open
Abstract
BACKGROUND Providing appropriate specialized treatment to the right patient at the right time is considered necessary in cancer management. Targeted therapy tailored to the genetic changes of each breast cancer patient is a desirable feature of precision oncology, which can not only reduce disease progression but also potentially increase patient survival. The use of artificial intelligence alongside precision oncology can help physicians by identifying and selecting more effective treatment factors for patients. METHOD A systematic review was conducted using the PubMed, Embase, Scopus, and Web of Science databases in September 2023. We performed the search strategy with keywords, namely: Breast Cancer, Artificial intelligence, and precision Oncology along with their synonyms in the article titles. Descriptive, qualitative, review, and non-English studies were excluded. The quality assessment of the articles and evaluation of bias were determined based on the SJR journal and JBI indices, as well as the PRISMA2020 guideline. RESULTS Forty-six studies were selected that focused on personalized breast cancer management using artificial intelligence models. Seventeen studies using various deep learning methods achieved a satisfactory outcome in predicting treatment response and prognosis, contributing to personalized breast cancer management. Two studies utilizing neural networks and clustering provided acceptable indicators for predicting patient survival and categorizing breast tumors. One study employed transfer learning to predict treatment response. Twenty-six studies utilizing machine-learning methods demonstrated that these techniques can improve breast cancer classification, screening, diagnosis, and prognosis. The most frequent modeling techniques used were NB, SVM, RF, XGBoost, and Reinforcement Learning. The average area under the curve (AUC) for the models was 0.91. Moreover, the average values for accuracy, sensitivity, specificity, and precision were reported to be in the range of 90-96% for the models. CONCLUSION Artificial intelligence has proven to be effective in assisting physicians and researchers in managing breast cancer treatment by uncovering hidden patterns in complex omics and genetic data. Intelligent processing of omics data through protein and gene pattern classification and the utilization of deep neural patterns has the potential to significantly transform the field of complex disease management.
Collapse
Affiliation(s)
- Solmaz Sohrabei
- Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Hamid Moghaddasi
- Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Azamossadat Hosseini
- Department of Health Information Technology and Management, Health Information Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Seyed Jafar Ehsanzadeh
- Department of English Language, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
12
|
Jiao P, Zheng Q, Yang R, Ni X, Wu J, Chen Z, Liu X. Prediction of HER2 Status Based on Deep Learning in H&E-Stained Histopathology Images of Bladder Cancer. Biomedicines 2024; 12:1583. [PMID: 39062155 PMCID: PMC11274957 DOI: 10.3390/biomedicines12071583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2024] [Revised: 07/06/2024] [Accepted: 07/12/2024] [Indexed: 07/28/2024] Open
Abstract
Epidermal growth factor receptor 2 (HER2) has been widely recognized as one of the targets for bladder cancer immunotherapy. The key to implementing personalized treatment for bladder cancer patients lies in achieving rapid and accurate diagnosis. To tackle this challenge, we have pioneered the application of deep learning techniques to predict HER2 expression status from H&E-stained pathological images of bladder cancer, bypassing the need for intricate IHC staining or high-throughput sequencing methods. Our model, when subjected to rigorous testing within the cohort from the People's Hospital of Wuhan University, which encompasses 106 cases, has exhibited commendable performance on both the validation and test datasets. Specifically, the validation set yielded an AUC of 0.92, an accuracy of 0.86, a sensitivity of 0.87, a specificity of 0.83, and an F1 score of 86.7%. The corresponding metrics for the test set were 0.88 for AUC, 0.67 for accuracy, 0.56 for sensitivity, 0.75 for specificity, and 77.8% for F1 score. Additionally, in a direct comparison with pathologists, our model demonstrated statistically superior performance, with a p-value less than 0.05, highlighting its potential as a powerful diagnostic tool.
Collapse
Affiliation(s)
- Panpan Jiao
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Qingyuan Zheng
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Rui Yang
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Xinmiao Ni
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Jiejun Wu
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Zhiyuan Chen
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Xiuheng Liu
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| |
Collapse
|
13
|
White BS, Woo XY, Koc S, Sheridan T, Neuhauser SB, Wang S, Evrard YA, Chen L, Foroughi pour A, Landua JD, Mashl RJ, Davies SR, Fang B, Rosa MG, Evans KW, Bailey MH, Chen Y, Xiao M, Rubinstein JC, Sanderson BJ, Lloyd MW, Domanskyi S, Dobrolecki LE, Fujita M, Fujimoto J, Xiao G, Fields RC, Mudd JL, Xu X, Hollingshead MG, Jiwani S, Acevedo S, Davis-Dusenbery BN, Robinson PN, Moscow JA, Doroshow JH, Mitsiades N, Kaochar S, Pan CX, Carvajal-Carmona LG, Welm AL, Welm BE, Govindan R, Li S, Davies MA, Roth JA, Meric-Bernstam F, Xie Y, Herlyn M, Ding L, Lewis MT, Bult CJ, Dean DA, Chuang JH. A Pan-Cancer Patient-Derived Xenograft Histology Image Repository with Genomic and Pathologic Annotations Enables Deep Learning Analysis. Cancer Res 2024; 84:2060-2072. [PMID: 39082680 PMCID: PMC11217732 DOI: 10.1158/0008-5472.can-23-1349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 10/13/2023] [Accepted: 03/27/2024] [Indexed: 08/04/2024]
Abstract
Patient-derived xenografts (PDX) model human intra- and intertumoral heterogeneity in the context of the intact tissue of immunocompromised mice. Histologic imaging via hematoxylin and eosin (H&E) staining is routinely performed on PDX samples, which could be harnessed for computational analysis. Prior studies of large clinical H&E image repositories have shown that deep learning analysis can identify intercellular and morphologic signals correlated with disease phenotype and therapeutic response. In this study, we developed an extensive, pan-cancer repository of >1,000 PDX and paired parental tumor H&E images. These images, curated from the PDX Development and Trial Centers Research Network Consortium, had a range of associated genomic and transcriptomic data, clinical metadata, pathologic assessments of cell composition, and, in several cases, detailed pathologic annotations of neoplastic, stromal, and necrotic regions. The amenability of these images to deep learning was highlighted through three applications: (i) development of a classifier for neoplastic, stromal, and necrotic regions; (ii) development of a predictor of xenograft-transplant lymphoproliferative disorder; and (iii) application of a published predictor of microsatellite instability. Together, this PDX Development and Trial Centers Research Network image repository provides a valuable resource for controlled digital pathology analysis, both for the evaluation of technical issues and for the development of computational image-based methods that make clinical predictions based on PDX treatment studies. Significance: A pan-cancer repository of >1,000 patient-derived xenograft hematoxylin and eosin-stained images will facilitate cancer biology investigations through histopathologic analysis and contributes important model system data that expand existing human histology repositories.
Collapse
Affiliation(s)
- Brian S. White
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
| | - Xing Yi Woo
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
- Bioinformatics Institute (BII), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore.
| | - Soner Koc
- Velsera, Charlestown, Massachusetts.
| | - Todd Sheridan
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
| | | | - Shidan Wang
- University of Texas Southwestern Medical Center, Dallas, Texas.
| | - Yvonne A. Evrard
- Leidos Biomedical Research Inc., Frederick National Laboratory for Cancer Research, Frederick, Maryland.
| | - Li Chen
- Leidos Biomedical Research Inc., Frederick National Laboratory for Cancer Research, Frederick, Maryland.
| | - Ali Foroughi pour
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
| | | | - R. Jay Mashl
- Washington University School of Medicine, St. Louis, Missouri.
| | | | - Bingliang Fang
- University of Texas MD Anderson Cancer Center, Houston, Texas.
| | | | - Kurt W. Evans
- University of Texas MD Anderson Cancer Center, Houston, Texas.
| | - Matthew H. Bailey
- Simmons Center for Cancer Research, Brigham Young University, Provo, Utah.
| | - Yeqing Chen
- The Wistar Institute, Philadelphia, Pennsylvania.
| | - Min Xiao
- The Wistar Institute, Philadelphia, Pennsylvania.
| | | | | | | | - Sergii Domanskyi
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
| | | | - Maihi Fujita
- Huntsman Cancer Institute, University of Utah, Salt Lake City, Utah.
| | - Junya Fujimoto
- University of Texas MD Anderson Cancer Center, Houston, Texas.
| | - Guanghua Xiao
- University of Texas Southwestern Medical Center, Dallas, Texas.
| | - Ryan C. Fields
- Washington University School of Medicine, St. Louis, Missouri.
| | | | - Xiaowei Xu
- The Wistar Institute, Philadelphia, Pennsylvania.
| | | | - Shahanawaz Jiwani
- Leidos Biomedical Research Inc., Frederick National Laboratory for Cancer Research, Frederick, Maryland.
| | | | | | | | - Peter N. Robinson
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
| | | | | | | | | | | | | | - Alana L. Welm
- Huntsman Cancer Institute, University of Utah, Salt Lake City, Utah.
| | - Bryan E. Welm
- Huntsman Cancer Institute, University of Utah, Salt Lake City, Utah.
| | | | - Shunqiang Li
- Washington University School of Medicine, St. Louis, Missouri.
| | | | - Jack A. Roth
- University of Texas MD Anderson Cancer Center, Houston, Texas.
| | | | - Yang Xie
- University of Texas Southwestern Medical Center, Dallas, Texas.
| | | | - Li Ding
- Washington University School of Medicine, St. Louis, Missouri.
| | | | | | | | - Jeffrey H. Chuang
- The Jackson Laboratory for Genomic Medicine, Farmington, Connecticut.
| |
Collapse
|
14
|
Yan C, Sun J, Guan Y, Feng J, Liu H, Liu J. PhiHER2: phenotype-informed weakly supervised model for HER2 status prediction from pathological images. Bioinformatics 2024; 40:i79-i90. [PMID: 38940163 PMCID: PMC11211833 DOI: 10.1093/bioinformatics/btae236] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/29/2024] Open
Abstract
MOTIVATION Human epidermal growth factor receptor 2 (HER2) status identification enables physicians to assess the prognosis risk and determine the treatment schedule for patients. In clinical practice, pathological slides serve as the gold standard, offering morphological information on cellular structure and tumoral regions. Computational analysis of pathological images has the potential to discover morphological patterns associated with HER2 molecular targets and achieve precise status prediction. However, pathological images are typically equipped with high-resolution attributes, and HER2 expression in breast cancer (BC) images often manifests the intratumoral heterogeneity. RESULTS We present a phenotype-informed weakly supervised multiple instance learning architecture (PhiHER2) for the prediction of the HER2 status from pathological images of BC. Specifically, a hierarchical prototype clustering module is designed to identify representative phenotypes across whole slide images. These phenotype embeddings are then integrated into a cross-attention module, enhancing feature interaction and aggregation on instances. This yields a phenotype-based feature space that leverages the intratumoral morphological heterogeneity for HER2 status prediction. Extensive results demonstrate that PhiHER2 captures a better WSI-level representation by the typical phenotype guidance and significantly outperforms existing methods on real-world datasets. Additionally, interpretability analyses of both phenotypes and WSIs provide explicit insights into the heterogeneity of morphological patterns associated with molecular HER2 status. AVAILABILITY AND IMPLEMENTATION Our model is available at https://github.com/lyotvincent/PhiHER2.
Collapse
Affiliation(s)
- Chaoyang Yan
- College of Computer Science, Nankai University, Tianjin 300071, China
- Centre for Bioinformatics and Intelligent Medicine, Nankai University, Tianjin 300071, China
| | - Jialiang Sun
- College of Computer Science, Nankai University, Tianjin 300071, China
- Centre for Bioinformatics and Intelligent Medicine, Nankai University, Tianjin 300071, China
| | - Yiming Guan
- College of Computer Science, Nankai University, Tianjin 300071, China
- Centre for Bioinformatics and Intelligent Medicine, Nankai University, Tianjin 300071, China
| | - Jiuxin Feng
- College of Computer Science, Nankai University, Tianjin 300071, China
- Centre for Bioinformatics and Intelligent Medicine, Nankai University, Tianjin 300071, China
| | - Hong Liu
- The Second Surgical Department of Breast Cancer, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute & Hospital, Tianjin 300060, China
| | - Jian Liu
- College of Computer Science, Nankai University, Tianjin 300071, China
- Centre for Bioinformatics and Intelligent Medicine, Nankai University, Tianjin 300071, China
| |
Collapse
|
15
|
Kiyuna T, Cosatto E, Hatanaka KC, Yokose T, Tsuta K, Motoi N, Makita K, Shimizu A, Shinohara T, Suzuki A, Takakuwa E, Takakuwa Y, Tsuji T, Tsujiwaki M, Yanai M, Yuzawa S, Ogura M, Hatanaka Y. Evaluating Cellularity Estimation Methods: Comparing AI Counting with Pathologists' Visual Estimates. Diagnostics (Basel) 2024; 14:1115. [PMID: 38893641 PMCID: PMC11171606 DOI: 10.3390/diagnostics14111115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2024] [Revised: 05/17/2024] [Accepted: 05/21/2024] [Indexed: 06/21/2024] Open
Abstract
The development of next-generation sequencing (NGS) has enabled the discovery of cancer-specific driver gene alternations, making precision medicine possible. However, accurate genetic testing requires a sufficient amount of tumor cells in the specimen. The evaluation of tumor content ratio (TCR) from hematoxylin and eosin (H&E)-stained images has been found to vary between pathologists, making it an important challenge to obtain an accurate TCR. In this study, three pathologists exhaustively labeled all cells in 41 regions from 41 lung cancer cases as either tumor, non-tumor or indistinguishable, thus establishing a "gold standard" TCR. We then compared the accuracy of the TCR estimated by 13 pathologists based on visual assessment and the TCR calculated by an AI model that we have developed. It is a compact and fast model that follows a fully convolutional neural network architecture and produces cell detection maps which can be efficiently post-processed to obtain tumor and non-tumor cell counts from which TCR is calculated. Its raw cell detection accuracy is 92% while its classification accuracy is 84%. The results show that the error between the gold standard TCR and the AI calculation was significantly smaller than that between the gold standard TCR and the pathologist's visual assessment (p<0.05). Additionally, the robustness of AI models across institutions is a key issue and we demonstrate that the variation in AI was smaller than that in the average of pathologists when evaluated by institution. These findings suggest that the accuracy of tumor cellularity assessments in clinical workflows is significantly improved by the introduction of robust AI models, leading to more efficient genetic testing and ultimately to better patient outcomes.
Collapse
Affiliation(s)
- Tomoharu Kiyuna
- Healthcare Life Science Division, NEC Corporation, Tokyo 108-8556, Japan;
| | - Eric Cosatto
- Department of Machine Learning, NEC Laboratories America, Princeton, NJ 08540, USA;
| | - Kanako C. Hatanaka
- Center for Development of Advanced Diagnostics (C-DAD), Hokkaido University Hospital, Sapporo 060-8648, Japan;
| | - Tomoyuki Yokose
- Department of Pathology, Kanagawa Cancer Center, Yokohama 241-8515, Japan;
| | - Koji Tsuta
- Department of Pathology, Kansai Medical University, Osaka 573-1010, Japan;
| | - Noriko Motoi
- Department of Pathology, Saitama Cancer Center, Saitama 362-0806, Japan;
| | - Keishi Makita
- Department of Pathology, Oji General Hospital, Tomakomai 053-8506, Japan
| | - Ai Shimizu
- Department of Surgical Pathology, Hokkaido University Hospital, Sapporo 060-8648, Japan; (A.S.); (E.T.)
| | - Toshiya Shinohara
- Department of Pathology, Teine Keijinkai Hospital, Sapporo 006-0811, Japan
| | - Akira Suzuki
- Department of Pathology, KKR Sapporo Medical Center, Sapporo 062-0931, Japan
| | - Emi Takakuwa
- Department of Surgical Pathology, Hokkaido University Hospital, Sapporo 060-8648, Japan; (A.S.); (E.T.)
| | - Yasunari Takakuwa
- Department of Pathology, NTT Medical Center Sapporo, Sapporo 060-0061, Japan;
| | - Takahiro Tsuji
- Department of Pathology, Sapporo City General Hospital, Sapporo 060-8604, Japan;
| | - Mitsuhiro Tsujiwaki
- Department of Surgical Pathology, Sapporo Medical University Hospital, Sapporo 060-8543, Japan;
| | - Mitsuru Yanai
- Department of Pathology, Sapporo Tokushukai Hospital, Sapporo 004-0041, Japan;
| | - Sayaka Yuzawa
- Department of Diagnostic Pathology, Asahikawa Medical University Hospital, Asahikawa 078-8510, Japan
| | - Maki Ogura
- Healthcare Life Science Division, NEC Corporation, Tokyo 108-8556, Japan;
| | - Yutaka Hatanaka
- Center for Development of Advanced Diagnostics (C-DAD), Hokkaido University Hospital, Sapporo 060-8648, Japan;
| |
Collapse
|
16
|
Ivanova M, Pescia C, Trapani D, Venetis K, Frascarelli C, Mane E, Cursano G, Sajjadi E, Scatena C, Cerbelli B, d’Amati G, Porta FM, Guerini-Rocco E, Criscitiello C, Curigliano G, Fusco N. Early Breast Cancer Risk Assessment: Integrating Histopathology with Artificial Intelligence. Cancers (Basel) 2024; 16:1981. [PMID: 38893102 PMCID: PMC11171409 DOI: 10.3390/cancers16111981] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Revised: 05/13/2024] [Accepted: 05/17/2024] [Indexed: 06/21/2024] Open
Abstract
Effective risk assessment in early breast cancer is essential for informed clinical decision-making, yet consensus on defining risk categories remains challenging. This paper explores evolving approaches in risk stratification, encompassing histopathological, immunohistochemical, and molecular biomarkers alongside cutting-edge artificial intelligence (AI) techniques. Leveraging machine learning, deep learning, and convolutional neural networks, AI is reshaping predictive algorithms for recurrence risk, thereby revolutionizing diagnostic accuracy and treatment planning. Beyond detection, AI applications extend to histological subtyping, grading, lymph node assessment, and molecular feature identification, fostering personalized therapy decisions. With rising cancer rates, it is crucial to implement AI to accelerate breakthroughs in clinical practice, benefiting both patients and healthcare providers. However, it is important to recognize that while AI offers powerful automation and analysis tools, it lacks the nuanced understanding, clinical context, and ethical considerations inherent to human pathologists in patient care. Hence, the successful integration of AI into clinical practice demands collaborative efforts between medical experts and computational pathologists to optimize patient outcomes.
Collapse
Affiliation(s)
- Mariia Ivanova
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Carlo Pescia
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Dario Trapani
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, 20141 Milan, Italy; (D.T.); (C.C.); (G.C.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Konstantinos Venetis
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Chiara Frascarelli
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Eltjona Mane
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Giulia Cursano
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Elham Sajjadi
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Cristian Scatena
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56126 Pisa, Italy;
| | - Bruna Cerbelli
- Department of Medical-Surgical Sciences and Biotechnologies, Sapienza University of Rome, 00185 Rome, Italy;
| | - Giulia d’Amati
- Department of Radiological, Oncological and Pathological Sciences, Sapienza University of Rome, 00185 Rome, Italy;
| | - Francesca Maria Porta
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Elena Guerini-Rocco
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Carmen Criscitiello
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, 20141 Milan, Italy; (D.T.); (C.C.); (G.C.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Giuseppe Curigliano
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, 20141 Milan, Italy; (D.T.); (C.C.); (G.C.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Nicola Fusco
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| |
Collapse
|
17
|
Cui Y, Li Y, Miedema JR, Edmiston SN, Farag S, Marron JS, Thomas NE. Region of Interest Detection in Melanocytic Skin Tumor Whole Slide Images - Nevus & Melanoma. ARXIV 2024:arXiv:2405.09851v1. [PMID: 38800658 PMCID: PMC11118677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Automated region of interest detection in histopathological image analysis is a challenging and important topic with tremendous potential impact on clinical practice. The deep-learning methods used in computational pathology may help us to reduce costs and increase the speed and accuracy of cancer diagnosis. We started with the UNC Melanocytic Tumor Dataset cohort that contains 160 hematoxylin and eosin whole-slide images of primary melanomas (86) and nevi (74). We randomly assigned 80% (134) as a training set and built an in-house deep-learning method to allow for classification, at the slide level, of nevi and melanomas. The proposed method performed well on the other 20% (26) test dataset; the accuracy of the slide classification task was 92.3% and our model also performed well in terms of predicting the region of interest annotated by the pathologists, showing excellent performance of our model on melanocytic skin tumors. Even though we tested the experiments on the skin tumor dataset, our work could also be extended to other medical image detection problems to benefit the clinical evaluation and diagnosis of different tumors.
Collapse
Affiliation(s)
- Yi Cui
- Department of Economics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Yao Li
- Department of Statistics & Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Jayson R Miedema
- School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA
| | - Sharon N Edmiston
- Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Sherif Farag
- School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA
| | - J S Marron
- Department of Statistics & Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
- School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA
| | - Nancy E Thomas
- School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA
| |
Collapse
|
18
|
McCaffrey C, Jahangir C, Murphy C, Burke C, Gallagher WM, Rahman A. Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer. Expert Rev Mol Diagn 2024; 24:363-377. [PMID: 38655907 DOI: 10.1080/14737159.2024.2346545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Accepted: 04/19/2024] [Indexed: 04/26/2024]
Abstract
INTRODUCTION Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes. AREAS COVERED In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings. EXPERT OPINION The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.
Collapse
Affiliation(s)
- Christine McCaffrey
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Chowdhury Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Clodagh Murphy
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Caoimbhe Burke
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Arman Rahman
- UCD School of Medicine, UCD Conway Institute, University College Dublin, Dublin, Ireland
| |
Collapse
|
19
|
Wang Q, Bi Q, Qu L, Deng Y, Wang X, Zheng Y, Li C, Meng Q, Miao K. MAMILNet: advancing precision oncology with multi-scale attentional multi-instance learning for whole slide image analysis. Front Oncol 2024; 14:1275769. [PMID: 38746682 PMCID: PMC11092915 DOI: 10.3389/fonc.2024.1275769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 04/08/2024] [Indexed: 08/28/2024] Open
Abstract
BACKGROUND Whole Slide Image (WSI) analysis, driven by deep learning algorithms, has the potential to revolutionize tumor detection, classification, and treatment response prediction. However, challenges persist, such as limited model generalizability across various cancer types, the labor-intensive nature of patch-level annotation, and the necessity of integrating multi-magnification information to attain a comprehensive understanding of pathological patterns. METHODS In response to these challenges, we introduce MAMILNet, an innovative multi-scale attentional multi-instance learning framework for WSI analysis. The incorporation of attention mechanisms into MAMILNet contributes to its exceptional generalizability across diverse cancer types and prediction tasks. This model considers whole slides as "bags" and individual patches as "instances." By adopting this approach, MAMILNet effectively eliminates the requirement for intricate patch-level labeling, significantly reducing the manual workload for pathologists. To enhance prediction accuracy, the model employs a multi-scale "consultation" strategy, facilitating the aggregation of test outcomes from various magnifications. RESULTS Our assessment of MAMILNet encompasses 1171 cases encompassing a wide range of cancer types, showcasing its effectiveness in predicting complex tasks. Remarkably, MAMILNet achieved impressive results in distinct domains: for breast cancer tumor detection, the Area Under the Curve (AUC) was 0.8872, with an Accuracy of 0.8760. In the realm of lung cancer typing diagnosis, it achieved an AUC of 0.9551 and an Accuracy of 0.9095. Furthermore, in predicting drug therapy responses for ovarian cancer, MAMILNet achieved an AUC of 0.7358 and an Accuracy of 0.7341. CONCLUSION The outcomes of this study underscore the potential of MAMILNet in driving the advancement of precision medicine and individualized treatment planning within the field of oncology. By effectively addressing challenges related to model generalization, annotation workload, and multi-magnification integration, MAMILNet shows promise in enhancing healthcare outcomes for cancer patients. The framework's success in accurately detecting breast tumors, diagnosing lung cancer types, and predicting ovarian cancer therapy responses highlights its significant contribution to the field and paves the way for improved patient care.
Collapse
Affiliation(s)
- Qinqing Wang
- Department of Pathology, The First People’s Hospital of Yunnan Province, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Qiu Bi
- Department of MRI, The First People’s Hospital of Yunnan Province, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| | | | - Yuchen Deng
- School of Clinical Medicine, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Xianhong Wang
- School of Clinical Medicine, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Yijun Zheng
- School of Clinical Medicine, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Chenrong Li
- School of Clinical Medicine, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Qingyin Meng
- Department of Pathology, The Third Affiliated Hospital of Kunming Medical University (Tumor Hospital of Yunnan Province), Kunming, Yunnan, China
| | - Kun Miao
- Department of Medical Oncology, The First People’s Hospital of Yunnan Province, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, Yunnan, China
| |
Collapse
|
20
|
Tanei T, Seno S, Sota Y, Hatano T, Kitahara Y, Abe K, Masunaga N, Tsukabe M, Yoshinami T, Miyake T, Shimoda M, Matsuda H, Shimazu K. High HER2 Intratumoral Heterogeneity Is a Predictive Factor for Poor Prognosis in Early-Stage and Locally Advanced HER2-Positive Breast Cancer. Cancers (Basel) 2024; 16:1062. [PMID: 38473420 DOI: 10.3390/cancers16051062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 02/20/2024] [Accepted: 02/26/2024] [Indexed: 03/14/2024] Open
Abstract
PURPOSE Breast cancer tumors frequently have intratumoral heterogeneity (ITH). Tumors with high ITH cause therapeutic resistance and have human epidermal growth factor receptor 2 (HER2) heterogeneity in response to HER2-targeted therapies. This study aimed to investigate whether high HER2 heterogeneity levels were clinically related to a poor prognosis for HER2-targeted adjuvant therapy resistance in primary breast cancers. METHODS This study included patients with primary breast cancer (n = 251) treated with adjuvant HER2-targeted therapies. HER2 heterogeneity was manifested by the shape of HER2 fluorescence in situ hybridization amplification (FISH) distributed histograms with the HER2 gene copy number within a tumor sample. Each tumor was classified into a biphasic grade graph (high heterogeneity [HH]) group or a monophasic grade graph (low heterogeneity [LH]) group based on heterogeneity. Both groups were evaluated for disease-free survival (DFS) and overall survival (OS) for a median of ten years of annual follow-up. RESULTS Of 251 patients with HER2-positive breast cancer, 46 (18.3%) and 205 (81.7%) were classified into the HH and LH groups, respectively. The HH group had more distant metastases and a poorer prognosis than the LH group (DFS: p < 0.001 (HH:63% vs. LH:91% at 10 years) and for the OS: p = 0.012 (HH:78% vs. LH:95% at 10 years). CONCLUSIONS High HER2 heterogeneity is a poor prognostic factor in patients with HER2-positive breast cancer. A novel approach to heterogeneity, which is manifested by the shape of HER2 FISH distributions, might be clinically useful in the prognosis prediction of patients after HER2 adjuvant therapy.
Collapse
Affiliation(s)
- Tomonori Tanei
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Shigeto Seno
- Department of Bioinformatic Engineering, Graduate School of Information Science and Technology, Osaka University, 1-5 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Yoshiaki Sota
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Takaaki Hatano
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Yuri Kitahara
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Kaori Abe
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Nanae Masunaga
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Masami Tsukabe
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Tetsuhiro Yoshinami
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Tomohiro Miyake
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Masafumi Shimoda
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Hideo Matsuda
- Department of Bioinformatic Engineering, Graduate School of Information Science and Technology, Osaka University, 1-5 Yamadaoka, Suita 565-0871, Osaka, Japan
| | - Kenzo Shimazu
- Department of Breast and Endocrine Surgery, Graduate School of Medicine, Osaka University, 2-2-E10 Yamadaoka, Suita 565-0871, Osaka, Japan
| |
Collapse
|
21
|
Ayana G, Lee E, Choe SW. Vision Transformers for Breast Cancer Human Epidermal Growth Factor Receptor 2 Expression Staging without Immunohistochemical Staining. THE AMERICAN JOURNAL OF PATHOLOGY 2024; 194:402-414. [PMID: 38096984 DOI: 10.1016/j.ajpath.2023.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 10/10/2023] [Accepted: 11/20/2023] [Indexed: 12/31/2023]
Abstract
Accurate staging of human epidermal growth factor receptor 2 (HER2) expression is vital for evaluating breast cancer treatment efficacy. However, it typically involves costly and complex immunohistochemical staining, along with hematoxylin and eosin staining. This work presents customized vision transformers for staging HER2 expression in breast cancer using only hematoxylin and eosin-stained images. The proposed algorithm comprised three modules: a localization module for weakly localizing critical image features using spatial transformers, an attention module for global learning via vision transformers, and a loss module to determine proximity to a HER2 expression level based on input images by calculating ordinal loss. Results, reported with 95% CIs, reveal the proposed approach's success in HER2 expression staging: area under the receiver operating characteristic curve, 0.9202 ± 0.01; precision, 0.922 ± 0.01; sensitivity, 0.876 ± 0.01; and specificity, 0.959 ± 0.02 over fivefold cross-validation. Comparatively, this approach significantly outperformed conventional vision transformer models and state-of-the-art convolutional neural network models (P < 0.001). Furthermore, it surpassed existing methods when evaluated on an independent test data set. This work holds great importance, aiding HER2 expression staging in breast cancer treatment while circumventing the costly and time-consuming immunohistochemical staining procedure, thereby addressing diagnostic disparities in low-resource settings and low-income countries.
Collapse
Affiliation(s)
- Gelan Ayana
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea; School of Biomedical Engineering, Jimma University, Jimma, Ethiopia
| | - Eonjin Lee
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea
| | - Se-Woon Choe
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea; Department of IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea.
| |
Collapse
|
22
|
Liu H, Xie X, Wang B. Deep learning infers clinically relevant protein levels and drug response in breast cancer from unannotated pathology images. NPJ Breast Cancer 2024; 10:18. [PMID: 38413598 PMCID: PMC10899601 DOI: 10.1038/s41523-024-00620-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 02/02/2024] [Indexed: 02/29/2024] Open
Abstract
The computational pathology has been demonstrated to effectively uncover tumor-related genomic alterations and transcriptomic patterns. Although proteomics has indeed shown great potential in the field of precision medicine, few studies have focused on the computational prediction of protein levels from pathology images. In this paper, we assume that deep learning-based pathological features imply the protein levels of tumor biomarkers that are indicative of prognosis and drug response. For this purpose, we propose wsi2rppa, a weakly supervised contrastive learning framework to infer the protein levels of tumor biomarkers from whole slide images (WSIs) in breast cancer. We first conducted contrastive learning-based pre-training on tessellated tiles to extract pathological features, which are then aggregated by attention pooling and adapted to downstream tasks. We conducted extensive evaluation experiments on the TCGA-BRCA cohort (1978 WSIs of 1093 patients with protein levels of 223 biomarkers) and the CPTAC-BRCA cohort (642 WSIs of 134 patients). The results showed that our method achieved state-of-the-art performance in tumor diagnostic tasks, and also performed well in predicting clinically relevant protein levels and drug response. To show the model interpretability, we spatially visualized the WSIs colored the tiles by their attention scores, and found that the regions with high scores were highly consistent with the tumor and necrotic regions annotated by a 10-year experienced pathologist. Moreover, spatial transcriptomic data further verified that the heatmap generated by attention scores agrees greatly with the spatial expression landscape of two typical tumor biomarker genes. In predicting the response to drug trastuzumab treatment, our method achieved a 0.79 AUC value which is much higher than the previous study reported 0.68. These findings showed the remarkable potential of computational pathology in the prediction of clinically relevant protein levels, drug response, and clinical outcomes.
Collapse
Affiliation(s)
- Hui Liu
- College of Computer and Information Engineering, Nanjing Tech University, 211816, Nanjing, Jiangsu, China
| | - Xiaodong Xie
- College of Computer and Information Engineering, Nanjing Tech University, 211816, Nanjing, Jiangsu, China
| | - Bin Wang
- Department of Cardiothoracic Surgery, The Third Affiliated Hospital of Soochow University, 213110, Changzhou, Jiangsu, China.
| |
Collapse
|
23
|
Samueli B, Aizenberg N, Shaco-Levy R, Katzav A, Kezerle Y, Krausz J, Mazareb S, Niv-Drori H, Peled HB, Sabo E, Tobar A, Asa SL. Complete digital pathology transition: A large multi-center experience. Pathol Res Pract 2024; 253:155028. [PMID: 38142526 DOI: 10.1016/j.prp.2023.155028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 12/08/2023] [Indexed: 12/26/2023]
Abstract
INTRODUCTION Transitioning from glass slide pathology to digital pathology for primary diagnostics requires an appropriate laboratory information system, an image management system, and slide scanners; it also reinforces the need for sophisticated pathology informatics including synoptic reporting. Previous reports have discussed the transition itself and relevant considerations for it, but not the selection criteria and considerations for the infrastructure. OBJECTIVE To describe the process used to evaluate slide scanners, image management systems, and synoptic reporting systems for a large multisite institution. METHODS Six network hospitals evaluated six slide scanners, three image management systems, and three synoptic reporting systems. Scanners were evaluated based on the quality of image, speed, ease of operation, and special capabilities (including z-stacking, fluorescence and others). Image management and synoptic reporting systems were evaluated for their ease of use and capacity. RESULTS Among the scanners evaluated, the Leica GT450 produced the highest quality images, while the 3DHistech Pannoramic provided fluorescence and superior z-stacking. The newest generation of scanners, released relatively recently, performed better than slightly older scanners from major manufacturers Although the Olympus VS200 was not fully vetted due to not meeting all inclusion criteria, it is discussed herein due to its exceptional versatility. For Image Management Software, the authors believe that Sectra is, at the time of writing the best developed option, but this could change in the very near future as other systems improve their capabilities. All synoptic reporting systems performed impressively. CONCLUSIONS Specifics regarding quality and abilities of different components will change rapidly with time, but large pathology practices considering such a transition should be aware of the issues discussed and evaluate the most current generation to arrive at appropriate conclusions.
Collapse
Affiliation(s)
- Benzion Samueli
- Department of Pathology, Soroka University Medical Center, P.O. Box 151, Be'er Sheva 8410101, Israel; Faculty of Health Sciences, Ben Gurion University of the Negev, P.O. Box 653, Be'er Sheva 8410501, Israel.
| | - Natalie Aizenberg
- Department of Pathology, Soroka University Medical Center, P.O. Box 151, Be'er Sheva 8410101, Israel; Faculty of Health Sciences, Ben Gurion University of the Negev, P.O. Box 653, Be'er Sheva 8410501, Israel
| | - Ruthy Shaco-Levy
- Department of Pathology, Soroka University Medical Center, P.O. Box 151, Be'er Sheva 8410101, Israel; Faculty of Health Sciences, Ben Gurion University of the Negev, P.O. Box 653, Be'er Sheva 8410501, Israel; Department of Pathology, Barzilai Medical Center, 2 Ha-Histadrut St, Ashkelon 7830604, Israel
| | - Aviva Katzav
- Pathology Institute, Meir Medical Center, Kfar Saba 4428164, Israel
| | - Yarden Kezerle
- Department of Pathology, Soroka University Medical Center, P.O. Box 151, Be'er Sheva 8410101, Israel; Faculty of Health Sciences, Ben Gurion University of the Negev, P.O. Box 653, Be'er Sheva 8410501, Israel
| | - Judit Krausz
- Department of Pathology, HaEmek Medical Center, 21 Yitzhak Rabin Ave, Afula 183411, Israel
| | - Salam Mazareb
- Department of Pathology, Carmel Medical Center, 7 Michal Street, Haifa 3436212, Israel
| | - Hagit Niv-Drori
- Department of Pathology, Rabin Medical Center, 39 Jabotinsky St, Petah Tikva 4941492, Israel; Faculty of Medicine, Tel Aviv University, P.O. Box 39040, Tel Aviv 6139001, Israel
| | - Hila Belhanes Peled
- Department of Pathology, HaEmek Medical Center, 21 Yitzhak Rabin Ave, Afula 183411, Israel
| | - Edmond Sabo
- Department of Pathology, Carmel Medical Center, 7 Michal Street, Haifa 3436212, Israel; Rappaport Faculty of Medicine, Technion Israel Institute of Technology, Haifa 3525433, Israel
| | - Ana Tobar
- Department of Pathology, Rabin Medical Center, 39 Jabotinsky St, Petah Tikva 4941492, Israel; Faculty of Medicine, Tel Aviv University, P.O. Box 39040, Tel Aviv 6139001, Israel
| | - Sylvia L Asa
- Institute of Pathology, University Hospitals Cleveland Medical Center, Case Western Reserve University, 11100 Euclid Avenue, Room 204, Cleveland, OH 44106, USA
| |
Collapse
|
24
|
Ng CW, Wong KK. Deep learning-enabled breast cancer endocrine response determination from H&E staining based on ESR1 signaling activity. Sci Rep 2023; 13:21454. [PMID: 38052873 PMCID: PMC10698147 DOI: 10.1038/s41598-023-48830-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 11/30/2023] [Indexed: 12/07/2023] Open
Abstract
Estrogen receptor (ER) positivity by immunohistochemistry has long been a main selection criterium for breast cancer patients to be treated with endocrine therapy. However, ER positivity might not directly correlate with activated ER signaling activity, which is a better predictor for endocrine therapy responsiveness. In this study, we investigated if a deep learning method using whole-slide H&E-stained images could predict ER signaling activity. First, ER signaling activity score was determined using RNAseq data available from each of the 1082 breast cancer samples in the TCGA Pan-Cancer dataset based on the Hallmark Estrogen Response Early gene set from the Molecular Signature Database (MSigDB). Then the processed H&E-stained images and ER signaling activity scores from a training cohort were fed into ResNet101 with three additional fully connected layers to generate a predicted ER activity score. The trained models were subsequently applied to an independent testing cohort. The result demonstrated that ER + /HER2- breast cancer patients with a higher predicted ER activity score had longer progression-free survival (p = 0.0368) than those with lower predicted ER activity score. In conclusion, a convolutional deep neural network can predict prognosis and endocrine therapy response in breast cancer patients based on whole-slide H&E-stained images. The trained models were found to robustly predict the prognosis of ER + /HER2- patients. This information is valuable for patient management, as it does not require RNA-seq or microarray data analyses. Thus, these models can reduce the cost of the diagnosis workflow if such information is required.
Collapse
Affiliation(s)
- Chun Wai Ng
- Department of Gynecologic Oncology and Reproductive Medicine, Unit 1362, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | - Kwong-Kwok Wong
- Department of Gynecologic Oncology and Reproductive Medicine, Unit 1362, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA.
| |
Collapse
|
25
|
Bourgade R, Rabilloud N, Perennec T, Pécot T, Garrec C, Guédon AF, Delnatte C, Bézieau S, Lespagnol A, de Tayrac M, Henno S, Sagan C, Toquet C, Mosnier JF, Kammerer-Jacquet SF, Loussouarn D. Deep Learning for Detecting BRCA Mutations in High-Grade Ovarian Cancer Based on an Innovative Tumor Segmentation Method From Whole Slide Images. Mod Pathol 2023; 36:100304. [PMID: 37580018 DOI: 10.1016/j.modpat.2023.100304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 07/15/2023] [Accepted: 08/08/2023] [Indexed: 08/16/2023]
Abstract
BRCA1 and BRCA2 genes play a crucial role in repairing DNA double-strand breaks through homologous recombination. Their mutations represent a significant proportion of homologous recombination deficiency and are a reliable effective predictor of sensitivity of high-grade ovarian cancer (HGOC) to poly(ADP-ribose) polymerase inhibitors. However, their testing by next-generation sequencing is costly and time-consuming and can be affected by various preanalytical factors. In this study, we present a deep learning classifier for BRCA mutational status prediction from hematoxylin-eosin-safran-stained whole slide images (WSI) of HGOC. We constituted the OvarIA cohort composed of 867 patients with HGOC with known BRCA somatic mutational status from 2 different pathology departments. We first developed a tumor segmentation model according to dynamic sampling and then trained a visual representation encoder with momentum contrastive learning on the predicted tumor tiles. We finally trained a BRCA classifier on more than a million tumor tiles in multiple instance learning with an attention-based mechanism. The tumor segmentation model trained on 8 WSI obtained a dice score of 0.915 and an intersection-over-union score of 0.847 on a test set of 50 WSI, while the BRCA classifier achieved the state-of-the-art area under the receiver operating characteristic curve of 0.739 in 5-fold cross-validation and 0.681 on the testing set. An additional multiscale approach indicates that the relevant information for predicting BRCA mutations is located more in the tumor context than in the cell morphology. Our results suggest that BRCA somatic mutations have a discernible phenotypic effect that could be detected by deep learning and could be used as a prescreening tool in the future.
Collapse
Affiliation(s)
- Raphaël Bourgade
- Department of Pathology, University Hospital of Nantes, Nantes, France.
| | - Noémie Rabilloud
- Laboratoire du Traitement du Signal et de l'Image - Inserm U1099, University of Rennes, Rennes, France
| | - Tanguy Perennec
- Department of Radiation Oncology, Institut de Cancérologie de l'Ouest Nantes, Saint-Herblain, France
| | - Thierry Pécot
- Facility for Artificial Intelligence and Image Analysis (FAIIA), Biosit UAR 3480 CNRS-US18 INSERM, University of Rennes, Rennes, France
| | - Céline Garrec
- Department of Medical Genetics, University Hospital of Nantes, Nantes, France
| | - Alexis F Guédon
- National Institute of Health and Medical Research, Pierre Louis Institute of Epidemiology and Public Health, Sorbonne University, Paris, France
| | - Capucine Delnatte
- Department of Medical Genetics, University Hospital of Nantes, Nantes, France
| | - Stéphane Bézieau
- Department of Medical Genetics, University Hospital of Nantes, Nantes, France
| | - Alexandra Lespagnol
- Department of Molecular Genetics and Genomics, University Hospital of Rennes, Rennes, France
| | - Marie de Tayrac
- Department of Molecular Genetics and Genomics, University Hospital of Rennes, Rennes, France
| | - Sébastien Henno
- Department of Pathology, University Hospital of Rennes, Rennes, France
| | - Christine Sagan
- Department of Pathology, University Hospital of Nantes, Nantes, France
| | - Claire Toquet
- Department of Pathology, University Hospital of Nantes, Nantes, France
| | | | - Solène-Florence Kammerer-Jacquet
- Laboratoire du Traitement du Signal et de l'Image - Inserm U1099, University of Rennes, Rennes, France; Department of Pathology, University Hospital of Rennes, Rennes, France
| | | |
Collapse
|
26
|
Ahn JS, Shin S, Yang SA, Park EK, Kim KH, Cho SI, Ock CY, Kim S. Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine. J Breast Cancer 2023; 26:405-435. [PMID: 37926067 PMCID: PMC10625863 DOI: 10.4048/jbc.2023.26.e45] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 09/25/2023] [Accepted: 10/06/2023] [Indexed: 11/07/2023] Open
Abstract
Breast cancer is a significant cause of cancer-related mortality in women worldwide. Early and precise diagnosis is crucial, and clinical outcomes can be markedly enhanced. The rise of artificial intelligence (AI) has ushered in a new era, notably in image analysis, paving the way for major advancements in breast cancer diagnosis and individualized treatment regimens. In the diagnostic workflow for patients with breast cancer, the role of AI encompasses screening, diagnosis, staging, biomarker evaluation, prognostication, and therapeutic response prediction. Although its potential is immense, its complete integration into clinical practice is challenging. Particularly, these challenges include the imperatives for extensive clinical validation, model generalizability, navigating the "black-box" conundrum, and pragmatic considerations of embedding AI into everyday clinical environments. In this review, we comprehensively explored the diverse applications of AI in breast cancer care, underlining its transformative promise and existing impediments. In radiology, we specifically address AI in mammography, tomosynthesis, risk prediction models, and supplementary imaging methods, including magnetic resonance imaging and ultrasound. In pathology, our focus is on AI applications for pathologic diagnosis, evaluation of biomarkers, and predictions related to genetic alterations, treatment response, and prognosis in the context of breast cancer diagnosis and treatment. Our discussion underscores the transformative potential of AI in breast cancer management and emphasizes the importance of focused research to realize the full spectrum of benefits of AI in patient care.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Seokhwi Kim
- Department of Pathology, Ajou University School of Medicine, Suwon, Korea
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Korea.
| |
Collapse
|
27
|
Erak E, Oliveira LD, Mendes AA, Dairo O, Ertunc O, Kulac I, Baena-Del Valle JA, Jones T, Hicks JL, Glavaris S, Guner G, Vidal ID, Markowski M, de la Calle C, Trock BJ, Meena A, Joshi U, Kondragunta C, Bonthu S, Singhal N, De Marzo AM, Lotan TL. Predicting Prostate Cancer Molecular Subtype With Deep Learning on Histopathologic Images. Mod Pathol 2023; 36:100247. [PMID: 37307876 PMCID: PMC11225718 DOI: 10.1016/j.modpat.2023.100247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Accepted: 06/06/2023] [Indexed: 06/14/2023]
Abstract
Microscopic examination of prostate cancer has failed to reveal a reproducible association between molecular and morphologic features. However, deep-learning algorithms trained on hematoxylin and eosin (H&E)-stained whole slide images (WSI) may outperform the human eye and help to screen for clinically-relevant genomic alterations. We created deep-learning algorithms to identify prostate tumors with underlying ETS-related gene (ERG) fusions or PTEN deletions using the following 4 stages: (1) automated tumor identification, (2) feature representation learning, (3) classification, and (4) explainability map generation. A novel transformer-based hierarchical architecture was trained on a single representative WSI of the dominant tumor nodule from a radical prostatectomy (RP) cohort with known ERG/PTEN status (n = 224 and n = 205, respectively). Two distinct vision transformer-based networks were used for feature extraction, and a distinct transformer-based model was used for classification. The ERG algorithm performance was validated across 3 RP cohorts, including 64 WSI from the pretraining cohort (AUC, 0.91) and 248 and 375 WSI from 2 independent RP cohorts (AUC, 0.86 and 0.89, respectively). In addition, we tested the ERG algorithm performance in 2 needle biopsy cohorts comprised of 179 and 148 WSI (AUC, 0.78 and 0.80, respectively). Focusing on cases with homogeneous (clonal) PTEN status, PTEN algorithm performance was assessed using 50 WSI reserved from the pretraining cohort (AUC, 0.81), 201 and 337 WSI from 2 independent RP cohorts (AUC, 0.72 and 0.80, respectively), and 151 WSI from a needle biopsy cohort (AUC, 0.75). For explainability, the PTEN algorithm was also applied to 19 WSI with heterogeneous (subclonal) PTEN loss, where the percentage tumor area with predicted PTEN loss correlated with that based on immunohistochemistry (r = 0.58, P = .0097). These deep-learning algorithms to predict ERG/PTEN status prove that H&E images can be used to screen for underlying genomic alterations in prostate cancer.
Collapse
Affiliation(s)
- Eric Erak
- Department of Pathology, Johns Hopkins University School of Medicine
| | | | - Adrianna A Mendes
- Department of Pathology, Johns Hopkins University School of Medicine
| | | | - Onur Ertunc
- Department of Pathology, Suleyman Demirel University, Turkey
| | | | | | - Tracy Jones
- Department of Pathology, Johns Hopkins University School of Medicine
| | - Jessica L Hicks
- Department of Pathology, Johns Hopkins University School of Medicine
| | | | | | | | - Mark Markowski
- Department of Oncology, Johns Hopkins University School of Medicine
| | | | - Bruce J Trock
- Department of Urology, Johns Hopkins University School of Medicine
| | | | | | | | | | | | - Angelo M De Marzo
- Department of Pathology, Johns Hopkins University School of Medicine; Department of Oncology, Johns Hopkins University School of Medicine; Department of Urology, Johns Hopkins University School of Medicine
| | - Tamara L Lotan
- Department of Pathology, Johns Hopkins University School of Medicine; Department of Oncology, Johns Hopkins University School of Medicine; Department of Urology, Johns Hopkins University School of Medicine.
| |
Collapse
|
28
|
Dent A, Faust K, Lam K, Alhangari N, Leon AJ, Tsang Q, Kamil ZS, Gao A, Pal P, Lheureux S, Oza A, Diamandis P. HAVOC: Small-scale histomic mapping of cancer biodiversity across large tissue distances using deep neural networks. SCIENCE ADVANCES 2023; 9:eadg1894. [PMID: 37774029 PMCID: PMC10541015 DOI: 10.1126/sciadv.adg1894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Accepted: 08/28/2023] [Indexed: 10/01/2023]
Abstract
Intratumoral heterogeneity can wreak havoc on current precision medicine strategies because of challenges in sufficient sampling of geographically separated areas of biodiversity distributed across centimeter-scale tumor distances. To address this gap, we developed a deep learning pipeline that leverages histomorphologic fingerprints of tissue to create "Histomic Atlases of Variation Of Cancers" (HAVOC). Using a number of objective molecular readouts, we demonstrate that HAVOC can define regional cancer boundaries with distinct biology. Using larger tumor specimens, we show that HAVOC can map biodiversity even across multiple tissue sections. By guiding profiling of 19 partitions across six high-grade gliomas, HAVOC revealed that distinct differentiation states can often coexist and be regionally distributed within these tumors. Last, to highlight generalizability, we benchmark HAVOC on additional tumor types. Together, we establish HAVOC as a versatile tool to generate small-scale maps of tissue heterogeneity and guide regional deployment of molecular resources to relevant biodiverse niches.
Collapse
Affiliation(s)
- Anglin Dent
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
| | - Kevin Faust
- Department of Computer Science, University of Toronto, 40 St. George Street, Toronto, ON M5S 2E4, Canada
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - K. H. Brian Lam
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Narges Alhangari
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
| | - Alberto J. Leon
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Queenie Tsang
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Zaid Saeed Kamil
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
| | - Andrew Gao
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
| | - Prodipto Pal
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
| | - Stephanie Lheureux
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Amit Oza
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
| | - Phedias Diamandis
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON M5S 1A8, Canada
- Princess Margaret Cancer Centre, 101 College Street, Toronto, ON M5G 1L7, Canada
- Laboratory Medicine Program, Department of Pathology, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada
- Department of Medical Biophysics, University of Toronto, 101 College St, Toronto, ON M5G 1L7, Canada
| |
Collapse
|
29
|
Hoang DT, Dinstag G, Hermida LC, Ben-Zvi DS, Elis E, Caley K, Sammut SJ, Sinha S, Sinha N, Dampier CH, Stossel C, Patil T, Rajan A, Lassoued W, Strauss J, Bailey S, Allen C, Redman J, Beker T, Jiang P, Golan T, Wilkinson S, Sowalsky AG, Pine SR, Caldas C, Gulley JL, Aldape K, Aharonov R, Stone EA, Ruppin E. Prediction of cancer treatment response from histopathology images through imputed transcriptomics. RESEARCH SQUARE 2023:rs.3.rs-3193270. [PMID: 37790315 PMCID: PMC10543028 DOI: 10.21203/rs.3.rs-3193270/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin (H&E)-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an approach for predicting response to multiple targeted and immunotherapies from H&E-slides. In difference from existing approaches that aim to predict treatment response directly from the slides, ENLIGHT-DeepPT is an indirect two-step approach consisting of (1) DeepPT, a new deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response based on the DeepPT inferred expression values. DeepPT successfully predicts transcriptomics in all 16 TCGA cohorts tested and generalizes well to two independent datasets. Our key contribution is showing that ENLIGHT-DeepPT successfully predicts true responders in five independent patients' cohorts involving four different treatments spanning six cancer types with an overall odds ratio of 2.44, increasing the baseline response rate by 43.47% among predicted responders, without the need for any treatment data for training. Furthermore, its prediction accuracy on these datasets is comparable to a supervised approach predicting the response directly from the images, which needs to be trained and tested on the same cohort. ENLIGHT-DeepPT future application could provide clinicians with rapid treatment recommendations to an array of different therapies and importantly, may contribute to advancing precision oncology in developing countries.
Collapse
Affiliation(s)
- Danh-Tai Hoang
- Biological Data Science Institute, College of Science, Australian National University, Canberra, ACT, Australia
| | | | - Leandro C. Hermida
- Department of Immunology, University of Pittsburgh, Pittsburgh, PA, USA
- Tumor Microenvironment Center, UPMC Hillman Cancer Center, University of Pittsburgh, Pittsburgh, PA, USA
| | | | | | - Katherine Caley
- Biological Data Science Institute, College of Science, Australian National University, Canberra, ACT, Australia
| | - Stephen-John Sammut
- Breast Cancer Now Toby Robins Research Centre, The Institute of Cancer Research, London, United Kingdom
- The Royal Marsden Hospital NHS Foundation Trust, London, United Kingdom
- Cancer Research UK Cambridge Institute, University of Cambridge, Li Ka Shing Centre, Cambridge, UK
| | - Sanju Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Neelam Sinha
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Christopher H. Dampier
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Chani Stossel
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Tejas Patil
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Arun Rajan
- Thoracic and GI Malignancies Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Wiem Lassoued
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Julius Strauss
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Shania Bailey
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Clint Allen
- Surgical Oncology Program, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Jason Redman
- Center for Immuno-Oncology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Peng Jiang
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Talia Golan
- Oncology Institute, Sheba Medical Center at Tel-Hashomer, Tel Aviv University, Tel Aviv, Israel
| | - Scott Wilkinson
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Adam G. Sowalsky
- Laboratory of Genitourinary Cancer Pathogenesis, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Sharon R. Pine
- Division of Medical Oncology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Carlos Caldas
- Cancer Research UK Cambridge Institute, University of Cambridge, Li Ka Shing Centre, Cambridge, UK
| | - James L. Gulley
- Genitourinary Malignancy Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | - Kenneth Aldape
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| | | | - Eric A. Stone
- Biological Data Science Institute, College of Science, Australian National University, Canberra, ACT, Australia
| | - Eytan Ruppin
- Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA
| |
Collapse
|
30
|
Liu Y, Han D, Parwani AV, Li Z. Applications of Artificial Intelligence in Breast Pathology. Arch Pathol Lab Med 2023; 147:1003-1013. [PMID: 36800539 DOI: 10.5858/arpa.2022-0457-ra] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/15/2022] [Indexed: 02/19/2023]
Abstract
CONTEXT.— Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology. OBJECTIVE.— To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes. DATA SOURCES.— We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience. CONCLUSIONS.— With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.
Collapse
Affiliation(s)
- Yueping Liu
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| | - Dandan Han
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| | - Anil V Parwani
- The Department of Pathology, The Ohio State University, Columbus (Parwani, Li)
| | - Zaibo Li
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| |
Collapse
|
31
|
Nuñez NA, Coombes BJ, Melhuish Beaupre L, Romo-Nava F, Gardea-Resendez M, Ozerdem A, Veldic M, Singh B, Sanchez Ruiz JA, Cuellar-Barboza A, Leung JG, Prieto ML, McElroy SL, Biernacka JM, Frye MA. Antidepressant-Associated Treatment Emergent Mania: A Meta-Analysis to Guide Risk Modeling Pharmacogenomic Targets of Potential Clinical Value. J Clin Psychopharmacol 2023; 43:428-433. [PMID: 37683232 PMCID: PMC10476595 DOI: 10.1097/jcp.0000000000001747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 06/09/2023] [Indexed: 08/17/2023]
Abstract
BACKGROUND The purpose of this study was to review the association between the SLC6A4 5-HTTLPR polymorphism and antidepressant (AD)-associated treatment emergent mania (TEM) in bipolar disorder alongside starting a discussion on the merits of developing risk stratification models to guide when not to provide AD treatment for bipolar depression. METHODS Studies that examined the association between clinical and genetic risk factors, specifically monoaminergic transporter genetic variation, and TEM were identified. A meta-analysis was performed using the odds ratio to estimate the effect size under the Der-Simonian and Laird model. RESULTS Seven studies, referencing the SLC6A4 5-HTTLPR polymorphism and TEM (total N = 1578; TEM+ =594, TEM- = 984), of 142 identified articles were included. The time duration between the start of the AD to emergence of TEM ranged from 4 to 12 weeks. There was a nominally significant association between the s allele of the 5-HTTLPR polymorphism and TEM (odds ratio, 1.434; 95% confidence interval, 1.001-2.055; P = 0.0493; I2 = 52%). No studies have investigated norepinephrine or dopamine transporters. CONCLUSION Although the serotonin transporter genetic variation is commercially available in pharmacogenomic decision support tools, greater efforts, more broadly, should focus on complete genome-wide approaches to determine genetic variants that may contribute to TEM. Moreover, these data are exemplary to the merits of developing risk stratification models, which include both clinical and biological risk factors, to guide when not to use ADs in bipolar disorder. Future studies will need to validate new risk models that best inform the development of personalized medicine best practices treating bipolar depression.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Marin Veldic
- From the Departments of Psychiatry and Psychology
| | | | | | | | | | - Miguel L. Prieto
- Department of Psychiatry, Faculty of Medicine, Universidad de Los Andes, Santiago, Chile
| | - Susan L. McElroy
- Department of Psychiatry, Universidad Autónoma de Nuevo León, Monterrey, México
| | - Joanna M. Biernacka
- From the Departments of Psychiatry and Psychology
- Quantitative Health Sciences, Mayo Clinic, Rochester, MN
| | - Mark A. Frye
- From the Departments of Psychiatry and Psychology
| |
Collapse
|
32
|
Franchina M, Pizzimenti C, Fiorentino V, Martini M, Ricciardi GRR, Silvestris N, Ieni A, Tuccari G. Low and Ultra-Low HER2 in Human Breast Cancer: An Effort to Define New Neoplastic Subtypes. Int J Mol Sci 2023; 24:12795. [PMID: 37628975 PMCID: PMC10454084 DOI: 10.3390/ijms241612795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 08/09/2023] [Accepted: 08/13/2023] [Indexed: 08/27/2023] Open
Abstract
HER2-low and ultra-low breast cancer (BC) have been recently proposed as new subcategories of HER2 BC, supporting a re-consideration of immunohistochemical negative scores of 0, 1+ and the 2+/in situ hybridization (ISH) negative phenotype. In the present review, we outline the criteria needed to exactly distinguish HER2-low and ultra-low BC. Recent clinical trials have demonstrated significant clinical benefits of novel HER2 directing antibody-drug conjugates (ADCs) in treating these groups of tumors. In particular, trastuzumab-deruxtecan (T-Dxd), a HER2-directing ADC, has been recently approved by the US Food and Drug Administration as the first targeted therapy to treat HER2-low BC. Furthermore, ongoing trials, such as the DESTINY-Breast06 trial, are currently evaluating ADCs in patients with HER2-ultra low BC. Finally, we hope that new guidelines may help to codify HER2-low and ultra-low BC, increasing our knowledge of tumor biology and improving a targetable new therapeutical treatment.
Collapse
Affiliation(s)
- Mariausilia Franchina
- Department of Human Pathology in Adult and Developmental Age “Gaetano Barresi”, Section of Pathology, University of Messina, 98125 Messina, Italy; (M.F.); (V.F.); (M.M.); (N.S.); (A.I.)
| | - Cristina Pizzimenti
- Department of Biomedical, Dental, Morphological and Functional Imaging Sciences, University of Messina, 98125 Messina, Italy;
| | - Vincenzo Fiorentino
- Department of Human Pathology in Adult and Developmental Age “Gaetano Barresi”, Section of Pathology, University of Messina, 98125 Messina, Italy; (M.F.); (V.F.); (M.M.); (N.S.); (A.I.)
| | - Maurizio Martini
- Department of Human Pathology in Adult and Developmental Age “Gaetano Barresi”, Section of Pathology, University of Messina, 98125 Messina, Italy; (M.F.); (V.F.); (M.M.); (N.S.); (A.I.)
| | | | - Nicola Silvestris
- Department of Human Pathology in Adult and Developmental Age “Gaetano Barresi”, Section of Pathology, University of Messina, 98125 Messina, Italy; (M.F.); (V.F.); (M.M.); (N.S.); (A.I.)
| | - Antonio Ieni
- Department of Human Pathology in Adult and Developmental Age “Gaetano Barresi”, Section of Pathology, University of Messina, 98125 Messina, Italy; (M.F.); (V.F.); (M.M.); (N.S.); (A.I.)
| | - Giovanni Tuccari
- Department of Human Pathology in Adult and Developmental Age “Gaetano Barresi”, Section of Pathology, University of Messina, 98125 Messina, Italy; (M.F.); (V.F.); (M.M.); (N.S.); (A.I.)
| |
Collapse
|
33
|
Shirman Y, Lubovsky S, Shai A. HER2-Low Breast Cancer: Current Landscape and Future Prospects. BREAST CANCER (DOVE MEDICAL PRESS) 2023; 15:605-616. [PMID: 37600670 PMCID: PMC10439285 DOI: 10.2147/bctt.s366122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Accepted: 08/09/2023] [Indexed: 08/22/2023]
Abstract
More than 50% of breast cancers are currently defined as "Human epidermal growth factor receptor 2 (HER2) low breast cancer (BC)", with HER2 immunohistochemistry (IHC) scores of +1 or +2 with a negative fluorescence in situ hybridization (FISH) test. In most studies that compared the clinical and biological characteristics of HER2-low BC with HER2-negative BC, HER2-low was not associated with unique clinical and molecular characteristics, and it seems that the importance of HER2 in these tumors is being a docking site for the antibody portion of antibody drug conjugates (ADCs). Current pathological methods may underestimate the proportion of BCs that express low levels of HER2 due to analytical limitations and tumor heterogeneity. In this review we summarize and contextualize the most recent literature on HER2-low breast cancers, including clinical and translational studies We also review the challenges of assessing low HER2 expression in BC and discuss the current and future therapeutic landscape for these tumors.
Collapse
Affiliation(s)
- Yelena Shirman
- Division of Oncology, Rambam Health Care Campus, Haifa, Israel
| | | | - Ayelet Shai
- Division of Oncology, Rambam Health Care Campus, Haifa, Israel
| |
Collapse
|
34
|
Li B, Yang L, Zhang H, Li H, Jiang C, Yao Y, Cheng S, Zou B, Fan B, Dong T, Wang L. Outcome-Supervised Deep Learning on Pathologic Whole Slide Images for Survival Prediction of Immunotherapy in Patients with Non-Small Cell Lung Cancer. Mod Pathol 2023; 36:100208. [PMID: 37149222 DOI: 10.1016/j.modpat.2023.100208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 04/19/2023] [Accepted: 04/21/2023] [Indexed: 05/08/2023]
Abstract
Although programmed death-(ligand) 1 (PD-(L)1) inhibitors are marked by durable efficacy in patients with non-small cell lung cancer (NSCLC), approximately 60% of the patients still suffer from recurrence and metastasis after PD-(L)1 inhibitor treatment. To accurately predict the response to PD-(L)1 inhibitors, we presented a deep learning model using a Vision Transformer (ViT) network based on hematoxylin and eosin (H&E)-stained specimens of patients with NSCLC. Two independent cohorts of patients with NSCLC receiving PD-(L)1 inhibitors from Shandong Cancer Hospital and Institute and Shandong Provincial Hospital were enrolled for model training and external validation, respectively. Whole slide images (WSIs) of H&E-stained histologic specimens were obtained from these patients and patched into 1024 × 1024 pixels. The patch-level model was trained based on ViT to identify the predictive patches, and patch-level probability distribution was performed. Then, we trained a patient-level survival model based on the ViT-Recursive Neural Network framework and externally validated it in the Shandong Provincial Hospital cohort. A total of 291 WSIs of H&E-stained histologic specimens from 198 patients with NSCLC in Shandong Cancer Hospital and 62 WSIs from 30 patients with NSCLC in Shandong Provincial Hospital were included in the model training and validation. The model achieved an accuracy of 88.6% in the internal validation cohort and 81% in the external validation cohort. The survival model also remained a statistically independent predictor of survival from PD-(L)1 inhibitors. In conclusion, the outcome-supervised ViT-Recursive Neural Network survival model based on pathologic WSIs could be used to predict immunotherapy efficacy in patients with NSCLC.
Collapse
Affiliation(s)
- Butuo Li
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Linlin Yang
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Huan Zhang
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Haoqian Li
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Chao Jiang
- Department of Otorhinolaryngology Head and Neck Surgery, Shandong Provincial Hospital Affiliated to Shandong First Medical University; Shandong Provincial Hospital, Cheeloo College of Medicine, Shandong University, Jinan, Shandong, China
| | - Yueyuan Yao
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Shuping Cheng
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Bing Zou
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Bingjie Fan
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Taotao Dong
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan, Shandong, China.
| | - Linlin Wang
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China.
| |
Collapse
|
35
|
Zhou J, Foroughi Pour A, Deirawan H, Daaboul F, Aung TN, Beydoun R, Ahmed FS, Chuang JH. Integrative deep learning analysis improves colon adenocarcinoma patient stratification at risk for mortality. EBioMedicine 2023; 94:104726. [PMID: 37499603 PMCID: PMC10388166 DOI: 10.1016/j.ebiom.2023.104726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 06/19/2023] [Accepted: 07/10/2023] [Indexed: 07/29/2023] Open
Abstract
BACKGROUND Colorectal cancers are the fourth most diagnosed cancer and the second leading cancer in number of deaths. Many clinical variables, pathological features, and genomic signatures are associated with patient risk, but reliable patient stratification in the clinic remains a challenging task. Here we assess how image, clinical, and genomic features can be combined to predict risk. METHODS We developed and evaluated integrative deep learning models combining formalin-fixed, paraffin-embedded (FFPE) whole slide images (WSIs), clinical variables, and mutation signatures to stratify colon adenocarcinoma (COAD) patients based on their risk of mortality. Our models were trained using a dataset of 108 patients from The Cancer Genome Atlas (TCGA), and were externally validated on newly generated dataset from Wayne State University (WSU) of 123 COAD patients and rectal adenocarcinoma (READ) patients in TCGA (N = 52). FINDINGS We first observe that deep learning models trained on FFPE WSIs of TCGA-COAD separate high-risk (OS < 3 years, N = 38) and low-risk (OS > 5 years, N = 25) patients (AUC = 0.81 ± 0.08, 5 year survival p < 0.0001, 5 year relative risk = 1.83 ± 0.04) though such models are less effective at predicting overall survival (OS) for moderate-risk (3 years < OS < 5 years, N = 45) patients (5 year survival p-value = 0.5, 5 year relative risk = 1.05 ± 0.09). We find that our integrative models combining WSIs, clinical variables, and mutation signatures can improve patient stratification for moderate-risk patients (5 year survival p < 0.0001, 5 year relative risk = 1.87 ± 0.07). Our integrative model combining image and clinical variables is also effective on an independent pathology dataset (WSU-COAD, N = 123) generated by our team (5 year survival p < 0.0001, 5 year relative risk = 1.52 ± 0.08), and the TCGA-READ data (5 year survival p < 0.0001, 5 year relative risk = 1.18 ± 0.17). Our multicenter integrative image and clinical model trained on combined TCGA-COAD and WSU-COAD is effective in predicting risk on TCGA-READ (5 year survival p < 0.0001, 5 year relative risk = 1.82 ± 0.13) data. Pathologist review of image-based heatmaps suggests that nuclear size pleomorphism, intense cellularity, and abnormal structures are associated with high-risk, while low-risk regions have more regular and small cells. Quantitative analysis shows high cellularity, high ratios of tumor cells, large tumor nuclei, and low immune infiltration are indicators of high-risk tiles. INTERPRETATION The improved stratification of colorectal cancer patients from our computational methods can be beneficial for treatment plans and enrollment of patients in clinical trials. FUNDING This study was supported by the National Cancer Institutes (Grant No. R01CA230031 and P30CA034196). The funders had no roles in study design, data collection and analysis or preparation of the manuscript.
Collapse
Affiliation(s)
- Jie Zhou
- The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA; Department of Genetics and Genome Sciences, UCONN Health, Farmington, CT, USA
| | | | - Hany Deirawan
- Department of Pathology, Wayne State University, Detroit, MI, USA; Department of Dermatology, Wayne State University, Detroit, MI, USA
| | - Fayez Daaboul
- Department of Pathology, Wayne State University, Detroit, MI, USA
| | - Thazin Nwe Aung
- Department of Pathology, Yale University, New Haven, CT, USA
| | - Rafic Beydoun
- Department of Pathology, Wayne State University, Detroit, MI, USA
| | | | - Jeffrey H Chuang
- The Jackson Laboratory for Genomic Medicine, Farmington, CT, USA; Department of Genetics and Genome Sciences, UCONN Health, Farmington, CT, USA.
| |
Collapse
|
36
|
Wu Y, Li Y, Xiong X, Liu X, Lin B, Xu B. Recent advances of pathomics in colorectal cancer diagnosis and prognosis. Front Oncol 2023; 13:1094869. [PMID: 37538112 PMCID: PMC10396402 DOI: 10.3389/fonc.2023.1094869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 06/13/2023] [Indexed: 08/05/2023] Open
Abstract
Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.
Collapse
Affiliation(s)
- Yihan Wu
- School of Medicine, Chongqing University, Chongqing, China
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
| | - Yi Li
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
- Bioengineering College, Chongqing University, Chongqing, China
| | - Xiaomin Xiong
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
- Bioengineering College, Chongqing University, Chongqing, China
| | - Xiaohua Liu
- Bioengineering College, Chongqing University, Chongqing, China
| | - Bo Lin
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
| | - Bo Xu
- School of Medicine, Chongqing University, Chongqing, China
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
| |
Collapse
|
37
|
Choi S, Kim S. Artificial Intelligence in the Pathology of Gastric Cancer. J Gastric Cancer 2023; 23:410-427. [PMID: 37553129 PMCID: PMC10412971 DOI: 10.5230/jgc.2023.23.e25] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 07/09/2023] [Accepted: 07/14/2023] [Indexed: 08/10/2023] Open
Abstract
Recent advances in artificial intelligence (AI) have provided novel tools for rapid and precise pathologic diagnosis. The introduction of digital pathology has enabled the acquisition of scanned slide images that are essential for the application of AI. The application of AI for improved pathologic diagnosis includes the error-free detection of potentially negligible lesions, such as a minute focus of metastatic tumor cells in lymph nodes, the accurate diagnosis of potentially controversial histologic findings, such as very well-differentiated carcinomas mimicking normal epithelial tissues, and the pathological subtyping of the cancers. Additionally, the utilization of AI algorithms enables the precise decision of the score of immunohistochemical markers for targeted therapies, such as human epidermal growth factor receptor 2 and programmed death-ligand 1. Studies have revealed that AI assistance can reduce the discordance of interpretation between pathologists and more accurately predict clinical outcomes. Several approaches have been employed to develop novel biomarkers from histologic images using AI. Moreover, AI-assisted analysis of the cancer microenvironment showed that the distribution of tumor-infiltrating lymphocytes was related to the response to the immune checkpoint inhibitor therapy, emphasizing its value as a biomarker. As numerous studies have demonstrated the significance of AI-assisted interpretation and biomarker development, the AI-based approach will advance diagnostic pathology.
Collapse
Affiliation(s)
- Sangjoon Choi
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Seokhwi Kim
- Department of Pathology, Ajou University School of Medicine, Suwon, Korea
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Korea.
| |
Collapse
|
38
|
To T, Lu T, Jorns JM, Patton M, Schmidt TG, Yen T, Yu B, Ye DH. Deep learning classification of deep ultraviolet fluorescence images toward intra-operative margin assessment in breast cancer. Front Oncol 2023; 13:1179025. [PMID: 37397361 PMCID: PMC10313133 DOI: 10.3389/fonc.2023.1179025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 05/22/2023] [Indexed: 07/04/2023] Open
Abstract
Background Breast-conserving surgery is aimed at removing all cancerous cells while minimizing the loss of healthy tissue. To ensure a balance between complete resection of cancer and preservation of healthy tissue, it is necessary to assess themargins of the removed specimen during the operation. Deep ultraviolet (DUV) fluorescence scanning microscopy provides rapid whole-surface imaging (WSI) of resected tissues with significant contrast between malignant and normal/benign tissue. Intra-operative margin assessment with DUV images would benefit from an automated breast cancer classification method. Methods Deep learning has shown promising results in breast cancer classification, but the limited DUV image dataset presents the challenge of overfitting to train a robust network. To overcome this challenge, the DUV-WSI images are split into small patches, and features are extracted using a pre-trained convolutional neural network-afterward, a gradient-boosting tree trains on these features for patch-level classification. An ensemble learning approach merges patch-level classification results and regional importance to determine the margin status. An explainable artificial intelligence method calculates the regional importance values. Results The proposed method's ability to determine the DUV WSI was high with 95% accuracy. The 100% sensitivity shows that the method can detect malignant cases efficiently. The method could also accurately localize areas that contain malignant or normal/benign tissue. Conclusion The proposed method outperforms the standard deep learning classification methods on the DUV breast surgical samples. The results suggest that it can be used to improve classification performance and identify cancerous regions more effectively.
Collapse
Affiliation(s)
- Tyrell To
- Department of Electrical and Computer Engineering, Marquette University, Opus College of Engineering, Milwaukee, WI, United States
| | - Tongtong Lu
- Joint Department of Biomedical Engineering, Marquette University and Medical College of Wisconsin, Milwaukee, WI, United States
| | - Julie M. Jorns
- Department of Pathology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - Mollie Patton
- Department of Pathology, Medical College of Wisconsin, Milwaukee, WI, United States
| | - Taly Gilat Schmidt
- Joint Department of Biomedical Engineering, Marquette University and Medical College of Wisconsin, Milwaukee, WI, United States
| | - Tina Yen
- Department of Surgery, Medical College of Wisconsin, Milwaukee, WI, United States
| | - Bing Yu
- Joint Department of Biomedical Engineering, Marquette University and Medical College of Wisconsin, Milwaukee, WI, United States
| | - Dong Hye Ye
- Department of Computer Science, Georgia State University, Atlanta, GA, United States
| |
Collapse
|
39
|
Hou Y, Nitta H, Li Z. HER2 Intratumoral Heterogeneity in Breast Cancer, an Evolving Concept. Cancers (Basel) 2023; 15:2664. [PMID: 37345001 DOI: 10.3390/cancers15102664] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 04/27/2023] [Accepted: 05/05/2023] [Indexed: 06/23/2023] Open
Abstract
Amplification and/or overexpression of human epidermal growth factor receptor 2 (HER2) in breast cancer is associated with an adverse prognosis. The introduction of anti-HER2 targeted therapy has dramatically improved the clinical outcomes of patients with HER2-positive breast cancer. Unfortunately, a significant number of patients eventually relapse and develop distant metastasis. HER2 intratumoral heterogeneity (ITH) has been reported to be associated with poor prognosis in patients with anti-HER2 targeted therapies and was proposed to be a potential mechanism for anti-HER2 resistance. In this review, we described the current definition, common types of HER2 ITH in breast cancer, the challenge in interpretation of HER2 status in cases showing ITH and the clinical applications of anti-HER2 agents in breast cancer showing heterogeneous HER2 expression. Digital image analysis has emerged as an objective and reproducible scoring method and its role in the assessment of HER2 status with ITH remains to be demonstrated.
Collapse
Affiliation(s)
- Yanjun Hou
- Department of Pathology and Laboratory Medicine, Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, NC 28659, USA
| | | | - Zaibo Li
- Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH 43210, USA
| |
Collapse
|
40
|
Mandair D, Reis-Filho JS, Ashworth A. Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology. NPJ Breast Cancer 2023; 9:21. [PMID: 37024522 PMCID: PMC10079681 DOI: 10.1038/s41523-023-00518-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Accepted: 02/27/2023] [Indexed: 04/08/2023] Open
Abstract
Breast cancer remains a highly prevalent disease with considerable inter- and intra-tumoral heterogeneity complicating prognostication and treatment decisions. The utilization and depth of genomic, transcriptomic and proteomic data for cancer has exploded over recent times and the addition of spatial context to this information, by understanding the correlating morphologic and spatial patterns of cells in tissue samples, has created an exciting frontier of research, histo-genomics. At the same time, deep learning (DL), a class of machine learning algorithms employing artificial neural networks, has rapidly progressed in the last decade with a confluence of technical developments - including the advent of modern graphic processing units (GPU), allowing efficient implementation of increasingly complex architectures at scale; advances in the theoretical and practical design of network architectures; and access to larger datasets for training - all leading to sweeping advances in image classification and object detection. In this review, we examine recent developments in the application of DL in breast cancer histology with particular emphasis of those producing biologic insights or novel biomarkers, spanning the extraction of genomic information to the use of stroma to predict cancer recurrence, with the aim of suggesting avenues for further advancing this exciting field.
Collapse
Affiliation(s)
- Divneet Mandair
- UCSF Helen Diller Family Comprehensive Cancer Center, San Francisco, CA, 94158, USA
| | | | - Alan Ashworth
- UCSF Helen Diller Family Comprehensive Cancer Center, San Francisco, CA, 94158, USA.
| |
Collapse
|
41
|
Lan J, Chen M, Wang J, Du M, Wu Z, Zhang H, Xue Y, Wang T, Chen L, Xu C, Han Z, Hu Z, Zhou Y, Zhou X, Tong T, Chen G. Using less annotation workload to establish a pathological auxiliary diagnosis system for gastric cancer. Cell Rep Med 2023; 4:101004. [PMID: 37044091 PMCID: PMC10140598 DOI: 10.1016/j.xcrm.2023.101004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 10/20/2022] [Accepted: 03/17/2023] [Indexed: 04/14/2023]
Abstract
Pathological diagnosis of gastric cancer requires pathologists to have extensive clinical experience. To help pathologists improve diagnostic accuracy and efficiency, we collected 1,514 cases of stomach H&E-stained specimens with complete diagnostic information to establish a pathological auxiliary diagnosis system based on deep learning. At the slide level, our system achieves a specificity of 0.8878 while maintaining a high sensitivity close to 1.0 on 269 biopsy specimens (147 malignancies) and 163 surgical specimens (80 malignancies). The classified accuracy of our system is 0.9034 at the slide level for 352 biopsy specimens (201 malignancies) from 50 medical centers. With the help of our system, the pathologists' average false-negative rate and average false-positive rate on 100 biopsy specimens (50 malignancies) are reduced to 1/5 and 1/2 of the original rates, respectively. At the same time, the average uncertainty rate and the average diagnosis time are reduced by approximately 22% and 20%, respectively.
Collapse
Affiliation(s)
- Junlin Lan
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Musheng Chen
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Jianchao Wang
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Min Du
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Zhida Wu
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Hejun Zhang
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Yuyang Xue
- School of Engineering, University of Edinburgh, Edinburgh EH8 9JU, UK
| | - Tao Wang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Lifan Chen
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Chaohui Xu
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Zixin Han
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Ziwei Hu
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Yuanbo Zhou
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Xiaogen Zhou
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Tong Tong
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China; Imperial Vision Technology, Fuzhou, Fujian 350100, China.
| | - Gang Chen
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China.
| |
Collapse
|
42
|
Wu S, Yue M, Zhang J, Li X, Li Z, Zhang H, Wang X, Han X, Cai L, Shang J, Jia Z, Wang X, Li J, Liu Y. The Role of Artificial Intelligence in Accurate Interpretation of HER2 Immunohistochemical Scores 0 and 1+ in Breast Cancer. Mod Pathol 2023; 36:100054. [PMID: 36788100 DOI: 10.1016/j.modpat.2022.100054] [Citation(s) in RCA: 23] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 10/28/2022] [Accepted: 11/20/2022] [Indexed: 01/11/2023]
Abstract
The new human epidermal growth factor receptor (HER)2-targeting antibody-drug conjugate offers the opportunity to treat patients with HER2-low breast cancer. Distinguishing HER2 immunohistochemical (IHC) scores of 0 and 1+ is not only critical but also challenging owing to HER2 heterogeneity and variability of observers. In this study, we aimed to increase the interpretation accuracy and consistency of HER2 IHC 0 and 1+ evaluation through assistance from an artificial intelligence (AI) algorithm. In addition, we examined the value of our AI algorithm in evaluating HER2 IHC scores in tumors with heterogeneity. AI-assisted interpretation consisted of AI algorithms and an augmenting reality module with a microscope. Fifteen pathologists (5 junior, 5 midlevel, and 5 senior) participated in this multi-institutional 2-round ring study that included 246 infiltrating duct carcinoma cases that were not otherwise specified. In round 1, pathologists analyzed 246 HER2 IHC slides by microscope without AI assistance. After a 2-week washout period, the pathologists read the same slides with AI algorithm assistance and rendered the definitive results by adjusting to the AI algorithm. The accuracy of interpretation accuracy with AI assistance (0.93 vs 0.80), thereby the evaluation precision of HER2 0 and the recall of HER2 1+. In addition, the AI algorithm improved the total consistency (intraclass correlation coefficient = 0.542-0.812), especially in HER2 1+ cases. In cases with heterogeneity, accuracy improved significantly (0.68 to 0.89) and to a similar level as in cases without heterogeneity (accuracy, 0.97). Both accuracy and consistency improved more for junior pathologists than those for the midlevel and senior pathologists. To the best of our knowledge, this is the first study to show that the accuracy and consistency of HER2 IHC 0 and 1+ evaluation and the accuracy of HER2 IHC evaluation in breast cancers with heterogeneity can be significantly improved using AI-assisted interpretation.
Collapse
Affiliation(s)
- Si Wu
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Meng Yue
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Jun Zhang
- Tencent AI Lab, Nanshan District, Tencent Binhai Building, Shenzhen, Guangdong, China
| | - Xiaoxian Li
- Department of Pathology and Laboratory Medicine, The Emory University School of Medicine, Atlanta, Georgia
| | - Zaibo Li
- Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Huina Zhang
- Department of Pathology, University of Rochester Medical Center, Rochester, New York
| | - Xinran Wang
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Xiao Han
- Tencent AI Lab, Nanshan District, Tencent Binhai Building, Shenzhen, Guangdong, China
| | - Lijing Cai
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Jiuyan Shang
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Zhanli Jia
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Xiaoxiao Wang
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Jinze Li
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Yueping Liu
- Department of Pathology, the Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China.
| |
Collapse
|
43
|
Wen Z, Wang S, Yang DM, Xie Y, Chen M, Bishop J, Xiao G. Deep learning in digital pathology for personalized treatment plans of cancer patients. Semin Diagn Pathol 2023; 40:109-119. [PMID: 36890029 DOI: 10.1053/j.semdp.2023.02.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 02/22/2023] [Indexed: 02/27/2023]
Abstract
Over the past decade, many new cancer treatments have been developed and made available to patients. However, in most cases, these treatments only benefit a specific subgroup of patients, making the selection of treatment for a specific patient an essential but challenging task for oncologists. Although some biomarkers were found to associate with treatment response, manual assessment is time-consuming and subjective. With the rapid developments and expanded implementation of artificial intelligence (AI) in digital pathology, many biomarkers can be quantified automatically from histopathology images. This approach allows for a more efficient and objective assessment of biomarkers, aiding oncologists in formulating personalized treatment plans for cancer patients. This review presents an overview and summary of the recent studies on biomarker quantification and treatment response prediction using hematoxylin-eosin (H&E) stained pathology images. These studies have shown that an AI-based digital pathology approach can be practical and will become increasingly important in improving the selection of cancer treatments for patients.
Collapse
Affiliation(s)
- Zhuoyu Wen
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Shidan Wang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Donghan M Yang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Yang Xie
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA; Simmons Comprehensive Cancer Center, UT Southwestern Medical Center, Dallas, TX, USA; Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Mingyi Chen
- Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX, USA
| | - Justin Bishop
- Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX, USA
| | - Guanghua Xiao
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA; Simmons Comprehensive Cancer Center, UT Southwestern Medical Center, Dallas, TX, USA; Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX, USA.
| |
Collapse
|
44
|
Predicting gene mutation status via artificial intelligence technologies based on multimodal integration (MMI) to advance precision oncology. Semin Cancer Biol 2023; 91:1-15. [PMID: 36801447 DOI: 10.1016/j.semcancer.2023.02.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 01/30/2023] [Accepted: 02/15/2023] [Indexed: 02/21/2023]
Abstract
Personalized treatment strategies for cancer frequently rely on the detection of genetic alterations which are determined by molecular biology assays. Historically, these processes typically required single-gene sequencing, next-generation sequencing, or visual inspection of histopathology slides by experienced pathologists in a clinical context. In the past decade, advances in artificial intelligence (AI) technologies have demonstrated remarkable potential in assisting physicians with accurate diagnosis of oncology image-recognition tasks. Meanwhile, AI techniques make it possible to integrate multimodal data such as radiology, histology, and genomics, providing critical guidance for the stratification of patients in the context of precision therapy. Given that the mutation detection is unaffordable and time-consuming for a considerable number of patients, predicting gene mutations based on routine clinical radiological scans or whole-slide images of tissue with AI-based methods has become a hot issue in actual clinical practice. In this review, we synthesized the general framework of multimodal integration (MMI) for molecular intelligent diagnostics beyond standard techniques. Then we summarized the emerging applications of AI in the prediction of mutational and molecular profiles of common cancers (lung, brain, breast, and other tumor types) pertaining to radiology and histology imaging. Furthermore, we concluded that there truly exist multiple challenges of AI techniques in the way of its real-world application in the medical field, including data curation, feature fusion, model interpretability, and practice regulations. Despite these challenges, we still prospect the clinical implementation of AI as a highly potential decision-support tool to aid oncologists in future cancer treatment management.
Collapse
|
45
|
Ackermann BL, Morrison RD, Hill S, Westfall MD, Butts BD, Soper MD, Fill JA, Schade AE, Liebler DC, Gruver AM. Targeted Quantitative Mass Spectrometry Analysis of Protein Biomarkers From Previously Stained Single Formalin-Fixed Paraffin-Embedded Tissue Sections. J Transl Med 2023; 103:100052. [PMID: 36870295 DOI: 10.1016/j.labinv.2022.100052] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 11/30/2022] [Accepted: 12/15/2022] [Indexed: 01/11/2023] Open
Abstract
Formalin-fixed, paraffin-embedded tissues represent a majority of all biopsy specimens commonly analyzed by histologic or immunohistochemical staining with adhesive coverslips attached. Mass spectrometry (MS) has recently been used to precisely quantify proteins in samples consisting of multiple unstained formalin-fixed, paraffin-embedded sections. Here, we report an MS method to analyze proteins from a single coverslipped 4-μm section previously stained with hematoxylin and eosin, Masson trichrome, or 3,3'-diaminobenzidine-based immunohistochemical staining. We analyzed serial unstained and stained sections from non-small cell lung cancer specimens for proteins of varying abundance (PD-L1, RB1, CD73, and HLA-DRA). Coverslips were removed by soaking in xylene, and after tryptic digestion, peptides were analyzed by targeted high-resolution liquid chromatography with tandem MS with stable isotope-labeled peptide standards. The low-abundance proteins RB1 and PD-L1 were quantified in 31 and 35 of 50 total sections analyzed, respectively, whereas higher abundance CD73 and HLA-DRA were quantified in 49 and 50 sections, respectively. The inclusion of targeted β-actin measurement enabled normalization in samples where residual stain interfered with bulk protein quantitation by colorimetric assay. Measurement coefficient of variations for 5 replicate slides (hematoxylin and eosin stained vs unstained) from each block ranged from 3% to 18% for PD-L1, from 1% to 36% for RB1, 3% to 21% for CD73, and 4% to 29% for HLA-DRA. Collectively, these results demonstrate that targeted MS protein quantification can add a valuable data layer to clinical tissue specimens after assessment for standard pathology end points.
Collapse
Affiliation(s)
| | | | | | | | - Brent D Butts
- Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana
| | - Michael D Soper
- Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana
| | - Jeff A Fill
- Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana
| | - Andrew E Schade
- Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana
| | | | - Aaron M Gruver
- Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana.
| |
Collapse
|
46
|
da Silva JL, Carvalho GDS, Zanetti de Albuquerque L, Rodrigues FR, Fernandes PV, Kischinhevsky D, de Melo AC. Exploring Real-World HER2-Low Data in Early-Stage Triple-Negative Breast Cancer: Insights and Implications. BREAST CANCER (DOVE MEDICAL PRESS) 2023; 15:337-347. [PMID: 37188066 PMCID: PMC10178312 DOI: 10.2147/bctt.s408743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/19/2023] [Accepted: 04/30/2023] [Indexed: 05/17/2023]
Abstract
Purpose This study aimed to compare the clinical behavior, clinicopathological and sociodemographic characteristics of patients with early-stage triple-negative breast cancer (TNBC) who belong to the HER2-low and HER2-zero subgroups. Patients and Methods This study involved a thorough search in the internal database of a single Brazilian institution to identify women with TNBC who underwent neoadjuvant chemotherapy (NACT) followed by curative surgery within the period from January 2010 to December 2014. HER2 analysis through immunohistochemistry (IHC) and, if required, amplification by in situ hybridization, was conducted using core biopsy samples. The study assesses outcomes of residual cancer burden (RCB), event-free survival (EFS), and overall survival (OS). Results A total of 170 cases were analyzed, with a mean age of 51.4 years (standard deviation, SD 11.2). The HER2 status was categorized as IHC 0, 1+, or 2+ in 80 (47.1%), 73 (42.9%), and 17 (10%) patients, respectively. No significant differences were observed in the prevalence of clinical pathological characteristics among the subgroups. The absence of significant results for clinicopathological and demographic features hindered the multivariate analysis of HER2 subgroups. Similarly, no significant differences were found in the RCB, EFS, and OS outcomes between HER2 subgroups. Conclusion The findings of this study suggest that, in early-stage TNBC, the clinical behavior and survival outcomes of the HER2-low subgroup may not differ significantly from those of the HER2-zero subgroup.
Collapse
Affiliation(s)
- Jesse Lopes da Silva
- Division of Clinical Research and Technological Development, Brazilian National Cancer Institute, Rio de Janeiro, Brazil
- Correspondence: Jesse Lopes da Silva, Brazilian National Cancer Institute (INCA), Clinical Research Division, 37 André Cavalcanti Street, 5th Floor, Annex Building, Rio de Janeiro, 20231-050, Brazil, Tel/Fax +55 21 32076585, Email
| | - Giselle de Souza Carvalho
- Division of Clinical Research and Technological Development, Brazilian National Cancer Institute, Rio de Janeiro, Brazil
| | - Lucas Zanetti de Albuquerque
- Division of Clinical Research and Technological Development, Brazilian National Cancer Institute, Rio de Janeiro, Brazil
| | | | | | - Daniel Kischinhevsky
- Division of Clinical Research and Technological Development, Brazilian National Cancer Institute, Rio de Janeiro, Brazil
| | - Andreia Cristina de Melo
- Division of Clinical Research and Technological Development, Brazilian National Cancer Institute, Rio de Janeiro, Brazil
| |
Collapse
|
47
|
Zhang H, Peng Y. Current Biological, Pathological and Clinical Landscape of HER2-Low Breast Cancer. Cancers (Basel) 2022; 15:126. [PMID: 36612123 PMCID: PMC9817919 DOI: 10.3390/cancers15010126] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 12/21/2022] [Accepted: 12/21/2022] [Indexed: 12/28/2022] Open
Abstract
HER2-low breast cancer (BC) is a newly defined subset of HER2-negative BC that has HER2 immunohistochemical (IHC) score of 1+ or score of 2+/in situ hybridization (ISH) negative phenotype. Recent clinical trials have demonstrated significant clinical benefits of novel HER2 directing antibody-drug conjugates (ADCs) in treating this group of tumors. Trastuzumab-deruxtecan (T-Dxd), a HER2-directing ADC was recently approved by the U.S. Food and Drug Administration as the first targeted therapy to treat HER2-low BC. However, HER2-low BC is still not well characterized clinically and pathologically. This review aims to update the current biological, pathological and clinical landscape of HER2-low BC based on the English literature published in the past two years and to propose the future directions on clinical management, pathology practice, and translational research in this subset of BC. We hope it would help better understand the tumor biology of HER2-low BC and the current efforts for identifying and treating this newly recognized targetable group of BC.
Collapse
Affiliation(s)
- Huina Zhang
- Department of Pathology, University of Rochester Medical Center, Rochester, NY 14642, USA
| | - Yan Peng
- Department of Pathology and Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| |
Collapse
|
48
|
Deep Learning Based Target Tracking Algorithm Model for Athlete Training Trajectory. Processes (Basel) 2022. [DOI: 10.3390/pr10122710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
The main function of the athlete tracking system is to collect the real-time competition data of the athletes. Deep learning is a research hotspot in the field of image and video. With the rapid development of science and technology, it has not only made a breakthrough in theory, but also achieved excellent results in practical application. SiamRPN (Siamese Region Proposal Network) is a single target tracking network model based on deep learning, which has high accuracy and fast operation speed. However, in long-term tracking, if the target is completely obscured and out of the sight of SiamRPN, the tracking of the network will be invalid. Considering the difficulty of long-term tracking, the algorithm is improved and tested by introducing channel attention mechanism and local global search strategy into SiamRPN. Experimental results show that this algorithm has higher accuracy and prediction average overlap rate than the original SiamRPN algorithm when performing tracking tasks on long-term tracking sequences. At the same time, the improved algorithm can still achieve good results in the case of target disappearance and other challenging factors. This study provides an important reference for the coaches of deep learning to realize long-term tracking of athletes.
Collapse
|
49
|
Ho DJ, Chui MH, Vanderbilt CM, Jung J, Robson ME, Park CS, Roh J, Fuchs TJ. Deep Interactive Learning-based ovarian cancer segmentation of H&E-stained whole slide images to study morphological patterns of BRCA mutation. J Pathol Inform 2022; 14:100160. [PMID: 36536772 PMCID: PMC9758515 DOI: 10.1016/j.jpi.2022.100160] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 11/22/2022] [Accepted: 11/22/2022] [Indexed: 11/27/2022] Open
Abstract
Deep learning has been widely used to analyze digitized hematoxylin and eosin (H&E)-stained histopathology whole slide images. Automated cancer segmentation using deep learning can be used to diagnose malignancy and to find novel morphological patterns to predict molecular subtypes. To train pixel-wise cancer segmentation models, manual annotation from pathologists is generally a bottleneck due to its time-consuming nature. In this paper, we propose Deep Interactive Learning with a pretrained segmentation model from a different cancer type to reduce manual annotation time. Instead of annotating all pixels from cancer and non-cancer regions on giga-pixel whole slide images, an iterative process of annotating mislabeled regions from a segmentation model and training/finetuning the model with the additional annotation can reduce the time. Especially, employing a pretrained segmentation model can further reduce the time than starting annotation from scratch. We trained an accurate ovarian cancer segmentation model with a pretrained breast segmentation model by 3.5 hours of manual annotation which achieved intersection-over-union of 0.74, recall of 0.86, and precision of 0.84. With automatically extracted high-grade serous ovarian cancer patches, we attempted to train an additional classification deep learning model to predict BRCA mutation. The segmentation model and code have been released at https://github.com/MSKCC-Computational-Pathology/DMMN-ovary.
Collapse
Affiliation(s)
- David Joon Ho
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - M. Herman Chui
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Chad M. Vanderbilt
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Jiwon Jung
- Department of Pathology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Mark E. Robson
- Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Chan-Sik Park
- Department of Pathology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Jin Roh
- Department of Pathology, Ajou University School of Medicine, Suwon, Republic of Korea
| | - Thomas J. Fuchs
- Hasso Plattner Institute for Digital Health, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
50
|
Shmatko A, Ghaffari Laleh N, Gerstung M, Kather JN. Artificial intelligence in histopathology: enhancing cancer research and clinical oncology. NATURE CANCER 2022; 3:1026-1038. [PMID: 36138135 DOI: 10.1038/s43018-022-00436-4] [Citation(s) in RCA: 134] [Impact Index Per Article: 67.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Accepted: 08/03/2022] [Indexed: 06/16/2023]
Abstract
Artificial intelligence (AI) methods have multiplied our capabilities to extract quantitative information from digital histopathology images. AI is expected to reduce workload for human experts, improve the objectivity and consistency of pathology reports, and have a clinical impact by extracting hidden information from routinely available data. Here, we describe how AI can be used to predict cancer outcome, treatment response, genetic alterations and gene expression from digitized histopathology slides. We summarize the underlying technologies and emerging approaches, noting limitations, including the need for data sharing and standards. Finally, we discuss the broader implications of AI in cancer research and oncology.
Collapse
Affiliation(s)
- Artem Shmatko
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | | | - Moritz Gerstung
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
- Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany.
| |
Collapse
|