1
|
Zhu M, Sali R, Baba F, Khasawneh H, Ryndin M, Leveillee RJ, Hurwitz MD, Lui K, Dixon C, Zhang DY. Artificial intelligence in pathologic diagnosis, prognosis and prediction of prostate cancer. AMERICAN JOURNAL OF CLINICAL AND EXPERIMENTAL UROLOGY 2024; 12:200-215. [PMID: 39308594 PMCID: PMC11411179 DOI: 10.62347/jsae9732] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/12/2024] [Accepted: 08/19/2024] [Indexed: 09/25/2024]
Abstract
Histopathology, which is the gold-standard for prostate cancer diagnosis, faces significant challenges. With prostate cancer ranking among the most common cancers in the United States and worldwide, pathologists experience an increased number for prostate biopsies. At the same time, precise pathological assessment and classification are necessary for risk stratification and treatment decisions in prostate cancer care, adding to the challenge to pathologists. Recent advancement in digital pathology makes artificial intelligence and learning tools adopted in histopathology feasible. In this review, we introduce the concept of AI and its various techniques in the field of histopathology. We summarize the clinical applications of AI pathology for prostate cancer, including pathological diagnosis, grading, prognosis evaluation, and treatment options. We also discuss how AI applications can be integrated into the routine pathology workflow. With these rapid advancements, it is evident that AI applications in prostate cancer go beyond the initial goal of being tools for diagnosis and grading. Instead, pathologists can provide additional information to improve long-term patient outcomes by assessing detailed histopathologic features at pixel level using digital pathology and AI. Our review not only provides a comprehensive summary of the existing research but also offers insights for future advancements.
Collapse
Affiliation(s)
- Min Zhu
- Department of Computational Pathology, NovinoAI1443 NE 4th Ave, Fort Lauderdale, FL 33304, USA
| | - Rasoul Sali
- Department of Computational Pathology, NovinoAI1443 NE 4th Ave, Fort Lauderdale, FL 33304, USA
- Department of Radiation Oncology, Stanford University School of MedicineStanford, CA 94305, USA
| | - Firas Baba
- Department of Computational Pathology, NovinoAI1443 NE 4th Ave, Fort Lauderdale, FL 33304, USA
| | - Hamdi Khasawneh
- King Hussein School of Computing Sciences, Princess Sumaya University for TechnologyAmman 11855, Jordan
| | - Michelle Ryndin
- College of Agriculture and Life Sciences, Cornell University616 Thurston Ave, Ithaca, NY 14853, USA
| | - Raymond J Leveillee
- Department of Surgery, Florida Atlantic University, Division of Urology, Bethesda Hospital East, Baptist Health South Florida2800 S. Seacrest Drive, Boynton Beach, FL 33435, USA
| | - Mark D Hurwitz
- Department of Radiation Medicine, New York Medical College and Westchester Medical CenterValhalla, NY 10595, USA
| | - Kin Lui
- Department of Urology, Mount Sinai HospitalNew York, NY 10029, USA
| | - Christopher Dixon
- Department of Urology, Good Samaritan Hospital, Westchester Medical Center Health NetworkSuffern, NY 10901, USA
| | - David Y Zhang
- Department of Computational Pathology, NovinoAI1443 NE 4th Ave, Fort Lauderdale, FL 33304, USA
- Pathology and Laboratory Services, Department of Veterans Affairs New York Harbor Healthcare SystemNew York, NY 10010, USA
| |
Collapse
|
2
|
Unger M, Kather JN. A systematic analysis of deep learning in genomics and histopathology for precision oncology. BMC Med Genomics 2024; 17:48. [PMID: 38317154 PMCID: PMC10845449 DOI: 10.1186/s12920-024-01796-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 01/02/2024] [Indexed: 02/07/2024] Open
Abstract
BACKGROUND Digitized histopathological tissue slides and genomics profiling data are available for many patients with solid tumors. In the last 5 years, Deep Learning (DL) has been broadly used to extract clinically actionable information and biological knowledge from pathology slides and genomic data in cancer. In addition, a number of recent studies have introduced multimodal DL models designed to simultaneously process both images from pathology slides and genomic data as inputs. By comparing patterns from one data modality with those in another, multimodal DL models are capable of achieving higher performance compared to their unimodal counterparts. However, the application of these methodologies across various tumor entities and clinical scenarios lacks consistency. METHODS Here, we present a systematic survey of the academic literature from 2010 to November 2023, aiming to quantify the application of DL for pathology, genomics, and the combined use of both data types. After filtering 3048 publications, our search identified 534 relevant articles which then were evaluated by basic (diagnosis, grading, subtyping) and advanced (mutation, drug response and survival prediction) application types, publication year and addressed cancer tissue. RESULTS Our analysis reveals a predominant application of DL in pathology compared to genomics. However, there is a notable surge in DL incorporation within both domains. Furthermore, while DL applied to pathology primarily targets the identification of histology-specific patterns in individual tissues, DL in genomics is more commonly used in a pan-cancer context. Multimodal DL, on the contrary, remains a niche topic, evidenced by a limited number of publications, primarily focusing on prognosis predictions. CONCLUSION In summary, our quantitative analysis indicates that DL not only has a well-established role in histopathology but is also being successfully integrated into both genomic and multimodal applications. In addition, there is considerable potential in multimodal DL for harnessing further advanced tasks, such as predicting drug response. Nevertheless, this review also underlines the need for further research to bridge the existing gaps in these fields.
Collapse
Affiliation(s)
- Michaela Unger
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
3
|
Foran DJ, Chen W, Kurc T, Gupta R, Kaczmarzyk JR, Torre-Healy LA, Bremer E, Ajjarapu S, Do N, Harris G, Stroup A, Durbin E, Saltz JH. An Intelligent Search & Retrieval System (IRIS) and Clinical and Research Repository for Decision Support Based on Machine Learning and Joint Kernel-based Supervised Hashing. Cancer Inform 2024; 23:11769351231223806. [PMID: 38322427 PMCID: PMC10840403 DOI: 10.1177/11769351231223806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 12/14/2023] [Indexed: 02/08/2024] Open
Abstract
Large-scale, multi-site collaboration is becoming indispensable for a wide range of research and clinical activities in oncology. To facilitate the next generation of advances in cancer biology, precision oncology and the population sciences it will be necessary to develop and implement data management and analytic tools that empower investigators to reliably and objectively detect, characterize and chronicle the phenotypic and genomic changes that occur during the transformation from the benign to cancerous state and throughout the course of disease progression. To facilitate these efforts it is incumbent upon the informatics community to establish the workflows and architectures that automate the aggregation and organization of a growing range and number of clinical data types and modalities ranging from new molecular and laboratory tests to sophisticated diagnostic imaging studies. In an attempt to meet those challenges, leading health care centers across the country are making steep investments to establish enterprise-wide, data warehouses. A significant limitation of many data warehouses, however, is that they are designed to support only alphanumeric information. In contrast to those traditional designs, the system that we have developed supports automated collection and mining of multimodal data including genomics, digital pathology and radiology images. In this paper, our team describes the design, development and implementation of a multi-modal, Clinical & Research Data Warehouse (CRDW) that is tightly integrated with a suite of computational and machine-learning tools to provide actionable insight into the underlying characteristics of the tumor environment that would not be revealed using standard methods and tools. The System features a flexible Extract, Transform and Load (ETL) interface that enables it to adapt to aggregate data originating from different clinical and research sources depending on the specific EHR and other data sources utilized at a given deployment site.
Collapse
Affiliation(s)
- David J Foran
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Wenjin Chen
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, The State University of New York, Stony Brook, NY, USA
| | - Rajarshi Gupta
- Department of Biomedical Informatics, Stony Brook University, The State University of New York, Stony Brook, NY, USA
| | | | | | - Erich Bremer
- Department of Biomedical Informatics, Stony Brook University, The State University of New York, Stony Brook, NY, USA
| | | | - Nhan Do
- VA Healthcare System Jamaica Plain Campus, Boston, MA, USA
| | - Gerald Harris
- New Jersey State Cancer Registry, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Antoinette Stroup
- New Jersey State Cancer Registry, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Eric Durbin
- Kentucky Cancer Registry, Markey Cancer Center, Lexington, KY, USA
| | - Joel H Saltz
- Department of Biomedical Informatics, Stony Brook University, The State University of New York, Stony Brook, NY, USA
| |
Collapse
|
4
|
Feng X, Shu W, Li M, Li J, Xu J, He M. Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview. J Transl Med 2024; 22:131. [PMID: 38310237 PMCID: PMC10837897 DOI: 10.1186/s12967-024-04915-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/20/2024] [Indexed: 02/05/2024] Open
Abstract
The capability to gather heterogeneous data, alongside the increasing power of artificial intelligence to examine it, leading a revolution in harnessing multimodal data in the life sciences. However, most approaches are limited to unimodal data, leaving integrated approaches across modalities relatively underdeveloped in computational pathology. Pathogenomics, as an invasive method to integrate advanced molecular diagnostics from genomic data, morphological information from histopathological imaging, and codified clinical data enable the discovery of new multimodal cancer biomarkers to propel the field of precision oncology in the coming decade. In this perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods in pathogenomics. It includes correlation between the pathological and genomic profile of cancer, fusion of histology, and genomics profile of cancer. We also present challenges, opportunities, and avenues for future work.
Collapse
Affiliation(s)
- Xiaobing Feng
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Wen Shu
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Mingya Li
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Junyu Li
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Junyao Xu
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Min He
- College of Electrical and Information Engineering, Hunan University, Changsha, China.
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China.
| |
Collapse
|
5
|
Al-Thelaya K, Gilal NU, Alzubaidi M, Majeed F, Agus M, Schneider J, Househ M. Applications of discriminative and deep learning feature extraction methods for whole slide image analysis: A survey. J Pathol Inform 2023; 14:100335. [PMID: 37928897 PMCID: PMC10622844 DOI: 10.1016/j.jpi.2023.100335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 07/17/2023] [Accepted: 07/19/2023] [Indexed: 11/07/2023] Open
Abstract
Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by "engineered" methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.
Collapse
Affiliation(s)
- Khaled Al-Thelaya
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Nauman Ullah Gilal
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Mahmood Alzubaidi
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Fahad Majeed
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Marco Agus
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Jens Schneider
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Mowafa Househ
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| |
Collapse
|
6
|
Rabilloud N, Allaume P, Acosta O, De Crevoisier R, Bourgade R, Loussouarn D, Rioux-Leclercq N, Khene ZE, Mathieu R, Bensalah K, Pecot T, Kammerer-Jacquet SF. Deep Learning Methodologies Applied to Digital Pathology in Prostate Cancer: A Systematic Review. Diagnostics (Basel) 2023; 13:2676. [PMID: 37627935 PMCID: PMC10453406 DOI: 10.3390/diagnostics13162676] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 08/09/2023] [Accepted: 08/11/2023] [Indexed: 08/27/2023] Open
Abstract
Deep learning (DL), often called artificial intelligence (AI), has been increasingly used in Pathology thanks to the use of scanners to digitize slides which allow us to visualize them on monitors and process them with AI algorithms. Many articles have focused on DL applied to prostate cancer (PCa). This systematic review explains the DL applications and their performances for PCa in digital pathology. Article research was performed using PubMed and Embase to collect relevant articles. A Risk of Bias (RoB) was assessed with an adaptation of the QUADAS-2 tool. Out of the 77 included studies, eight focused on pre-processing tasks such as quality assessment or staining normalization. Most articles (n = 53) focused on diagnosis tasks like cancer detection or Gleason grading. Fifteen articles focused on prediction tasks, such as recurrence prediction or genomic correlations. Best performances were reached for cancer detection with an Area Under the Curve (AUC) up to 0.99 with algorithms already available for routine diagnosis. A few biases outlined by the RoB analysis are often found in these articles, such as the lack of external validation. This review was registered on PROSPERO under CRD42023418661.
Collapse
Affiliation(s)
- Noémie Rabilloud
- Impact TEAM, Laboratoire Traitement du Signal et de l’Image (LTSI) INSERM, Rennes University, 35033 Rennes, France (S.-F.K.-J.)
| | - Pierre Allaume
- Department of Pathology, Rennes University Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France; (P.A.)
| | - Oscar Acosta
- Impact TEAM, Laboratoire Traitement du Signal et de l’Image (LTSI) INSERM, Rennes University, 35033 Rennes, France (S.-F.K.-J.)
| | - Renaud De Crevoisier
- Impact TEAM, Laboratoire Traitement du Signal et de l’Image (LTSI) INSERM, Rennes University, 35033 Rennes, France (S.-F.K.-J.)
- Department of Radiotherapy, Centre Eugène Marquis, 35033 Rennes, France
| | - Raphael Bourgade
- Department of Pathology, Nantes University Hospital, 44000 Nantes, France
| | | | - Nathalie Rioux-Leclercq
- Department of Pathology, Rennes University Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France; (P.A.)
| | - Zine-eddine Khene
- Impact TEAM, Laboratoire Traitement du Signal et de l’Image (LTSI) INSERM, Rennes University, 35033 Rennes, France (S.-F.K.-J.)
- Department of Urology, Rennes University Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Romain Mathieu
- Department of Urology, Rennes University Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Karim Bensalah
- Department of Urology, Rennes University Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Thierry Pecot
- Facility for Artificial Intelligence and Image Analysis (FAIIA), Biosit UAR 3480 CNRS-US18 INSERM, Rennes University, 2 Avenue du Professeur Léon Bernard, 35042 Rennes, France
| | - Solene-Florence Kammerer-Jacquet
- Impact TEAM, Laboratoire Traitement du Signal et de l’Image (LTSI) INSERM, Rennes University, 35033 Rennes, France (S.-F.K.-J.)
- Department of Pathology, Rennes University Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France; (P.A.)
| |
Collapse
|
7
|
Rivero Belenchón I, Checcucci E, Gómez Rivas J, Puliatti S, Taratkin M, Kowalewski KF, Rodler S, Veccia A, Medina Lopez RA, Cacciamani G. Comment on "Artificial intelligence to predict oncological outcome directly from hematoxylin and eosin-stained slides in urology: a systematic review". Minerva Urol Nephrol 2022; 74:810-812. [PMID: 36629813 DOI: 10.23736/s2724-6051.22.05180-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Affiliation(s)
- Inés Rivero Belenchón
- Department of Urology and Nephrology, Biomedical Institute of Seville (IBiS), Virgen del Rocío University Hospital, University of Seville, Seville, Spain - .,Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands -
| | - Enrico Checcucci
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Division of Urology, Department of Surgery, IRCC Candiolo Cancer Institute, Candiolo, Turin, Italy
| | - Juan Gómez Rivas
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, San Carlos Hospital, Madrid, Spain
| | - Stefano Puliatti
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Mark Taratkin
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Institute for Urology and Reproductive Health, Sechenov University, Moscow, Russia
| | - Karl-Friedrich Kowalewski
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany
| | - Severin Rodler
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, Klinikum der Univertität München, Munich, Germany
| | - Alessandro Veccia
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Unit of Urology, AOUI Verona, Verona, Italy
| | - Rafael A Medina Lopez
- Department of Urology and Nephrology, Biomedical Institute of Seville (IBiS), Virgen del Rocío University Hospital, University of Seville, Seville, Spain
| | - Giovanni Cacciamani
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Catherine and Joseph Aresty Department of Urology, US Institute of Urology, University of Southern California, Los Angeles, CA, USA
| | | |
Collapse
|
8
|
Wessels F, Kuntz S, Krieghoff-Henning E, Schmitt M, Braun V, Worst TS, Neuberger M, Steeg M, Gaiser T, Fröhling S, Michel MS, Nuhn P, Brinker TJ. Artificial intelligence to predict oncological outcome directly from hematoxylin and eosin-stained slides in urology. Minerva Urol Nephrol 2022; 74:538-550. [PMID: 35274903 DOI: 10.23736/s2724-6051.22.04758-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
INTRODUCTION Artificial intelligence (AI) has been successfully applied for automatic tumor detection and grading in histopathological image analysis in urologic oncology. The aim of this review was to assess the applicability of these approaches in image-based oncological outcome prediction. EVIDENCE ACQUISITION A systematic literature search was conducted using the databases MEDLINE through PubMed and Web of Science up to April 20, 2021. Studies investigating AI approaches to determine the risk of recurrence, metastasis, or survival directly from H&E-stained tissue sections in prostate, renal cell or urothelial carcinoma were included. Characteristics of the AI approach and performance metrics were extracted and summarized. Risk of bias (RoB) was assessed using the PROBAST tool. EVIDENCE SYNTHESIS 16 studies yielding a total of 6658 patients and reporting on 17 outcome predictions were included. Six studies focused on renal cell, six on prostate and three on urothelial carcinoma while one study investigated renal cell and urothelial carcinoma. Handcrafted feature extraction was used in five, a convolutional neural network (CNN) in six and a deep feature extraction in four studies. One study compared a CNN with handcrafted feature extraction. In seven outcome predictions, a multivariable comparison with clinicopathological parameters was reported. Five of them showed statistically significant hazard ratios for the AI's model's-prediction. However, RoB was high in 15 outcome predictions and unclear in two. CONCLUSIONS The included studies are promising but predominantly early pilot studies, therefore primarily highlighting the potential of AI approaches. Additional well-designed studies are needed to assess the actual clinical applicability.
Collapse
Affiliation(s)
- Frederik Wessels
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.,Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Sara Kuntz
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Max Schmitt
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Volker Braun
- Library for the Medical Faculty Mannheim of Heidelberg University, Mannheim, Germany
| | - Thomas S Worst
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Manuel Neuberger
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Matthias Steeg
- Institute of Pathology, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Timo Gaiser
- Institute of Pathology, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Stefan Fröhling
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Maurice-Stephan Michel
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Philipp Nuhn
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany -
| |
Collapse
|
9
|
Foran DJ, Durbin EB, Chen W, Sadimin E, Sharma A, Banerjee I, Kurc T, Li N, Stroup AM, Harris G, Gu A, Schymura M, Gupta R, Bremer E, Balsamo J, DiPrima T, Wang F, Abousamra S, Samaras D, Hands I, Ward K, Saltz JH. An Expandable Informatics Framework for Enhancing Central Cancer Registries with Digital Pathology Specimens, Computational Imaging Tools, and Advanced Mining Capabilities. J Pathol Inform 2022; 13:5. [PMID: 35136672 PMCID: PMC8794027 DOI: 10.4103/jpi.jpi_31_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 04/30/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Population-based state cancer registries are an authoritative source for cancer statistics in the United States. They routinely collect a variety of data, including patient demographics, primary tumor site, stage at diagnosis, first course of treatment, and survival, on every cancer case that is reported across all U.S. states and territories. The goal of our project is to enrich NCI's Surveillance, Epidemiology, and End Results (SEER) registry data with high-quality population-based biospecimen data in the form of digital pathology, machine-learning-based classifications, and quantitative histopathology imaging feature sets (referred to here as Pathomics features). MATERIALS AND METHODS As part of the project, the underlying informatics infrastructure was designed, tested, and implemented through close collaboration with several participating SEER registries to ensure consistency with registry processes, computational scalability, and ability to support creation of population cohorts that span multiple sites. Utilizing computational imaging algorithms and methods to both generate indices and search for matches makes it possible to reduce inter- and intra-observer inconsistencies and to improve the objectivity with which large image repositories are interrogated. RESULTS Our team has created and continues to expand a well-curated repository of high-quality digitized pathology images corresponding to subjects whose data are routinely collected by the collaborating registries. Our team has systematically deployed and tested key, visual analytic methods to facilitate automated creation of population cohorts for epidemiological studies and tools to support visualization of feature clusters and evaluation of whole-slide images. As part of these efforts, we are developing and optimizing advanced search and matching algorithms to facilitate automated, content-based retrieval of digitized specimens based on their underlying image features and staining characteristics. CONCLUSION To meet the challenges of this project, we established the analytic pipelines, methods, and workflows to support the expansion and management of a growing repository of high-quality digitized pathology and information-rich, population cohorts containing objective imaging and clinical attributes to facilitate studies that seek to discriminate among different subtypes of disease, stratify patient populations, and perform comparisons of tumor characteristics within and across patient cohorts. We have also successfully developed a suite of tools based on a deep-learning method to perform quantitative characterizations of tumor regions, assess infiltrating lymphocyte distributions, and generate objective nuclear feature measurements. As part of these efforts, our team has implemented reliable methods that enable investigators to systematically search through large repositories to automatically retrieve digitized pathology specimens and correlated clinical data based on their computational signatures.
Collapse
Affiliation(s)
- David J. Foran
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
- Department of Pathology and Laboratory Medicine, Rutgers-Robert Wood Johnson Medical School, Piscataway, NJ, USA
| | - Eric B. Durbin
- Kentucky Cancer Registry, Markey Cancer Center, University of Kentucky, Lexington, KY, USA
- Division of Biomedical Informatics, Department of Internal Medicine, College of Medicine, Lexington, KY, USA
| | - Wenjin Chen
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Evita Sadimin
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
- Department of Pathology and Laboratory Medicine, Rutgers-Robert Wood Johnson Medical School, Piscataway, NJ, USA
| | - Ashish Sharma
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Imon Banerjee
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Nan Li
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Antoinette M. Stroup
- New Jersey State Cancer Registry, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Gerald Harris
- New Jersey State Cancer Registry, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Annie Gu
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Maria Schymura
- New York State Cancer Registry, New York State Department of Health, Albany, NY, USA
| | - Rajarsi Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Erich Bremer
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Joseph Balsamo
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Tammy DiPrima
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Feiqiao Wang
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Shahira Abousamra
- Department of Computer Science, Stony Brook University, Stony Brook, NY, USA
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY, USA
| | - Isaac Hands
- Division of Biomedical Informatics, Department of Internal Medicine, College of Medicine, Lexington, KY, USA
| | - Kevin Ward
- Georgia State Cancer Registry, Georgia Department of Public Health, Atlanta, GA, USA
| | - Joel H. Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| |
Collapse
|
10
|
Lu C, Shiradkar R, Liu Z. Integrating pathomics with radiomics and genomics for cancer prognosis: A brief review. Chin J Cancer Res 2021; 33:563-573. [PMID: 34815630 PMCID: PMC8580801 DOI: 10.21147/j.issn.1000-9604.2021.05.03] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 10/22/2021] [Indexed: 11/18/2022] Open
Abstract
In the last decade, the focus of computational pathology research community has shifted from replicating the pathological examination for diagnosis done by pathologists to unlocking and discovering "sub-visual" prognostic image cues from the histopathological image. While we are getting more knowledge and experience in digital pathology, the emerging goal is to integrate other-omics or modalities that will contribute for building a better prognostic assay. In this paper, we provide a brief review of representative works that focus on integrating pathomics with radiomics and genomics for cancer prognosis. It includes: correlation of pathomics and genomics; fusion of pathomics and genomics; fusion of pathomics and radiomics. We also present challenges, potential opportunities, and avenues for future work.
Collapse
Affiliation(s)
- Cheng Lu
- Biomedical Engineering Department, Case Western Reserve University, Cleveland 44106, OH, USA
| | - Rakesh Shiradkar
- Biomedical Engineering Department, Case Western Reserve University, Cleveland 44106, OH, USA
| | - Zaiyi Liu
- Department of Radiology, Guangzhou First People's Hospital, School of Medicine, South China University of Technology, Guangzhou 510080, China
| |
Collapse
|
11
|
Tolkach Y, Dohmgörgen T, Toma M, Kristiansen G. High-accuracy prostate cancer pathology using deep learning. NAT MACH INTELL 2020. [DOI: 10.1038/s42256-020-0200-7] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
|
12
|
Panayides AS, Amini A, Filipovic ND, Sharma A, Tsaftaris SA, Young A, Foran D, Do N, Golemati S, Kurc T, Huang K, Nikita KS, Veasey BP, Zervakis M, Saltz JH, Pattichis CS. AI in Medical Imaging Informatics: Current Challenges and Future Directions. IEEE J Biomed Health Inform 2020; 24:1837-1857. [PMID: 32609615 PMCID: PMC8580417 DOI: 10.1109/jbhi.2020.2991043] [Citation(s) in RCA: 112] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.
Collapse
|
13
|
She Y, Jin Z, Wu J, Deng J, Zhang L, Su H, Jiang G, Liu H, Xie D, Cao N, Ren Y, Chen C. Development and Validation of a Deep Learning Model for Non-Small Cell Lung Cancer Survival. JAMA Netw Open 2020; 3:e205842. [PMID: 32492161 PMCID: PMC7272121 DOI: 10.1001/jamanetworkopen.2020.5842] [Citation(s) in RCA: 121] [Impact Index Per Article: 30.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
IMPORTANCE There is a lack of studies exploring the performance of a deep learning survival neural network in non-small cell lung cancer (NSCLC). OBJECTIVES To compare the performances of DeepSurv, a deep learning survival neural network with a tumor, node, and metastasis staging system in the prediction of survival and test the reliability of individual treatment recommendations provided by the deep learning survival neural network. DESIGN, SETTING, AND PARTICIPANTS In this population-based cohort study, a deep learning-based algorithm was developed and validated using consecutive cases of newly diagnosed stages I to IV NSCLC between January 2010 and December 2015 in a Surveillance, Epidemiology, and End Results database. A total of 127 features, including patient characteristics, tumor stage, and treatment strategies, were assessed for analysis. The algorithm was externally validated on an independent test cohort, comprising 1182 patients with stage I to III NSCLC diagnosed between January 2009 and December 2013 in Shanghai Pulmonary Hospital. Analysis began January 2018 and ended June 2019. MAIN OUTCOMES AND MEASURES The deep learning survival neural network model was compared with the tumor, node, and metastasis staging system for lung cancer-specific survival. The C statistic was used to assess the performance of models. A user-friendly interface was provided to facilitate the survival predictions and treatment recommendations of the deep learning survival neural network model. RESULTS Of 17 322 patients with NSCLC included in the study, 13 361 (77.1%) were white and the median (interquartile range) age was 68 (61-74) years. The majority of tumors were stage I disease (10 273 [59.3%]) and adenocarcinoma (11 985 [69.2%]). The median (interquartile range) follow-up time was 24 (10-43) months. There were 3119 patients who had lung cancer-related death during the follow-up period. The deep learning survival neural network model showed more promising results in the prediction of lung cancer-specific survival than the tumor, node, and metastasis stage on the test data set (C statistic = 0.739 vs 0.706). The population who received the recommended treatments had superior survival rates than those who received treatments not recommended (hazard ratio, 2.99; 95% CI, 2.49-3.59; P < .001), which was verified by propensity score-matched groups. The deep learning survival neural network model visualization was realized by a user-friendly graphic interface. CONCLUSIONS AND RELEVANCE The deep learning survival neural network model shows potential benefits in prognostic evaluation and treatment recommendation with respect to lung cancer-specific survival. This novel analytical approach may provide reliable individual survival information and treatment recommendations.
Collapse
Affiliation(s)
- Yunlang She
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Zhuochen Jin
- College of Design and Innovation, Tongji University, Shanghai, China
| | - Junqi Wu
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Jiajun Deng
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Lei Zhang
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Hang Su
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Gening Jiang
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Haipeng Liu
- Shanghai Key Laboratory of Tuberculosis, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Dong Xie
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Nan Cao
- College of Design and Innovation, Tongji University, Shanghai, China
- Computer Science, NYU Shanghai, Shanghai, China
| | - Yijiu Ren
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Chang Chen
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| |
Collapse
|
14
|
Gupta R, Kurc T, Sharma A, Almeida JS, Saltz J. The Emergence of Pathomics. CURRENT PATHOBIOLOGY REPORTS 2019. [DOI: 10.1007/s40139-019-00200-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|