1
|
Jiang S, Hondelink L, Suriawinata AA, Hassanpour S. Masked pre-training of transformers for histology image analysis. J Pathol Inform 2024; 15:100386. [PMID: 39006998 PMCID: PMC11246055 DOI: 10.1016/j.jpi.2024.100386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Revised: 04/02/2024] [Accepted: 05/28/2024] [Indexed: 07/16/2024] Open
Abstract
In digital pathology, whole-slide images (WSIs) are widely used for applications such as cancer diagnosis and prognosis prediction. Vision transformer (ViT) models have recently emerged as a promising method for encoding large regions of WSIs while preserving spatial relationships among patches. However, due to the large number of model parameters and limited labeled data, applying transformer models to WSIs remains challenging. In this study, we propose a pretext task to train the transformer model in a self-supervised manner. Our model, MaskHIT, uses the transformer output to reconstruct masked patches, measured by contrastive loss. We pre-trained MaskHIT model using over 7000 WSIs from TCGA and extensively evaluated its performance in multiple experiments, covering survival prediction, cancer subtype classification, and grade prediction tasks. Our experiments demonstrate that the pre-training procedure enables context-aware understanding of WSIs, facilitates the learning of representative histological features based on patch positions and visual patterns, and is essential for the ViT model to achieve optimal results on WSI-level tasks. The pre-trained MaskHIT surpasses various multiple instance learning approaches by 3% and 2% on survival prediction and cancer subtype classification tasks, and also outperforms recent state-of-the-art transformer-based methods. Finally, a comparison between the attention maps generated by the MaskHIT model with pathologist's annotations indicates that the model can accurately identify clinically relevant histological structures on the whole slide for each task.
Collapse
Affiliation(s)
- Shuai Jiang
- Department of Biomedical Data Science, Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA
| | - Liesbeth Hondelink
- Department of Biomedical Data Science, Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA
| | - Arief A. Suriawinata
- Department of Pathology and Laboratory Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, NH 03756, USA
| | - Saeed Hassanpour
- Department of Biomedical Data Science, Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA
- Department of Epidemiology, Geisel School of Medicine at Dartmouth and the Department of Computer Science, Dartmouth College, Hanover, NH 03755, USA
| |
Collapse
|
2
|
Brevet M, Li Z, Parwani A. Computational pathology in the identification of HER2-low breast cancer: Opportunities and challenges. J Pathol Inform 2024; 15:100343. [PMID: 38125925 PMCID: PMC10730362 DOI: 10.1016/j.jpi.2023.100343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 09/18/2023] [Accepted: 11/01/2023] [Indexed: 12/23/2023] Open
Abstract
For the past 2 decades, pathologists have been accustomed to reporting the HER2 status of breast cancer as either positive or negative, based on HER2 IHC. Today, however, there is a clinical imperative to employ a 3-tier approach to interpreting HER2 IHC that can also identify tumours categorised as HER2-low. Meeting this need for a finer degree of discrimination may be challenging, and in this article, we consider the potential for the integration of computational approaches to support pathologists in achieving accurate and reproducible HER2 IHC scoring as well as outlining some of the practicalities involved.
Collapse
Affiliation(s)
| | - Zaibo Li
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Anil Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| |
Collapse
|
3
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
4
|
Gonzalez R, Saha A, Campbell CJ, Nejat P, Lokker C, Norgan AP. Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities. J Pathol Inform 2024; 15:100347. [PMID: 38162950 PMCID: PMC10755052 DOI: 10.1016/j.jpi.2023.100347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 10/06/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024] Open
Abstract
This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.
Collapse
Affiliation(s)
- Ricardo Gonzalez
- DeGroote School of Business, McMaster University, Hamilton, Ontario, Canada
- Division of Computational Pathology and Artificial Intelligence, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| | - Ashirbani Saha
- Department of Oncology, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- Escarpment Cancer Research Institute, McMaster University and Hamilton Health Sciences, Hamilton, Ontario, Canada
| | - Clinton J.V. Campbell
- William Osler Health System, Brampton, Ontario, Canada
- Department of Pathology and Molecular Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Peyman Nejat
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Cynthia Lokker
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Andrew P. Norgan
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
5
|
Wei B, Li L, Feng Y, Liu S, Fu P, Tian L. Exploring prognostic biomarkers in pathological images of colorectal cancer patients via deep learning. J Pathol Clin Res 2024; 10:e70003. [PMID: 39343999 DOI: 10.1002/2056-4538.70003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2024] [Revised: 07/17/2024] [Accepted: 08/30/2024] [Indexed: 10/01/2024]
Abstract
Hematoxylin and eosin (H&E) whole slide images provide valuable information for predicting prognostic outcomes in colorectal cancer (CRC) patients. However, extracting prognostic indicators from pathological images is challenging due to the subtle complexities of phenotypic information. We trained a weakly supervised deep learning model on data from 640 CRC patients in the prostate, lung, colorectal, and ovarian (PLCO) cancer screening trial dataset and validated it using data from 522 CRC patients in the cancer genome atlas (TCGA) dataset. We created the colorectal cancer risk score (CRCRS) to assess patient prognosis, visualized the pathological phenotype of the risk score using Grad-CAM, and employed multiomics data from the TCGA CRC cohort to investigate the potential biological mechanisms underlying the risk score. The overall survival analysis revealed that the CRCRS served as an independent prognostic indicator for both the PLCO cohort (p < 0.001) and the TCGA cohort (p < 0.001), with its predictive efficacy remaining unaffected by the clinical staging system. Additionally, satisfactory chemotherapeutic benefits were observed in stage II/III CRC patients with high CRCRS but not in those with low CRCRS. A pathomics nomogram constructed by integrating the CRCRS with the tumor-node-metastasis (TNM) staging system enhanced prognostic prediction accuracy compared with using the TNM staging system alone. Noteworthy features of the risk score were identified, such as immature tumor mesenchyme, disorganized gland structures, small clusters of cancer cells associated with unfavorable prognosis, and infiltrating inflammatory cells associated with favorable prognosis. The TCGA multiomics data revealed potential correlations between the CRCRS and the activation of energy production and metabolic pathways, the tumor immune microenvironment, and genetic mutations in APC, SMAD2, EEF1AKMT4, EPG5, and TANC1. In summary, our deep learning algorithm identified the CRCRS as a prognostic indicator in CRC, providing a significant approach for prognostic risk stratification and tailoring precise treatment strategies for individual patients.
Collapse
Affiliation(s)
- Binshen Wei
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, Harbin, PR China
| | - Linqing Li
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, Harbin, PR China
| | - Yenan Feng
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, Harbin, PR China
| | - Sihan Liu
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, Harbin, PR China
| | - Peng Fu
- Department of Nuclear Medicine, The First Affiliated Hospital of Harbin Medical University, Harbin, PR China
| | - Lin Tian
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, Harbin, PR China
| |
Collapse
|
6
|
Prassas I, Clarke B, Youssef T, Phlamon J, Dimitrakopoulos L, Rofaeil A, Yousef GM. Computational pathology: an evolving concept. Clin Chem Lab Med 2024; 62:2148-2155. [PMID: 38646706 DOI: 10.1515/cclm-2023-1124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 04/10/2024] [Indexed: 04/23/2024]
Abstract
The initial enthusiasm about computational pathology (CP) and artificial intelligence (AI) was that they will replace pathologists entirely on the way to fully automated diagnostics. It is becoming clear that currently this is not the immediate model to pursue. On top of the legal and regulatory complexities surrounding its implementation, the majority of tested machine learning (ML)-based predictive algorithms do not display the exquisite performance needed to render them unequivocal, standalone decision makers for matters with direct implications to human health. We are thus moving into a different model of "computer-assisted diagnostics", where AI is there to provide support, rather than replacing, the pathologist. Herein we focus on the practical aspects of CP, from a pathologist perspective. There is a wide range of potential applications where CP can enhance precision of pathology diagnosis, tailor prognostic and predictive information, as well as save time. There are, however, a number of potential limitations for CP that currently hinder their wider adoption in the clinical setting. We address the key necessary steps towards clinical implementation of computational pathology, discuss the significant obstacles that hinders its adoption in the clinical context and summarize some proposed solutions. We conclude that the advancement of CP in the clinic is a promising resource-intensive endeavour that requires broad and inclusive collaborations between academia, industry, and regulatory bodies.
Collapse
Affiliation(s)
- Ioannis Prassas
- Laboratory Medicine Program, 7989 University Health Network , Toronto, ON, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada
| | - Blaise Clarke
- Laboratory Medicine Program, 7989 University Health Network , Toronto, ON, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada
| | - Timothy Youssef
- Laboratory Medicine Program, 7989 University Health Network , Toronto, ON, Canada
| | - Juliana Phlamon
- Laboratory Medicine Program, 7989 University Health Network , Toronto, ON, Canada
| | | | - Andrew Rofaeil
- Laboratory Medicine Program, 7989 University Health Network , Toronto, ON, Canada
| | - George M Yousef
- Laboratory Medicine Program, 7989 University Health Network , Toronto, ON, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
7
|
Winter L, Mendelsohn DH, Walter N, Popp D, Geis S, Niedermair T, Mamilos A, Gessner A, Salzberger B, Pfister K, Stroszczynski C, Alt V, Rupp M, Brochhausen C. Multidisciplinary Teams in Musculoskeletal Infection - From a Pathologist's Perspective. Pathol Res Pract 2024; 262:155539. [PMID: 39151251 DOI: 10.1016/j.prp.2024.155539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2024] [Accepted: 08/11/2024] [Indexed: 08/19/2024]
Abstract
Multidisciplinary team (MDT) meetings have emerged as a promising approach for the treatment of cancer patients. These meetings involve a team of healthcare professionals from different disciplines working together to develop a holistic, patient-centered treatment. Although MDT meetings are well established in oncology, they play a minor role in other diseases. Recent evidence suggests that the implementation of MDT meetings can improve patient outcomes in musculoskeletal infections. The aim of this retrospective, observational study was to present the agenda of our multidisciplinary limb board including live microscopy with a special focus on the pathologist's role. The descriptive analysis of the limb board included 66 cases receiving live microscopy at the meeting and a total of 124 histopathological findings and 181 stainings. We could elucidate that pathologists seem to play an important role especially in clarifying the correct diagnosis. In 80.3 % of the findings, the pathologist specified the clinical diagnosis of the requesting physician leading to a consensus-based treatment plan for each patient. The implementation of MDT meetings including live microscopy in patients with musculoskeletal infections holds potential benefits, such as improved communication, scientific collaboration, and raising clinicians' awareness and understanding of histopathology findings. However, potential challenges, such as organizational effort and technical prerequisites should be considered.
Collapse
Affiliation(s)
- Lina Winter
- Institute of Pathology, University of Regensburg, Regensburg, Germany; Institute of Pathology, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany.
| | - Daniel H Mendelsohn
- Institute of Pathology, University of Regensburg, Regensburg, Germany; Institute of Pathology, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany; Department for Trauma Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | - Nike Walter
- Department for Trauma Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | - Daniel Popp
- Department for Trauma Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | - Sebastian Geis
- Department for Plastic, Hand & Reconstructive Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | - Tanja Niedermair
- Institute of Pathology, University of Regensburg, Regensburg, Germany.
| | - Andreas Mamilos
- Institute of Pathology, University of Regensburg, Regensburg, Germany; Department of Pathology, German Oncology Center, Limassol, Cyprus.
| | - André Gessner
- Department for Microbiology and Hygiene, University Medical Center Regensburg, Regensburg, Germany.
| | - Bernd Salzberger
- Department of Infection Prevention and Infectious Diseases, University Medical Center Regensburg, Regensburg, Germany.
| | - Karin Pfister
- Department of Vascular and Endovascular Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | | | - Volker Alt
- Department for Trauma Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | - Markus Rupp
- Department for Trauma Surgery, University Medical Center Regensburg, Regensburg, Germany.
| | - Christoph Brochhausen
- Institute of Pathology, University of Regensburg, Regensburg, Germany; Institute of Pathology, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany.
| |
Collapse
|
8
|
Wang Z, Ma J, Gao Q, Bain C, Imoto S, Liò P, Cai H, Chen H, Song J. Dual-stream multi-dependency graph neural network enables precise cancer survival analysis. Med Image Anal 2024; 97:103252. [PMID: 38963973 DOI: 10.1016/j.media.2024.103252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 05/24/2024] [Accepted: 06/21/2024] [Indexed: 07/06/2024]
Abstract
Histopathology image-based survival prediction aims to provide a precise assessment of cancer prognosis and can inform personalized treatment decision-making in order to improve patient outcomes. However, existing methods cannot automatically model the complex correlations between numerous morphologically diverse patches in each whole slide image (WSI), thereby preventing them from achieving a more profound understanding and inference of the patient status. To address this, here we propose a novel deep learning framework, termed dual-stream multi-dependency graph neural network (DM-GNN), to enable precise cancer patient survival analysis. Specifically, DM-GNN is structured with the feature updating and global analysis branches to better model each WSI as two graphs based on morphological affinity and global co-activating dependencies. As these two dependencies depict each WSI from distinct but complementary perspectives, the two designed branches of DM-GNN can jointly achieve the multi-view modeling of complex correlations between the patches. Moreover, DM-GNN is also capable of boosting the utilization of dependency information during graph construction by introducing the affinity-guided attention recalibration module as the readout function. This novel module offers increased robustness against feature perturbation, thereby ensuring more reliable and stable predictions. Extensive benchmarking experiments on five TCGA datasets demonstrate that DM-GNN outperforms other state-of-the-art methods and offers interpretable prediction insights based on the morphological depiction of high-attention patches. Overall, DM-GNN represents a powerful and auxiliary tool for personalized cancer prognosis from histopathology images and has great potential to assist clinicians in making personalized treatment decisions and improving patient outcomes.
Collapse
Affiliation(s)
- Zhikang Wang
- Xiangya Hospital, Central South University, Changsha, China; Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, Australia; Wenzhou Medical University-Monash Biomedicine Discovery Institute (BDI) Alliance in Clinical and Experimental Biomedicine, Wenzhou, China
| | - Jiani Ma
- School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, China
| | - Qian Gao
- Xiangya Hospital, Central South University, Changsha, China
| | - Chris Bain
- Faculty of Information Technology, Monash University, Melbourne, Australia
| | - Seiya Imoto
- Human Genome Center, Institute of Medical Science, The University of Tokyo, Tokyo, Japan
| | - Pietro Liò
- Department of Computer Science and Technology, The University of Cambridge, Cambridge, United Kingdom
| | - Hongmin Cai
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
| | - Hao Chen
- Department of Computer Science and Engineering and Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong, China
| | - Jiangning Song
- Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, Australia; Wenzhou Medical University-Monash Biomedicine Discovery Institute (BDI) Alliance in Clinical and Experimental Biomedicine, Wenzhou, China.
| |
Collapse
|
9
|
Wang AY, Lin S, Tran C, Homer RJ, Wilsdon D, Walsh JC, Goebel EA, Sansano I, Sonawane S, Cockenpot V, Mukhopadhyay S, Taskin T, Zahra N, Cima L, Semerci O, Özamrak BG, Mishra P, Vennavalli NS, Chen PHC, Cecchini MJ. Assessment of Pathology Domain-Specific Knowledge of ChatGPT and Comparison to Human Performance. Arch Pathol Lab Med 2024; 148:1152-1158. [PMID: 38244054 DOI: 10.5858/arpa.2023-0296-oa] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/30/2023] [Indexed: 01/22/2024]
Abstract
CONTEXT.— Artificial intelligence algorithms hold the potential to fundamentally change many aspects of society. Application of these tools, including the publicly available ChatGPT, has demonstrated impressive domain-specific knowledge in many areas, including medicine. OBJECTIVES.— To understand the level of pathology domain-specific knowledge for ChatGPT using different underlying large language models, GPT-3.5 and the updated GPT-4. DESIGN.— An international group of pathologists (n = 15) was recruited to generate pathology-specific questions at a similar level to those that could be seen on licensing (board) examinations. The questions (n = 15) were answered by GPT-3.5, GPT-4, and a staff pathologist who recently passed their Canadian pathology licensing exams. Participants were instructed to score answers on a 5-point scale and to predict which answer was written by ChatGPT. RESULTS.— GPT-3.5 performed at a similar level to the staff pathologist, while GPT-4 outperformed both. The overall score for both GPT-3.5 and GPT-4 was within the range of meeting expectations for a trainee writing licensing examinations. In all but one question, the reviewers were able to correctly identify the answers generated by GPT-3.5. CONCLUSIONS.— By demonstrating the ability of ChatGPT to answer pathology-specific questions at a level similar to (GPT-3.5) or exceeding (GPT-4) a trained pathologist, this study highlights the potential of large language models to be transformative in this space. In the future, more advanced iterations of these algorithms with increased domain-specific knowledge may have the potential to assist pathologists and enhance pathology resident training.
Collapse
Affiliation(s)
- Andrew Y Wang
- From the Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada (Wang, Wilsdon)
| | - Sherman Lin
- Department of Pathology and Laboratory Medicine, Western University and London Health Sciences Centre, London, Ontario, Canada (Lin, Tran, Walsh, Goebel, Cecchini)
| | - Christopher Tran
- Department of Pathology and Laboratory Medicine, Western University and London Health Sciences Centre, London, Ontario, Canada (Lin, Tran, Walsh, Goebel, Cecchini)
| | - Robert J Homer
- Department of Pathology, Yale School of Medicine, New Haven, Connecticut (Homer)
| | - Dan Wilsdon
- From the Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada (Wang, Wilsdon)
| | - Joanna C Walsh
- Department of Pathology and Laboratory Medicine, Western University and London Health Sciences Centre, London, Ontario, Canada (Lin, Tran, Walsh, Goebel, Cecchini)
| | - Emily A Goebel
- Department of Pathology and Laboratory Medicine, Western University and London Health Sciences Centre, London, Ontario, Canada (Lin, Tran, Walsh, Goebel, Cecchini)
| | - Irene Sansano
- Department of Pathology, Hospital Universitari Vall d'Hebron, Barcelona, Spain (Sansano)
| | - Snehal Sonawane
- Department of Pathology, University of Illinois at Chicago, Chicago (Sonawane)
| | - Vincent Cockenpot
- Department of Pathology-Genetics and Immunology, Institut Curie, PSL Research University, Paris, France (Cockenpot)
| | - Sanjay Mukhopadhyay
- Department of Anatomic Pathology, Cleveland Clinic, Cleveland, Ohio (Mukhopadhyay)
| | - Toros Taskin
- Department of Pathology, Agri Training and Research Hospital, Agri Ibrahim Cecen University, Agri, Turkey (Taskin)
| | - Nusrat Zahra
- Department of Pathology, Specialized Healthcare and Medical Education, Punjab, Pakistan (Zahra)
| | - Luca Cima
- Pathology Unit, Department of Laboratory Medicine, Santa Chiara University Hospital, APSS, Trento, Italy (Cima)
| | - Orhan Semerci
- Department of Pathology, Trabzon Kanuni Training and Research Hospital, University of Health Sciences, Trabzon, Turkey (Semerci)
| | - Birsen Gizem Özamrak
- Department of Pathology, Izmir Provincial Directorate of Health, Health Sciences University Izmir Tepecik Education and Research Hospital, Izmir, Turkey (Özamrak)
| | - Pallavi Mishra
- Department of Pathology, Queen Elizabeth Hospital, Lewisham and Greenwich NHS Trust, London, United Kingdom (Mishra)
| | | | | | - Matthew J Cecchini
- Department of Pathology and Laboratory Medicine, Western University and London Health Sciences Centre, London, Ontario, Canada (Lin, Tran, Walsh, Goebel, Cecchini)
| |
Collapse
|
10
|
Marini N, Marchesin S, Wodzinski M, Caputo A, Podareanu D, Guevara BC, Boytcheva S, Vatrano S, Fraggetta F, Ciompi F, Silvello G, Müller H, Atzori M. Multimodal representations of biomedical knowledge from limited training whole slide images and reports using deep learning. Med Image Anal 2024; 97:103303. [PMID: 39154617 DOI: 10.1016/j.media.2024.103303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 08/08/2024] [Accepted: 08/09/2024] [Indexed: 08/20/2024]
Abstract
The increasing availability of biomedical data creates valuable resources for developing new deep learning algorithms to support experts, especially in domains where collecting large volumes of annotated data is not trivial. Biomedical data include several modalities containing complementary information, such as medical images and reports: images are often large and encode low-level information, while reports include a summarized high-level description of the findings identified within data and often only concerning a small part of the image. However, only a few methods allow to effectively link the visual content of images with the textual content of reports, preventing medical specialists from properly benefitting from the recent opportunities offered by deep learning models. This paper introduces a multimodal architecture creating a robust biomedical data representation encoding fine-grained text representations within image embeddings. The architecture aims to tackle data scarcity (combining supervised and self-supervised learning) and to create multimodal biomedical ontologies. The architecture is trained on over 6,000 colon whole slide Images (WSI), paired with the corresponding report, collected from two digital pathology workflows. The evaluation of the multimodal architecture involves three tasks: WSI classification (on data from pathology workflow and from public repositories), multimodal data retrieval, and linking between textual and visual concepts. Noticeably, the latter two tasks are available by architectural design without further training, showing that the multimodal architecture that can be adopted as a backbone to solve peculiar tasks. The multimodal data representation outperforms the unimodal one on the classification of colon WSIs and allows to halve the data needed to reach accurate performance, reducing the computational power required and thus the carbon footprint. The combination of images and reports exploiting self-supervised algorithms allows to mine databases without needing new annotations provided by experts, extracting new information. In particular, the multimodal visual ontology, linking semantic concepts to images, may pave the way to advancements in medicine and biomedical analysis domains, not limited to histopathology.
Collapse
Affiliation(s)
- Niccolò Marini
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland.
| | - Stefano Marchesin
- Department of Information Engineering, University of Padua, Padua, Italy.
| | - Marek Wodzinski
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland; Department of Measurement and Electronics, AGH University of Kraków, Krakow, Poland
| | - Alessandro Caputo
- Department of Pathology, Ruggi University Hospital, Salerno, Italy; Pathology Unit, Gravina Hospital Caltagirone ASP, Catania, Italy
| | | | | | - Svetla Boytcheva
- Ontotext, Sofia, Bulgaria; Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Sofia, Bulgaria
| | - Simona Vatrano
- Pathology Unit, Gravina Hospital Caltagirone ASP, Catania, Italy
| | - Filippo Fraggetta
- Pathology Unit, Gravina Hospital Caltagirone ASP, Catania, Italy; Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Gianmaria Silvello
- Department of Information Engineering, University of Padua, Padua, Italy
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland; Medical faculty, University of Geneva, 1211 Geneva, Switzerland
| | - Manfredo Atzori
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland; Department of Neurosciences, University of Padua, Padua, Italy
| |
Collapse
|
11
|
Murray AL, O'Boyle DS, Walsh BH, Murray DM. Validation of a machine learning algorithm for identifying infants at risk of hypoxic ischaemic encephalopathy in a large unseen data set. Arch Dis Child Fetal Neonatal Ed 2024:fetalneonatal-2024-327366. [PMID: 39251344 DOI: 10.1136/archdischild-2024-327366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/06/2024] [Accepted: 08/21/2024] [Indexed: 09/11/2024]
Abstract
OBJECTIVE To validate a hypoxic ischaemic encephalopathy (HIE) prediction algorithm to identify infants at risk of HIE immediately after birth using readily available clinical data. DESIGN Secondary review of electronic health record data of term deliveries from January 2017 to December 2021. SETTING A tertiary maternity hospital. PATIENTS Infants >36 weeks' gestation with the following clinical variables available: Apgar Score at 1 min and 5 min, postnatal pH, base deficit, and lactate values taken within 1 hour of birth INTERVENTIONS: Previously trained open-source logistic regression and random forest (RF) prediction algorithms were used to calculate a probability index (PI) for each infant for the occurrence of HIE. MAIN OUTCOME Validation of a machine learning algorithm to identify infants at risk of HIE in the immediate postnatal period. RESULTS 1081 had a complete data set available within 1 hour of birth: 76 (6.95%) with HIE and 1005 non-HIE. Of the 76 infants with HIE, 37 were classified as mild, 29 moderate and 10 severe. The best overall accuracy was seen with the RF model. Median (IQR) PI in the HIE group was 0.70 (0.53-0.86) vs 0.05 (0.02-0.15), (p<0.001) in the non-HIE group. The area under the receiver operating characteristics curve for prediction of HIE=0.926 (0.893-0.959, p<0.001). Using a PI cut-off to optimise sensitivity of 0.30, 936 of the 1081 (86.5%) infants were correctly classified. CONCLUSION In a large unseen data set an open-source algorithm could identify infants at risk of HIE in the immediate postnatal period. This may aid focused clinical examination, transfer to tertiary care (if necessary) and timely intervention.
Collapse
Affiliation(s)
- Anne L Murray
- Cork University Maternity Hospital, Wilton, Cork, Ireland
- INFANT Centre, Paediatric Academic Unit, Cork University Hospital, Wilton, Cork, Ireland
| | - Daragh S O'Boyle
- INFANT Centre, Paediatric Academic Unit, Cork University Hospital, Wilton, Cork, Ireland
| | - Brian H Walsh
- Cork University Maternity Hospital, Wilton, Cork, Ireland
- INFANT Centre, Paediatric Academic Unit, Cork University Hospital, Wilton, Cork, Ireland
- Department of Paediatrics and Child Health, University College Cork, Cork, Ireland
| | - Deirdre M Murray
- INFANT Centre, Paediatric Academic Unit, Cork University Hospital, Wilton, Cork, Ireland
- Department of Paediatrics and Child Health, University College Cork, Cork, Ireland
| |
Collapse
|
12
|
Koga R, Koide S, Tanaka H, Taguchi K, Kugler M, Yokota T, Ohshima K, Miyoshi H, Nagaishi M, Hashimoto N, Takeuchi I, Hontani H. A study of criteria for grading follicular lymphoma using a cell type classifier from pathology images based on complementary-label learning. Micron 2024; 184:103663. [PMID: 38843576 DOI: 10.1016/j.micron.2024.103663] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2024] [Revised: 05/21/2024] [Accepted: 05/21/2024] [Indexed: 06/30/2024]
Abstract
We propose a criterion for grading follicular lymphoma that is consistent with the intuitive evaluation, which is conducted by experienced pathologists. A criterion for grading follicular lymphoma is defined by the World Health Organization (WHO) based on the number of centroblasts and centrocytes within the field of view. However, the WHO criterion is not often used in clinical practice because it is impractical for pathologists to visually identify the cell type of each cell and count the number of centroblasts and centrocytes. Hence, based on the widespread use of digital pathology, we make it practical to identify and count the cell type by using image processing and then construct a criterion for grading based on the number of cells. Here, the problem is that labeling the cell type is not easy even for experienced pathologists. To alleviate this problem, we build a new dataset for cell type classification, which contains the pathologists' confusion records during labeling, and we construct the cell type classifier using complementary-label learning from this dataset. Then we propose a criterion based on the composition ratio of cell types that is consistent with the pathologists' grading. Our experiments demonstrate that the classifier can accurately identify cell types and the proposed criterion is more consistent with the pathologists' grading than the current WHO criterion.
Collapse
Affiliation(s)
- Ryoichi Koga
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan
| | - Shingo Koide
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan
| | - Hiromu Tanaka
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan
| | - Kei Taguchi
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan
| | - Mauricio Kugler
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan
| | - Tatsuya Yokota
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan
| | - Koichi Ohshima
- Department of Pathology, 67 Asahi-cho, Kurume-shi, Fukuoka 830-0011, Japan
| | - Hiroaki Miyoshi
- Department of Pathology, 67 Asahi-cho, Kurume-shi, Fukuoka 830-0011, Japan
| | - Miharu Nagaishi
- Department of Pathology, 67 Asahi-cho, Kurume-shi, Fukuoka 830-0011, Japan
| | - Noriaki Hashimoto
- RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan
| | - Ichiro Takeuchi
- RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan; Department of Mechanical Systems Engineering, Furo-sho, Chikusa-ku, Nagoya-shi Aichi 464-8601, Japan
| | - Hidekata Hontani
- Dapartment of Computer Science, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi 466-8555, Japan.
| |
Collapse
|
13
|
Li J, Dong P, Wang X, Zhang J, Zhao M, Shen H, Cai L, He J, Han M, Miao J, Liu H, Yang W, Han X, Liu Y. Artificial intelligence enhances whole-slide interpretation of PD-L1 CPS in triple-negative breast cancer: A multi-institutional ring study. Histopathology 2024; 85:451-467. [PMID: 38747491 DOI: 10.1111/his.15205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 03/11/2024] [Accepted: 04/21/2024] [Indexed: 08/09/2024]
Abstract
BACKGROUND AND AIMS Evaluation of the programmed cell death ligand-1 (PD-L1) combined positive score (CPS) is vital to predict the efficacy of the immunotherapy in triple-negative breast cancer (TNBC), but pathologists show substantial variability in the consistency and accuracy of the interpretation. It is of great importance to establish an objective and effective method which is highly repeatable. METHODS We proposed a model in a deep learning-based framework, which at the patch level incorporated cell analysis and tissue region analysis, followed by the whole-slide level fusion of patch results. Three rounds of ring studies (RSs) were conducted. Twenty-one pathologists of different levels from four institutions evaluated the PD-L1 CPS in TNBC specimens as continuous scores by visual assessment and our artificial intelligence (AI)-assisted method. RESULTS In the visual assessment, the interpretation results of PD-L1 (Dako 22C3) CPS by different levels of pathologists have significant differences and showed weak consistency. Using AI-assisted interpretation, there were no significant differences between all pathologists (P = 0.43), and the intraclass correlation coefficient (ICC) value was increased from 0.618 [95% confidence interval (CI) = 0.524-0.719] to 0.931 (95% CI = 0.902-0.955). The accuracy of interpretation result is further improved to 0.919 (95% CI = 0.886-0.947). Acceptance of AI results by junior pathologists was the highest among all levels, and 80% of the AI results were accepted overall. CONCLUSION With the help of the AI-assisted diagnostic method, different levels of pathologists achieved excellent consistency and repeatability in the interpretation of PD-L1 (Dako 22C3) CPS. Our AI-assisted diagnostic approach was proved to strengthen the consistency and repeatability in clinical practice.
Collapse
Affiliation(s)
- Jinze Li
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Pei Dong
- AI Lab, Tencent, Shenzhen, Guangdong, China
| | - Xinran Wang
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Jun Zhang
- AI Lab, Tencent, Shenzhen, Guangdong, China
| | - Meng Zhao
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | | | - Lijing Cai
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Jiankun He
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Mengxue Han
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Jiaxian Miao
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Hongbo Liu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| | - Wei Yang
- AI Lab, Tencent, Shenzhen, Guangdong, China
| | - Xiao Han
- AI Lab, Tencent, Shenzhen, Guangdong, China
| | - Yueping Liu
- Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei, China
| |
Collapse
|
14
|
Zhang DY, Venkat A, Khasawneh H, Sali R, Zhang V, Pei Z. Implementation of Digital Pathology and Artificial Intelligence in Routine Pathology Practice. J Transl Med 2024; 104:102111. [PMID: 39053633 DOI: 10.1016/j.labinv.2024.102111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Revised: 07/07/2024] [Accepted: 07/17/2024] [Indexed: 07/27/2024] Open
Abstract
The advent of affordable technology has significantly influenced the practice of digital pathology, leading to its growing adoption within the pathology community. This review article aimed to outline the latest developments in digital pathology, the cutting-edge advancements in artificial intelligence (AI) applications within this field, and the pertinent United States regulatory frameworks. The content is based on a thorough analysis of original research articles and official United States Federal guidelines. Findings from our review indicate that several Food and Drug Administration-approved digital scanners and image management systems are establishing a solid foundation for the seamless integration of advanced technologies into everyday pathology workflows, which may reduce device and operational costs in the future. AI is particularly transforming the way morphologic diagnoses are automated, notably in cancers like prostate and colorectal, within screening initiatives, albeit challenges such as data privacy issues and algorithmic biases remain. The regulatory environment, shaped by standards from the Food and Drug Administration, Centers for Medicare & Medicaid Services/Clinical Laboratory Improvement Amendments, and College of American Pathologists, is evolving to accommodate these innovations while ensuring safety and reliability. Centers for Medicare & Medicaid Services/Clinical Laboratory Improvement Amendments have issued policies to allow pathologists to review and render diagnoses using digital pathology remotely. Moreover, the introduction of new digital pathology Current Procedural Terminology codes designed to complement existing pathology Current Procedural Terminology codes is facilitating reimbursement processes. Overall, these advancements are heralding a new era in pathology that promises enhanced diagnostic precision and efficiency through digital and AI technologies, potentially improving patient care as well as bolstering educational and research activities.
Collapse
Affiliation(s)
- David Y Zhang
- Department of Computation, NovinoAI, Fort Lauderdale, Florida; Department of Veterans Affairs New York Harbor Healthcare System, New York, New York.
| | - Arsha Venkat
- School of Medicine, New York Medical College, New York, New York
| | - Hamdi Khasawneh
- King Hussein School of Computing Sciences, Princess Sumaya University for Technology, Amman, Jordan
| | - Rasoul Sali
- Department of Computation, NovinoAI, Fort Lauderdale, Florida; Department of Radiation Oncology, Stanford University School of Medicine, Stanford, California
| | - Valerio Zhang
- Department of Computation, NovinoAI, Fort Lauderdale, Florida
| | - Zhiheng Pei
- Department of Veterans Affairs New York Harbor Healthcare System, New York, New York; Department of Pathology, New York University School of Medicine, New York, New York; Department of Medicine, New York University School of Medicine, New York, New York.
| |
Collapse
|
15
|
Laohawetwanit T, Gonzalez RS, Bychkov A. Learning at a distance: results of an international survey on the adoption of virtual conferences and whole slide imaging by pathologists. J Clin Pathol 2024; 77:632-638. [PMID: 37208158 DOI: 10.1136/jcp-2023-208912] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 05/08/2023] [Indexed: 05/21/2023]
Abstract
AIMS This study presents the findings of a global survey of pathologists' views of online conferences and digital pathology. METHODS An online anonymous survey consisting of 11 questions focusing on pathologists' perceptions of virtual conferences and digital slides was distributed to practising pathologists and trainees across the globe using the authors' social media accounts and professional society connections. Participants were asked to rank their preference for various aspects of pathology meetings on a 5-point Likert scale. RESULTS There were 562 respondents from 79 countries. Several advantages of virtual meetings were recognised, including that they are less expensive to attend than in-person meetings (mean 4.4), more convenient to attend remotely (mean 4.3) and more efficient due to no loss of time for travel (mean 4.3). The lack of networking was reported as the main disadvantage of virtual conferences (mean 4.0). Most respondents (n=450, 80.1%) preferred hybrid or virtual meetings. About two-thirds (n=356, 63.3%) had no concern regarding the use of virtual slides for educational purposes and viewed them as an acceptable substitute for glass slides. CONCLUSIONS Online meetings and whole slide imaging are viewed as valuable tools in pathology education. Virtual conferences allow affordable registration fees and flexibility for participants. However, networking opportunities are limited, meaning in-person meetings cannot be entirely replaced by virtual conferences. Hybrid meetings may be a solution to maximise the benefits of both virtual and in-person meetings.
Collapse
Affiliation(s)
- Thiyaphat Laohawetwanit
- Division of Pathology, Chulabhorn International College of Medicine, Thammasat University, Pathum Thani, Thailand
| | - Raul S Gonzalez
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia, USA
| | - Andrey Bychkov
- Department of Pathology, Kameda Medical Center, Kamogawa, Chiba, Japan
| |
Collapse
|
16
|
Hölscher DL, Bülow RD. Decoding pathology: the role of computational pathology in research and diagnostics. Pflugers Arch 2024:10.1007/s00424-024-03002-2. [PMID: 39095655 DOI: 10.1007/s00424-024-03002-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2024] [Revised: 07/25/2024] [Accepted: 07/25/2024] [Indexed: 08/04/2024]
Abstract
Traditional histopathology, characterized by manual quantifications and assessments, faces challenges such as low-throughput and inter-observer variability that hinder the introduction of precision medicine in pathology diagnostics and research. The advent of digital pathology allowed the introduction of computational pathology, a discipline that leverages computational methods, especially based on deep learning (DL) techniques, to analyze histopathology specimens. A growing body of research shows impressive performances of DL-based models in pathology for a multitude of tasks, such as mutation prediction, large-scale pathomics analyses, or prognosis prediction. New approaches integrate multimodal data sources and increasingly rely on multi-purpose foundation models. This review provides an introductory overview of advancements in computational pathology and discusses their implications for the future of histopathology in research and diagnostics.
Collapse
Affiliation(s)
- David L Hölscher
- Department for Nephrology and Clinical Immunology, RWTH Aachen University Hospital, Pauwelsstraße 30, 52074, Aachen, Germany
- Institute for Pathology, RWTH Aachen University Hospital, Pauwelsstraße 30, 52074, Aachen, Germany
| | - Roman D Bülow
- Institute for Pathology, RWTH Aachen University Hospital, Pauwelsstraße 30, 52074, Aachen, Germany.
| |
Collapse
|
17
|
Schukow CP, Allen TC. Digital and Computational Pathology Are Pathologists' Physician Extenders. Arch Pathol Lab Med 2024; 148:866-870. [PMID: 38531382 DOI: 10.5858/arpa.2023-0537-ed] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/11/2024] [Indexed: 03/28/2024]
Affiliation(s)
- Casey P Schukow
- From the Department of Pathology, Corewell Health's Beaumont Hospital, Royal Oak, Michigan
| | - Timothy Craig Allen
- From the Department of Pathology, Corewell Health's Beaumont Hospital, Royal Oak, Michigan
| |
Collapse
|
18
|
Vorontsov E, Bozkurt A, Casson A, Shaikovski G, Zelechowski M, Severson K, Zimmermann E, Hall J, Tenenholtz N, Fusi N, Yang E, Mathieu P, van Eck A, Lee D, Viret J, Robert E, Wang YK, Kunz JD, Lee MCH, Bernhard JH, Godrich RA, Oakley G, Millar E, Hanna M, Wen H, Retamero JA, Moye WA, Yousfi R, Kanan C, Klimstra DS, Rothrock B, Liu S, Fuchs TJ. A foundation model for clinical-grade computational pathology and rare cancers detection. Nat Med 2024:10.1038/s41591-024-03141-0. [PMID: 39039250 DOI: 10.1038/s41591-024-03141-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Accepted: 06/19/2024] [Indexed: 07/24/2024]
Abstract
The analysis of histopathology images with artificial intelligence aims to enable clinical decision support systems and precision medicine. The success of such applications depends on the ability to model the diverse patterns observed in pathology images. To this end, we present Virchow, the largest foundation model for computational pathology to date. In addition to the evaluation of biomarker prediction and cell identification, we demonstrate that a large foundation model enables pan-cancer detection, achieving 0.95 specimen-level area under the (receiver operating characteristic) curve across nine common and seven rare cancers. Furthermore, we show that with less training data, the pan-cancer detector built on Virchow can achieve similar performance to tissue-specific clinical-grade models in production and outperform them on some rare variants of cancer. Virchow's performance gains highlight the value of a foundation model and open possibilities for many high-impact applications with limited amounts of labeled training data.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | | | - Ellen Yang
- Memorial Sloan Kettering Cancer Center, New York, NY, US
| | | | | | | | | | | | | | | | | | | | | | | | - Ewan Millar
- NSW Health Pathology, St George Hospital, Sydney, New South Wales, Australia
| | - Matthew Hanna
- Memorial Sloan Kettering Cancer Center, New York, NY, US
| | - Hannah Wen
- Memorial Sloan Kettering Cancer Center, New York, NY, US
| | | | | | | | | | | | | | | | | |
Collapse
|
19
|
Li Q, Zhang X, Zhang J, Huang H, Li L, Guo C, Li W, Guo Y. Deep learning-based hyperspectral technique identifies metastatic lymph nodes in oral squamous cell carcinoma-A pilot study. Oral Dis 2024. [PMID: 39005220 DOI: 10.1111/odi.15067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 05/31/2024] [Accepted: 07/02/2024] [Indexed: 07/16/2024]
Abstract
AIMS To establish a system based on hyperspectral imaging and deep learning for the detection of cancer cells in metastatic lymph nodes. MAIN METHODS The continuous sections of metastatic lymph nodes from 45 oral squamous cell carcinoma (OSCC) patients were collected. An improved ResUNet algorithm was established for deep learning to analyze the spectral curve differences between cancer cells and lymphocytes, and that between tumor tissue and normal tissue. KEY FINDINGS It was found that cancer cells, lymphocytes, and erythrocytes in the metastatic lymph nodes could be distinguished basing hyperspectral image, with overall accuracy (OA) as 87.30% and average accuracy (AA) as 85.46%. Cancerous area could be recognized by hyperspectral image and deep learning, and the average intersection over union (IOU) and accuracy were 0.6253 and 0.7692, respectively. SIGNIFICANCE This study indicated that deep learning-based hyperspectral techniques can identify tumor tissue in OSCC metastatic lymph nodes, achieving high accuracy of pathological diagnosis, high work efficiency, and reducing work burden. But these are preliminary results limited to a small sample.
Collapse
Affiliation(s)
- Qingxiang Li
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Xueyu Zhang
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Jianyun Zhang
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
- Department of Oral Pathology, Peking University School and Hospital of Stomatology, Beijing, China
| | - Hongyuan Huang
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Liangliang Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Chuanbin Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Wei Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Yuxing Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| |
Collapse
|
20
|
Dominguez-Morales JP, Duran-Lopez L, Marini N, Vicente-Diaz S, Linares-Barranco A, Atzori M, Müller H. A systematic comparison of deep learning methods for Gleason grading and scoring. Med Image Anal 2024; 95:103191. [PMID: 38728903 DOI: 10.1016/j.media.2024.103191] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 01/16/2024] [Accepted: 05/02/2024] [Indexed: 05/12/2024]
Abstract
Prostate cancer is the second most frequent cancer in men worldwide after lung cancer. Its diagnosis is based on the identification of the Gleason score that evaluates the abnormality of cells in glands through the analysis of the different Gleason patterns within tissue samples. The recent advancements in computational pathology, a domain aiming at developing algorithms to automatically analyze digitized histopathology images, lead to a large variety and availability of datasets and algorithms for Gleason grading and scoring. However, there is no clear consensus on which methods are best suited for each problem in relation to the characteristics of data and labels. This paper provides a systematic comparison on nine datasets with state-of-the-art training approaches for deep neural networks (including fully-supervised learning, weakly-supervised learning, semi-supervised learning, Additive-MIL, Attention-Based MIL, Dual-Stream MIL, TransMIL and CLAM) applied to Gleason grading and scoring tasks. The nine datasets are collected from pathology institutes and openly accessible repositories. The results show that the best methods for Gleason grading and Gleason scoring tasks are fully supervised learning and CLAM, respectively, guiding researchers to the best practice to adopt depending on the task to solve and the labels that are available.
Collapse
Affiliation(s)
- Juan P Dominguez-Morales
- Robotics and Technology of Computers Lab., ETSII-EPS, Universidad de Sevilla, Sevilla 41012, Spain; SCORE Lab, I3US. Universidad de Sevilla, Spain.
| | - Lourdes Duran-Lopez
- Robotics and Technology of Computers Lab., ETSII-EPS, Universidad de Sevilla, Sevilla 41012, Spain; SCORE Lab, I3US. Universidad de Sevilla, Spain
| | - Niccolò Marini
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Technopôle 3, Sierre 3960, Switzerland; Centre Universitaire d'Informatique, University of Geneva, Carouge 1227, Switzerland
| | - Saturnino Vicente-Diaz
- Robotics and Technology of Computers Lab., ETSII-EPS, Universidad de Sevilla, Sevilla 41012, Spain; SCORE Lab, I3US. Universidad de Sevilla, Spain
| | - Alejandro Linares-Barranco
- Robotics and Technology of Computers Lab., ETSII-EPS, Universidad de Sevilla, Sevilla 41012, Spain; SCORE Lab, I3US. Universidad de Sevilla, Spain
| | - Manfredo Atzori
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Technopôle 3, Sierre 3960, Switzerland; Department of Neuroscience, University of Padua, Via Giustiniani 2, Padua, 35128, Italy
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Technopôle 3, Sierre 3960, Switzerland; Medical faculty, University of Geneva, Geneva 1211, Switzerland
| |
Collapse
|
21
|
Liu S, Amgad M, More D, Rathore MA, Salgado R, Cooper LAD. A panoptic segmentation dataset and deep-learning approach for explainable scoring of tumor-infiltrating lymphocytes. NPJ Breast Cancer 2024; 10:52. [PMID: 38942745 PMCID: PMC11213912 DOI: 10.1038/s41523-024-00663-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Accepted: 06/17/2024] [Indexed: 06/30/2024] Open
Abstract
Tumor-Infiltrating Lymphocytes (TILs) have strong prognostic and predictive value in breast cancer, but their visual assessment is subjective. To improve reproducibility, the International Immuno-oncology Working Group recently released recommendations for the computational assessment of TILs that build on visual scoring guidelines. However, existing resources do not adequately address these recommendations due to the lack of annotation datasets that enable joint, panoptic segmentation of tissue regions and cells. Moreover, existing deep-learning methods focus entirely on either tissue segmentation or cell nuclei detection, which complicates the process of TILs assessment by necessitating the use of multiple models and reconciling inconsistent predictions. We introduce PanopTILs, a region and cell-level annotation dataset containing 814,886 nuclei from 151 patients, openly accessible at: sites.google.com/view/panoptils . Using PanopTILs we developed MuTILs, a neural network optimized for assessing TILs in accordance with clinical recommendations. MuTILs is a concept bottleneck model designed to be interpretable and to encourage sensible predictions at multiple resolutions. Using a rigorous internal-external cross-validation procedure, MuTILs achieves an AUROC of 0.93 for lymphocyte detection and a DICE coefficient of 0.81 for tumor-associated stroma segmentation. Our computational score closely matched visual scores from 2 pathologists (Spearman R = 0.58-0.61, p < 0.001). Moreover, computational TILs scores had a higher prognostic value than visual scores, independent of TNM stage and patient age. In conclusion, we introduce a comprehensive open data resource and a modeling approach for detailed mapping of the breast tumor microenvironment.
Collapse
Affiliation(s)
- Shangke Liu
- Department of Pathology, Northwestern University, Chicago, IL, USA
| | - Mohamed Amgad
- Department of Pathology, Northwestern University, Chicago, IL, USA.
| | - Deeptej More
- Department of Pathology, Northwestern University, Chicago, IL, USA
| | | | - Roberto Salgado
- Department of Pathology, GZA-ZNA Ziekenhuizen, Antwerp, Belgium
- Division of Research, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia
| | - Lee A D Cooper
- Department of Pathology, Northwestern University, Chicago, IL, USA.
| |
Collapse
|
22
|
Poalelungi DG, Neagu AI, Fulga A, Neagu M, Tutunaru D, Nechita A, Fulga I. Revolutionizing Pathology with Artificial Intelligence: Innovations in Immunohistochemistry. J Pers Med 2024; 14:693. [PMID: 39063947 PMCID: PMC11278211 DOI: 10.3390/jpm14070693] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 06/25/2024] [Accepted: 06/26/2024] [Indexed: 07/28/2024] Open
Abstract
Artificial intelligence (AI) is a reality of our times, and it has been successfully implemented in all fields, including medicine. As a relatively new domain, all efforts are directed towards creating algorithms applicable in most medical specialties. Pathology, as one of the most important areas of interest for precision medicine, has received significant attention in the development and implementation of AI algorithms. This focus is especially important for achieving accurate diagnoses. Moreover, immunohistochemistry (IHC) serves as a complementary diagnostic tool in pathology. It can be further augmented through the application of deep learning (DL) and machine learning (ML) algorithms for assessing and analyzing immunohistochemical markers. Such advancements can aid in delineating targeted therapeutic approaches and prognostic stratification. This article explores the applications and integration of various AI software programs and platforms used in immunohistochemical analysis. It concludes by highlighting the application of these technologies to pathologies such as breast, prostate, lung, melanocytic proliferations, and hematologic conditions. Additionally, it underscores the necessity for further innovative diagnostic algorithms to assist physicians in the diagnostic process.
Collapse
Affiliation(s)
- Diana Gina Poalelungi
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint Apostle Andrew Emergency County Clinical Hospital, 177 Brailei St., 800578 Galati, Romania
| | - Anca Iulia Neagu
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint John Clinical Emergency Hospital for Children, 800487 Galati, Romania
| | - Ana Fulga
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint Apostle Andrew Emergency County Clinical Hospital, 177 Brailei St., 800578 Galati, Romania
| | - Marius Neagu
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint Apostle Andrew Emergency County Clinical Hospital, 177 Brailei St., 800578 Galati, Romania
| | - Dana Tutunaru
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint Apostle Andrew Emergency County Clinical Hospital, 177 Brailei St., 800578 Galati, Romania
| | - Aurel Nechita
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint John Clinical Emergency Hospital for Children, 800487 Galati, Romania
| | - Iuliu Fulga
- Faculty of Medicine and Pharmacy, Dunarea de Jos University of Galati, 35 AI Cuza St., 800010 Galati, Romania; (D.G.P.); (M.N.); (D.T.); (A.N.); (I.F.)
- Saint Apostle Andrew Emergency County Clinical Hospital, 177 Brailei St., 800578 Galati, Romania
| |
Collapse
|
23
|
Chang J, Hatfield B. Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond. Adv Cancer Res 2024; 161:431-478. [PMID: 39032956 DOI: 10.1016/bs.acr.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/23/2024]
Abstract
The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.
Collapse
Affiliation(s)
- Justin Chang
- Virginia Commonwealth University Health System, Richmond, VA, United States
| | - Bryce Hatfield
- Virginia Commonwealth University Health System, Richmond, VA, United States.
| |
Collapse
|
24
|
Furlanello C, Bussola N, Merzi N, Pievani Trapletti G, Cadei M, Del Sordo R, Sidoni A, Ricci C, Lanzarotto F, Parigi TL, Villanacci V. The development of artificial intelligence in the histological diagnosis of Inflammatory Bowel Disease (IBD-AI). Dig Liver Dis 2024:S1590-8658(24)00791-6. [PMID: 38853093 DOI: 10.1016/j.dld.2024.05.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 05/12/2024] [Accepted: 05/28/2024] [Indexed: 06/11/2024]
Abstract
BACKGROUND Inflammatory bowel disease (IBD) includes Crohn's Disease (CD) and Ulcerative Colitis (UC). Correct diagnosis requires the identification of precise morphological features such basal plasmacytosis. However, histopathological interpretation can be challenging, and it is subject to high variability. AIM The IBD-Artificial Intelligence (AI) project aims at the development of an AI-based evaluation system to support the diagnosis of IBD, semi-automatically quantifying basal plasmacytosis. METHODS A deep learning model was trained to detect and quantify plasma cells on a public dataset of 4981 annotated images. The model was then tested on an external validation cohort of 356 intestinal biopsies of CD, UC and healthy controls. AI diagnostic performance was calculated compared to human gold standard. RESULTS The system correctly found that CD and UC samples had a greater prevalence of basal plasma cells with mean number of PCs within ROIs of 38.22 (95 % CI: 31.73, 49.04) for CD, 55.16 (46.57, 65.93) for UC, and 17.25 (CI: 12.17, 27.05) for controls. Overall, OR=4.968 (CI: 1.835, 14.638) was found for IBD compared to normal mucosa (CD: +59 %; UC: +129 %). Additionally, as expected, UC samples were found to have more plasma cells in colon than CD cases. CONCLUSION Our model accurately replicated human assessment of basal plasmacytosis, underscoring the value of AI models as a potential aid IBD diagnosis.
Collapse
Affiliation(s)
| | | | | | | | - Moris Cadei
- Institute of Pathology, ASST Spedali Civili and University of Brescia, Brescia, Italy
| | - Rachele Del Sordo
- Department of Medicine and Surgery, Section of Anatomic Pathology and Histology, Medical School, University of Perugia, Perugia, Italy
| | - Angelo Sidoni
- Department of Medicine and Surgery, Section of Anatomic Pathology and Histology, Medical School, University of Perugia, Perugia, Italy
| | - Chiara Ricci
- Gastroenterology Unit, Clinical and Experimental Sciences Department, Spedali Civili Hospital, University of Brescia, Brescia, Italy
| | - Francesco Lanzarotto
- Gastroenterology Unit, Clinical and Experimental Sciences Department, Spedali Civili Hospital, University of Brescia, Brescia, Italy
| | - Tommaso Lorenzo Parigi
- Division of Immunology, Transplantation and Infectious Disease, University Vita-Salute San Raffaele, Milan, Italy
| | - Vincenzo Villanacci
- Institute of Pathology, ASST Spedali Civili and University of Brescia, Brescia, Italy.
| |
Collapse
|
25
|
Hartman DJ. Applications of Artificial Intelligence in Lung Pathology. Surg Pathol Clin 2024; 17:321-328. [PMID: 38692814 DOI: 10.1016/j.path.2023.11.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/03/2024]
Abstract
Artificial intelligence/machine learning tools are being created for use in pathology. Some examples related to lung pathology include acid-fast stain evaluation, programmed death ligand-1 (PDL-1) interpretation, evaluating histologic patterns of non-small-cell lung carcinoma, evaluating histologic features in mesothelioma associated with adverse outcomes, predicting response to anti-PDL-1 therapy from hematoxylin and eosin-stained slides, evaluation of tumor microenvironment, evaluating patterns of interstitial lung disease, nondestructive methods for tissue evaluation, and others. There are still some frameworks (regulatory, workflow, and payment) that need to be established for these tools to be integrated into pathology.
Collapse
Affiliation(s)
- Douglas J Hartman
- University of Pittsburgh Medical Center, 200 Lothrop Street C-620, Pittsburgh, PA 15213, USA.
| |
Collapse
|
26
|
Ivanova M, Pescia C, Trapani D, Venetis K, Frascarelli C, Mane E, Cursano G, Sajjadi E, Scatena C, Cerbelli B, d’Amati G, Porta FM, Guerini-Rocco E, Criscitiello C, Curigliano G, Fusco N. Early Breast Cancer Risk Assessment: Integrating Histopathology with Artificial Intelligence. Cancers (Basel) 2024; 16:1981. [PMID: 38893102 PMCID: PMC11171409 DOI: 10.3390/cancers16111981] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Revised: 05/13/2024] [Accepted: 05/17/2024] [Indexed: 06/21/2024] Open
Abstract
Effective risk assessment in early breast cancer is essential for informed clinical decision-making, yet consensus on defining risk categories remains challenging. This paper explores evolving approaches in risk stratification, encompassing histopathological, immunohistochemical, and molecular biomarkers alongside cutting-edge artificial intelligence (AI) techniques. Leveraging machine learning, deep learning, and convolutional neural networks, AI is reshaping predictive algorithms for recurrence risk, thereby revolutionizing diagnostic accuracy and treatment planning. Beyond detection, AI applications extend to histological subtyping, grading, lymph node assessment, and molecular feature identification, fostering personalized therapy decisions. With rising cancer rates, it is crucial to implement AI to accelerate breakthroughs in clinical practice, benefiting both patients and healthcare providers. However, it is important to recognize that while AI offers powerful automation and analysis tools, it lacks the nuanced understanding, clinical context, and ethical considerations inherent to human pathologists in patient care. Hence, the successful integration of AI into clinical practice demands collaborative efforts between medical experts and computational pathologists to optimize patient outcomes.
Collapse
Affiliation(s)
- Mariia Ivanova
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Carlo Pescia
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Dario Trapani
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, 20141 Milan, Italy; (D.T.); (C.C.); (G.C.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Konstantinos Venetis
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Chiara Frascarelli
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Eltjona Mane
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Giulia Cursano
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Elham Sajjadi
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Cristian Scatena
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56126 Pisa, Italy;
| | - Bruna Cerbelli
- Department of Medical-Surgical Sciences and Biotechnologies, Sapienza University of Rome, 00185 Rome, Italy;
| | - Giulia d’Amati
- Department of Radiological, Oncological and Pathological Sciences, Sapienza University of Rome, 00185 Rome, Italy;
| | - Francesca Maria Porta
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
| | - Elena Guerini-Rocco
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Carmen Criscitiello
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, 20141 Milan, Italy; (D.T.); (C.C.); (G.C.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Giuseppe Curigliano
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology IRCCS, 20141 Milan, Italy; (D.T.); (C.C.); (G.C.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Nicola Fusco
- Division of Pathology, European Institute of Oncology IRCCS, 20141 Milan, Italy; (M.I.); (C.P.); (K.V.); (C.F.); (E.M.); (G.C.); (E.S.); (F.M.P.); (E.G.-R.)
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| |
Collapse
|
27
|
Bui DC, Song B, Kim K, Kwak JT. DAX-Net: A dual-branch dual-task adaptive cross-weight feature fusion network for robust multi-class cancer classification in pathology images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 248:108112. [PMID: 38479146 DOI: 10.1016/j.cmpb.2024.108112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 02/15/2024] [Accepted: 03/01/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND AND OBJECTIVE Multi-class cancer classification has been extensively studied in digital and computational pathology due to its importance in clinical decision-making. Numerous computational tools have been proposed for various types of cancer classification. Many of them are built based on convolutional neural networks. Recently, Transformer-style networks have shown to be effective for cancer classification. Herein, we present a hybrid design that leverages both convolutional neural networks and transformer architecture to obtain superior performance in cancer classification. METHODS We propose a dual-branch dual-task adaptive cross-weight feature fusion network, called DAX-Net, which exploits heterogeneous feature representations from the convolutional neural network and Transformer network, adaptively combines them to boost their representation power, and conducts cancer classification as categorical classification and ordinal classification. For an efficient and effective optimization of the proposed model, we introduce two loss functions that are tailored to the two classification tasks. RESULTS To evaluate the proposed method, we employed colorectal and prostate cancer datasets, of which each contains both in-domain and out-of-domain test sets. For colorectal cancer, the proposed method obtained an accuracy of 88.4%, a quadratic kappa score of 0.945, and an F1 score of 0.831 for the in-domain test set, and 84.4%, 0.910, and 0.768 for the out-of-domain test set. For prostate cancer, it achieved an accuracy of 71.6%, a kappa score of 0.635, and an F1 score of 0.655 for the in-domain test set, 79.2% accuracy, 0.721 kappa score, and 0.686 F1 score for the first out-of-domain test set, and 58.1% accuracy, 0.564 kappa score, and 0.493 F1 score for the second out-of-domain test set. It is worth noting that the performance of the proposed method outperformed other competitors by significant margins, in particular, with respect to the out-of-domain test sets. CONCLUSIONS The experimental results demonstrate that the proposed method is not only accurate but also robust to varying conditions of the test sets in comparison to several, related methods. These results suggest that the proposed method can facilitate automated cancer classification in various clinical settings.
Collapse
Affiliation(s)
- Doanh C Bui
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Boram Song
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Kyungeun Kim
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Jin Tae Kwak
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea.
| |
Collapse
|
28
|
Jain E, Patel A, Parwani AV, Shafi S, Brar Z, Sharma S, Mohanty SK. Whole Slide Imaging Technology and Its Applications: Current and Emerging Perspectives. Int J Surg Pathol 2024; 32:433-448. [PMID: 37437093 DOI: 10.1177/10668969231185089] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/14/2023]
Abstract
Background. Whole slide imaging (WSI) represents a paradigm shift in pathology, serving as a necessary first step for a wide array of digital tools to enter the field. It utilizes virtual microscopy wherein glass slides are converted into digital slides and are viewed by pathologists by automated image analysis. Its impact on pathology workflow, reproducibility, dissemination of educational material, expansion of service to underprivileged areas, and institutional collaboration exemplifies a significant innovative movement. The recent US Food and Drug Administration approval to WSI for its use in primary surgical pathology diagnosis has opened opportunities for wider application of this technology in routine practice. Main Text. The ongoing technological advances in digital scanners, image visualization methods, and the integration of artificial intelligence-derived algorithms with these systems provide avenues to exploit its applications. Its benefits are innumerable such as ease of access through the internet, avoidance of physical storage space, and no risk of deterioration of staining quality or breakage of slides to name a few. Although the benefits of WSI to pathology practices are many, the complexities of implementation remain an obstacle to widespread adoption. Some barriers including the high cost, technical glitches, and most importantly professional hesitation to adopt a new technology have hindered its use in routine pathology. Conclusions. In this review, we summarize the technical aspects of WSI, its applications in diagnostic pathology, training, and research along with future perspectives. It also highlights improved understanding of the current challenges to implementation, as well as the benefits and successes of the technology. WSI provides a golden opportunity for pathologists to guide its evolution, standardization, and implementation to better acquaint them with the key aspects of this technology and its judicial use. Also, implementation of routine digital pathology is an extra step requiring resources which (currently) does not usually result increased efficiency or payment.
Collapse
Affiliation(s)
- Ekta Jain
- Department of Pathology and Laboratory Medicine, CORE Diagnostics, Gurgaon, India
| | - Ankush Patel
- Department of Pathology, Wexner Medical Center, Columbus, OH, USA
| | - Anil V Parwani
- Department of Pathology, Wexner Medical Center, Columbus, OH, USA
| | - Saba Shafi
- Department of Pathology, Wexner Medical Center, Columbus, OH, USA
| | - Zoya Brar
- Department of Pathology and Laboratory Medicine, CORE Diagnostics, Gurgaon, India
| | - Shivani Sharma
- Department of Pathology and Laboratory Medicine, CORE Diagnostics, Gurgaon, India
| | - Sambit K Mohanty
- Department of Pathology and Laboratory Medicine, CORE Diagnostics, Gurgaon, India
| |
Collapse
|
29
|
van Diest PJ, Flach RN, van Dooijeweert C, Makineli S, Breimer GE, Stathonikos N, Pham P, Nguyen TQ, Veta M. Pros and cons of artificial intelligence implementation in diagnostic pathology. Histopathology 2024; 84:924-934. [PMID: 38433288 DOI: 10.1111/his.15153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 12/29/2023] [Accepted: 01/19/2024] [Indexed: 03/05/2024]
Abstract
The rapid introduction of digital pathology has greatly facilitated development of artificial intelligence (AI) models in pathology that have shown great promise in assisting morphological diagnostics and quantitation of therapeutic targets. We are now at a tipping point where companies have started to bring algorithms to the market, and questions arise whether the pathology community is ready to implement AI in routine workflow. However, concerns also arise about the use of AI in pathology. This article reviews the pros and cons of introducing AI in diagnostic pathology.
Collapse
Affiliation(s)
- Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Rachel N Flach
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Oncological Urology, University Medical Center Utrecht, Utrecht, the Netherlands
| | | | - Seher Makineli
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Surgical Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Gerben E Breimer
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Nikolas Stathonikos
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Paul Pham
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Tri Q Nguyen
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Mitko Veta
- Department of Oncological Urology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| |
Collapse
|
30
|
Tobe Y, Robertson AM, Ramezanpour M, Cebral JR, Watkins SC, Charbel FT, Amin-Hanjani S, Yu AK, Cheng BC, Woo HH. Comapping Cellular Content and Extracellular Matrix with Hemodynamics in Intact Arterial Tissues Using Scanning Immunofluorescent Multiphoton Microscopy. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2024; 30:342-358. [PMID: 38525887 PMCID: PMC11057816 DOI: 10.1093/mam/ozae025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 03/01/2024] [Accepted: 03/04/2024] [Indexed: 03/26/2024]
Abstract
Deviation of blood flow from an optimal range is known to be associated with the initiation and progression of vascular pathologies. Important open questions remain about how the abnormal flow drives specific wall changes in pathologies such as cerebral aneurysms where the flow is highly heterogeneous and complex. This knowledge gap precludes the clinical use of readily available flow data to predict outcomes and improve treatment of these diseases. As both flow and the pathological wall changes are spatially heterogeneous, a crucial requirement for progress in this area is a methodology for acquiring and comapping local vascular wall biology data with local hemodynamic data. Here, we developed an imaging pipeline to address this pressing need. A protocol that employs scanning multiphoton microscopy was developed to obtain three-dimensional (3D) datasets for smooth muscle actin, collagen, and elastin in intact vascular specimens. A cluster analysis was introduced to objectively categorize the smooth muscle cells (SMC) across the vascular specimen based on SMC actin density. Finally, direct quantitative comparison of local flow and wall biology in 3D intact specimens was achieved by comapping both heterogeneous SMC data and wall thickness to patient-specific hemodynamic results.
Collapse
Affiliation(s)
- Yasutaka Tobe
- Department of Mechanical Engineering and Materials Science, University of Pittsburgh, Pittsburgh, PA 15261, USA
| | - Anne M Robertson
- Department of Mechanical Engineering and Materials Science, University of Pittsburgh, Pittsburgh, PA 15261, USA
| | - Mehdi Ramezanpour
- Department of Mechanical Engineering and Materials Science, University of Pittsburgh, Pittsburgh, PA 15261, USA
| | - Juan R Cebral
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA
| | - Simon C Watkins
- Department of Cell Biology, University of Pittsburgh, PA 15261, USA
| | - Fady T Charbel
- Department of Neurosurgery, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Sepideh Amin-Hanjani
- Department of Neurological Surgery, University Hospital Cleveland Medical Center, Cleveland, OH 44106, USA
| | - Alexander K Yu
- Department of Neurological Surgery, Allegheny Health Network, Pittsburgh, PA 15212, USA
| | - Boyle C Cheng
- Neuroscience and Orthopedic Institutes, Allegheny Health Network, Pittsburgh, PA 15212, USA
| | - Henry H Woo
- Department of Neurosurgery, Donald and Barbara Zucker School of Medicine at Hofstra Northwell, Manhasset, NY 11549, USA
| |
Collapse
|
31
|
Peng J, Xu Z, Dan H, Li J, Wang J, Luo X, Xu H, Zeng X, Chen Q. Oral epithelial dysplasia detection and grading in oral leukoplakia using deep learning. BMC Oral Health 2024; 24:434. [PMID: 38594651 PMCID: PMC11005210 DOI: 10.1186/s12903-024-04191-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Accepted: 03/27/2024] [Indexed: 04/11/2024] Open
Abstract
BACKGROUND The grading of oral epithelial dysplasia is often time-consuming for oral pathologists and the results are poorly reproducible between observers. In this study, we aimed to establish an objective, accurate and useful detection and grading system for oral epithelial dysplasia in the whole-slides of oral leukoplakia. METHODS Four convolutional neural networks were compared using the image patches from 56 whole-slide of oral leukoplakia labeled by pathologists as the gold standard. Sequentially, feature detection models were trained, validated and tested with 1,000 image patches using the optimal network. Lastly, a comprehensive system named E-MOD-plus was established by combining feature detection models and a multiclass logistic model. RESULTS EfficientNet-B0 was selected as the optimal network to build feature detection models. In the internal dataset of whole-slide images, the prediction accuracy of E-MOD-plus was 81.3% (95% confidence interval: 71.4-90.5%) and the area under the receiver operating characteristic curve was 0.793 (95% confidence interval: 0.650 to 0.925); in the external dataset of 229 tissue microarray images, the prediction accuracy was 86.5% (95% confidence interval: 82.4-90.0%) and the area under the receiver operating characteristic curve was 0.669 (95% confidence interval: 0.496 to 0.843). CONCLUSIONS E-MOD-plus was objective and accurate in the detection of pathological features as well as the grading of oral epithelial dysplasia, and had potential to assist pathologists in clinical practice.
Collapse
Affiliation(s)
- Jiakuan Peng
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
- Department of Stomatology, North Sichuan Medical College, Nanchong, Sichuan, China
| | - Ziang Xu
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Hongxia Dan
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Jing Li
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Jiongke Wang
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Xiaobo Luo
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China
| | - Hao Xu
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China.
| | - Xin Zeng
- State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral Diseases, Research Unit of Oral Carcinogenesis and Management, Chinese Academy of Medical Sciences, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan, China.
| | - Qianming Chen
- Key Laboratory of Oral Biomedical Research of Zhejiang Province, Affiliated Stomatology Hospital, Zhejiang University School of Stomatology, Hangzhou, Zhejiang, China
| |
Collapse
|
32
|
Liang M, Jiang X, Cao J, Zhang S, Liu H, Li B, Wang L, Zhang C, Jia X. HSG-MGAF Net: Heterogeneous subgraph-guided multiscale graph attention fusion network for interpretable prediction of whole-slide image. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 247:108099. [PMID: 38442623 DOI: 10.1016/j.cmpb.2024.108099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/17/2023] [Revised: 02/12/2024] [Accepted: 02/22/2024] [Indexed: 03/07/2024]
Abstract
BACKGROUND AND OBJECTIVE Pathological whole slide image (WSI) prediction and region of interest (ROI) localization are important issues in computer-aided diagnosis and postoperative analysis in clinical applications. Existing computer-aided methods for predicting WSI are mainly based on multiple instance learning (MIL) and its variants. However, most of the methods are based on instance independence and identical distribution assumption and performed at a single scale, which not fully exploit the hierarchical multiscale heterogeneous information contained in WSI. METHODS Heterogeneous Subgraph-Guided Multiscale Graph Attention Fusion Network (HSG-MGAF Net) is proposed to build the topology of critical image patches at two scales for adaptive WSI prediction and lesion localization. The HSG-MGAF Net simulates the hierarchical heterogeneous information of WSI through graph and hypergraph at two scales, respectively. This framework not only fully exploits the low-order and potential high-order correlations of image patches at each scale, but also leverages the heterogeneous information of the two scales for adaptive WSI prediction. RESULTS We validate the superiority of the proposed method on the CAMELYON16 and the TCGA- NSCLC, and the results show that HSG-MGAF Net outperforms the state-of-the-art method on both datasets. The average ACC, AUC and F1 score of HSG-MGAF Net can reach 92.7 %/0.951/0.892 and 92.2 %/0.957/0.919, respectively. The obtained heatmaps can also localize the positive regions more accurately, which have great consistency with the pixel-level labels. CONCLUSIONS The results demonstrate that HSG-MGAF Net outperforms existing weakly supervised learning methods by introducing critical heterogeneous information between the two scales. This approach paves the way for further research on light weighted heterogeneous graph-based WSI prediction and ROI localization.
Collapse
Affiliation(s)
- Meiyan Liang
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan 030006, China.
| | - Xing Jiang
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan 030006, China
| | - Jie Cao
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China.
| | - Shupeng Zhang
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan 030006, China
| | - Haishun Liu
- Department of Automation, Tsinghua University, Beijing 100084, China
| | - Bo Li
- Department of Rehabilitation Treatment, Shanxi Rongjun Hospital, Taiyuan 030000, China
| | - Lin Wang
- Department of Pathology, Shanxi Bethune Hospital, Shanxi Academy of Medical Sciences, Tongji Shanxi Hospital, Third Hospital of Shanxi Medical University, Taiyuan 030032, China
| | - Cunlin Zhang
- Department of physics, Capital Normal University, Beijing 100048, China
| | - Xiaojun Jia
- School of Physics and Electronic Engineering, Shanxi University, Taiyuan 030006, China
| |
Collapse
|
33
|
Sajithkumar A, Thomas J, Saji AM, Ali F, E K HH, Adampulan HAG, Sarathchand S. Artificial Intelligence in pathology: current applications, limitations, and future directions. Ir J Med Sci 2024; 193:1117-1121. [PMID: 37542634 DOI: 10.1007/s11845-023-03479-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Accepted: 07/26/2023] [Indexed: 08/07/2023]
Abstract
PURPOSE Given AI's recent success in computer vision applications, majority of pathologists anticipate that it will be able to assist them with a variety of digital pathology activities. Massive improvements in deep learning have enabled a synergy between Artificial Intelligence (AI) and deep learning, enabling image-based diagnosis against the backdrop of digital pathology. AI-based solutions are being developed to eliminate errors and save pathologists time. AIMS In this paper, we will discuss the components that went into the use of Artificial Intelligence in Pathology, its use in the medical profession, the obstacles and constraints that it encounters, and the future possibilities of AI in the medical field. CONCLUSIONS Based on these factors, we elaborate upon the use of AI in medical pathology and provide future recommendations for its successful implementation in this field.
Collapse
Affiliation(s)
- Akhil Sajithkumar
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India.
| | - Jubin Thomas
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Ajish Meprathumalil Saji
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Fousiya Ali
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Haneena Hasin E K
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Hannan Abdul Gafoor Adampulan
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Swathy Sarathchand
- Sree Narayana Institute of Medical Sciences, Chalakka - Kuthiathode Rd, North Kuthiathode, Kunnukara, Kerala, 683594, India
| |
Collapse
|
34
|
Kim PJ, Hwang HS, Choi G, Sung HJ, Ahn B, Uh JS, Yoon S, Kim D, Chun SM, Jang SJ, Go H. A new model using deep learning to predict recurrence after surgical resection of lung adenocarcinoma. Sci Rep 2024; 14:6366. [PMID: 38493247 PMCID: PMC10944489 DOI: 10.1038/s41598-024-56867-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 03/12/2024] [Indexed: 03/18/2024] Open
Abstract
This study aimed to develop a deep learning (DL) model for predicting the recurrence risk of lung adenocarcinoma (LUAD) based on its histopathological features. Clinicopathological data and whole slide images from 164 LUAD cases were collected and used to train DL models with an ImageNet pre-trained efficientnet-b2 architecture, densenet201, and resnet152. The models were trained to classify each image patch into high-risk or low-risk groups, and the case-level result was determined by multiple instance learning with final FC layer's features from a model from all patches. Analysis of the clinicopathological and genetic characteristics of the model-based risk group was performed. For predicting recurrence, the model had an area under the curve score of 0.763 with 0.750, 0.633 and 0.680 of sensitivity, specificity, and accuracy in the test set, respectively. High-risk cases for recurrence predicted by the model (HR group) were significantly associated with shorter recurrence-free survival and a higher stage (both, p < 0.001). The HR group was associated with specific histopathological features such as poorly differentiated components, complex glandular pattern components, tumor spread through air spaces, and a higher grade. In the HR group, pleural invasion, necrosis, and lymphatic invasion were more frequent, and the size of the invasion was larger (all, p < 0.001). Several genetic mutations, including TP53 (p = 0.007) mutations, were more frequently found in the HR group. The results of stages I-II were similar to those of the general cohort. DL-based model can predict the recurrence risk of LUAD and identify the presence of the TP53 gene mutation by analyzing histopathologic features.
Collapse
Affiliation(s)
- Pil-Jong Kim
- School of Dentistry and Dental Research Institute, Seoul National University, Seoul, Republic of Korea
| | - Hee Sang Hwang
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Gyuheon Choi
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Hyun-Jung Sung
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Bokyung Ahn
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Ji-Su Uh
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Shinkyo Yoon
- Department of Oncology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Deokhoon Kim
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Sung-Min Chun
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Se Jin Jang
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Heounjeong Go
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
35
|
Hölscher DL, Goedertier M, Klinkhammer BM, Droste P, Costa IG, Boor P, Bülow RD. tRigon: an R package and Shiny App for integrative (path-)omics data analysis. BMC Bioinformatics 2024; 25:98. [PMID: 38443821 PMCID: PMC10916305 DOI: 10.1186/s12859-024-05721-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Accepted: 02/26/2024] [Indexed: 03/07/2024] Open
Abstract
BACKGROUND Pathomics facilitates automated, reproducible and precise histopathology analysis and morphological phenotyping. Similar to molecular omics, pathomics datasets are high-dimensional, but also face large outlier variability and inherent data missingness, making quick and comprehensible data analysis challenging. To facilitate pathomics data analysis and interpretation as well as support a broad implementation we developed tRigon (Toolbox foR InteGrative (path-)Omics data aNalysis), a Shiny application for fast, comprehensive and reproducible pathomics analysis. RESULTS tRigon is available via the CRAN repository ( https://cran.r-project.org/web/packages/tRigon ) with its source code available on GitLab ( https://git-ce.rwth-aachen.de/labooratory-ai/trigon ). The tRigon package can be installed locally and its application can be executed from the R console via the command 'tRigon::run_tRigon()'. Alternatively, the application is hosted online and can be accessed at https://labooratory.shinyapps.io/tRigon . We show fast computation of small, medium and large datasets in a low- and high-performance hardware setting, indicating broad applicability of tRigon. CONCLUSIONS tRigon allows researchers without coding abilities to perform exploratory feature analyses of pathomics and non-pathomics datasets on their own using a variety of hardware.
Collapse
Affiliation(s)
- David L Hölscher
- Institute of Pathology, RWTH Aachen University Clinic, Aachen, Germany
- Department of Nephrology and Immunology, RWTH Aachen University Clinic, Aachen, Germany
| | - Michael Goedertier
- Institute of Pathology, RWTH Aachen University Clinic, Aachen, Germany
- Institute for Computational Genomics, RWTH Aachen University Clinic, Aachen, Germany
| | | | - Patrick Droste
- Institute of Pathology, RWTH Aachen University Clinic, Aachen, Germany
- Department of Nephrology and Immunology, RWTH Aachen University Clinic, Aachen, Germany
| | - Ivan G Costa
- Institute for Computational Genomics, RWTH Aachen University Clinic, Aachen, Germany
| | - Peter Boor
- Institute of Pathology, RWTH Aachen University Clinic, Aachen, Germany
- Department of Nephrology and Immunology, RWTH Aachen University Clinic, Aachen, Germany
| | - Roman D Bülow
- Institute of Pathology, RWTH Aachen University Clinic, Aachen, Germany.
| |
Collapse
|
36
|
Rau TT, Cross W, Lastra RR, Lo RC, Matoso A, Herrington CS. Closing the loop - the role of pathologists in digital and computational pathology research. J Pathol Clin Res 2024; 10:e12366. [PMID: 38462794 PMCID: PMC10925822 DOI: 10.1002/2056-4538.12366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Accepted: 02/08/2024] [Indexed: 03/12/2024]
Abstract
An increasing number of manuscripts related to digital and computational pathology are being submitted to The Journal of Pathology: Clinical Research as part of the continuous evolution from digital imaging and algorithm-based digital pathology to computational pathology and artificial intelligence. However, despite these technological advances, tissue analysis still relies heavily on pathologists' annotations. There are three crucial elements to the pathologist's role during annotation tasks: granularity, time constraints, and responsibility for the interpretation of computational results. Granularity involves detailed annotations, including case level, regional, and cellular features; and integration of attributions from different sources. Time constraints due to pathologist shortages have led to the development of techniques to expedite annotation tasks from cell-level attributions up to so-called unsupervised learning. The impact of pathologists may seem diminished, but their role is crucial in providing ground truth and connecting pathological knowledge generation with computational advancements. Measures to display results back to pathologists and reflections about correctly applied diagnostic criteria are mandatory to maintain fidelity during human-machine interactions. Collaboration and iterative processes, such as human-in-the-loop machine learning are key for continuous improvement, ensuring the pathologist's involvement in evaluating computational results and closing the loop for clinical applicability. The journal is interested particularly in the clinical diagnostic application of computational pathology and invites submissions that address the issues raised in this editorial.
Collapse
|
37
|
Jahangir CA, Page DB, Broeckx G, Gonzalez CA, Burke C, Murphy C, Reis-Filho JS, Ly A, Harms PW, Gupta RR, Vieth M, Hida AI, Kahila M, Kos Z, van Diest PJ, Verbandt S, Thagaard J, Khiroya R, Abduljabbar K, Haab GA, Acs B, Adams S, Almeida JS, Alvarado-Cabrero I, Azmoudeh-Ardalan F, Badve S, Baharun NB, Bellolio ER, Bheemaraju V, Blenman KRM, Fujimoto LBM, Burgues O, Chardas A, Cheang MCU, Ciompi F, Cooper LAD, Coosemans A, Corredor G, Portela FLD, Deman F, Demaria S, Dudgeon SN, Elghazawy M, Fernandez-Martín C, Fineberg S, Fox SB, Giltnane JM, Gnjatic S, Gonzalez-Ericsson PI, Grigoriadis A, Halama N, Hanna MG, Harbhajanka A, Hart SN, Hartman J, Hewitt S, Horlings HM, Husain Z, Irshad S, Janssen EAM, Kataoka TR, Kawaguchi K, Khramtsov AI, Kiraz U, Kirtani P, Kodach LL, Korski K, Akturk G, Scott E, Kovács A, Lænkholm AV, Lang-Schwarz C, Larsimont D, Lennerz JK, Lerousseau M, Li X, Madabhushi A, Maley SK, Narasimhamurthy VM, Marks DK, McDonald ES, Mehrotra R, Michiels S, Kharidehal D, Minhas FUAA, Mittal S, Moore DA, Mushtaq S, Nighat H, Papathomas T, Penault-Llorca F, Perera RD, Pinard CJ, Pinto-Cardenas JC, Pruneri G, Pusztai L, Rajpoot NM, Rapoport BL, Rau TT, Ribeiro JM, Rimm D, Vincent-Salomon A, Saltz J, Sayed S, Hytopoulos E, Mahon S, Siziopikou KP, Sotiriou C, Stenzinger A, Sughayer MA, Sur D, Symmans F, Tanaka S, Taxter T, Tejpar S, Teuwen J, Thompson EA, Tramm T, Tran WT, van der Laak J, Verghese GE, Viale G, Wahab N, Walter T, Waumans Y, Wen HY, Yang W, Yuan Y, Bartlett J, Loibl S, Denkert C, Savas P, Loi S, Stovgaard ES, Salgado R, Gallagher WM, Rahman A. Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer. J Pathol 2024; 262:271-288. [PMID: 38230434 PMCID: PMC11288342 DOI: 10.1002/path.6238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 11/17/2023] [Indexed: 01/18/2024]
Abstract
Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high-throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi-marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Chowdhury Arif Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - David B Page
- Earle A Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA
| | - Glenn Broeckx
- Department of Pathology PA, GZA-ZNA Hospitals, Antwerp, Belgium
- Centre for Oncological Research (CORE), MIPPRO, Faculty of Medicine, Antwerp University, Antwerp, Belgium
| | - Claudia A Gonzalez
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Caoimbhe Burke
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Clodagh Murphy
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Jorge S Reis-Filho
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Amy Ly
- Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - Paul W Harms
- Departments of Pathology and Dermatology, University of Michigan, Ann Arbor, Ml, USA
| | - Rajarsi R Gupta
- Department of Biomedical informatics, Stony Brook University, Stony Brook, NY, USA
| | - Michael Vieth
- Institute of Pathology, Klinikum Bayreuth GmbH, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Akira I Hida
- Department of Pathology, Matsuyama Shimin Hospital, Matsuyama, Japan
| | - Mohamed Kahila
- Department of Pathology, Yale School of Medicine, New Haven, CT, USA
| | - Zuzana Kos
- Department of Pathology and Laboratory Medicine, University of British Columbia, BC Cancer, Vancouver, British Columbia, Canada
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
- Johns Hopkins Oncology Center, Baltimore, MD, USA
| | - Sara Verbandt
- Digestive Oncology, Department of Oncology, KU Leuven, Leuven, Belgium
| | - Jeppe Thagaard
- Technical University of Denmark, Kgs. Lyngby, Denmark
- Visiopharm A/S, Hørsholm, Denmark
| | - Reena Khiroya
- Department of Cellular Pathology, University College Hospital, London, UK
| | - Khalid Abduljabbar
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, UK
| | | | - Balazs Acs
- Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
- Department of Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
| | - Sylvia Adams
- Perlmutter Cancer Center, NYU Langone Health, New York, NY, USA
- Department of Medicine, NYU Grossman School of Medicine, Manhattan, NY, USA
| | - Jonas S Almeida
- Division of Cancer Epidemiology and Genetics (DCEG), National Cancer Institute (NCI), Rockville, MD, USA
| | | | | | - Sunil Badve
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Emory University Winship Cancer Institute, Atlanta, GA, USA
| | | | - Enrique R Bellolio
- Departamento de Anatomía Patológica, Facultad de Medicina, Universidad de La Frontera, Temuco, Chile
| | | | - Kim RM Blenman
- Department of internal Medicine Section of Medical Oncology and Yale Cancer Center, Yale School of Medicine, New Haven, CT, USA
- Department of Computer Science, Yale School of Engineering and Applied Science, New Haven, CT, USA
| | | | - Octavio Burgues
- Pathology Department, Hospital Cliníco Universitario de Valencia/lncliva, Valencia, Spain
| | - Alexandros Chardas
- Department of Pathobiology & Population Sciences, The Royal Veterinary College, London, UK
| | - Maggie Chon U Cheang
- Head of Integrative Genomics Analysis in Clinical Trials, ICR-CTSU, Division of Clinical Studies, The Institute of Cancer Research, London, UK
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Lee AD Cooper
- Department of Pathology, Northwestern Feinberg School of Medicine, Chicago, IL, USA
| | - An Coosemans
- Department of Oncology, Laboratory of Tumor Immunology and Immunotherapy, KU Leuven, Leuven, Belgium
| | - Germán Corredor
- Biomedical Engineering Department, Emory University, Atlanta, GA, USA
| | | | - Frederik Deman
- Department of Pathology PA, GZA-ZNA Hospitals, Antwerp, Belgium
| | - Sandra Demaria
- Department of Radiation Oncology, Weill Cornell Medical College, New York, NY, USA
- Department of Pathology, Weill Cornell Medicine, New York NY, USA
| | - Sarah N Dudgeon
- Computational Biology and Bioinformatics, Yale University, New Haven, CT, USA
| | - Mahmoud Elghazawy
- University of Surrey, Guildford, UK
- Ain Shams University, Cairo, Egypt
| | - Claudio Fernandez-Martín
- Institute Universitario de Investigatión en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain
| | - Susan Fineberg
- Montefiore Medical Center and the Albert Einstein College of Medicine, New York, NY, USA
| | - Stephen B Fox
- Pathology, Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria, Australia
| | | | - Sacha Gnjatic
- Department of Oncological Sciences, Medicine Hem/One, and Pathology, Tisch Cancer Institute – Precision Immunology Institute, Icahn School of Medicine at Mount Sinai, New York NY, USA
| | | | - Anita Grigoriadis
- Cancer Bioinformatics, Faculty of Life Sciences and Medicine, School of Cancer & Pharmaceutical Sciences, King’s College London, London, UK
- The Breast Cancer Now Research Unit Faculty of Life Sciences and Medicine, School of Cancer and Pharmaceutical Sciences, King’s College London, London, UK
| | - Niels Halama
- Department of Translational Immunotherapy, German Cancer Research Center, Heidelberg, Germany
| | | | | | - Steven N Hart
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Johan Hartman
- Tehran University of Medical Sciences, Tehran, Iran
- Department of Oncology-Pathology, Karolinska Institutet, Stockholm, Sweden
| | - Stephen Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Hugo M Horlings
- Division of Pathology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | | | - Sheeba Irshad
- King's College London & Guys & St Thomas NHS Trust London, UK
| | - Emiel AM Janssen
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Technology, University of Stavanger, Stavanger, Norway
| | | | - Kosuke Kawaguchi
- Department of Breast Surgery, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Andrey I Khramtsov
- Department of Pathology and Laboratory Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL, USA
| | - Umay Kiraz
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Technology, University of Stavanger, Stavanger, Norway
| | - Pawan Kirtani
- Histopathology, Aakash Healthcare Super Speciality Hospital, New Delhi, India
| | - Liudmila L Kodach
- Department of Pathology, Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - Konstanty Korski
- Data, Analytics and Imaging, Product Development, F. Hoffmann-La Roche AG, Basel, Switzerland
| | - Guray Akturk
- Translational Molecular Biomarkers, Merck & Co., Inc., Kenilworth, NJ, USA
| | - Ely Scott
- Translational Medicine, Bristol Myers Squibb, Princeton, NJ, USA
| | - Anikó Kovács
- Department of Clinical Pathology, Sahlgrenska University Hospital, Gothenburg, Sweden
- Institute of Biomedicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Anne-Vibeke Lænkholm
- Department of Surgical Pathology, Zealand University Hospital, Roskilde, Denmark
- Department of Surgical Pathology, University of Copenhagen, Copenhagen, Denmark
| | - Corinna Lang-Schwarz
- Institute of Pathology, Klinikum Bayreuth GmbH, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Denis Larsimont
- Institut Jules Bordet Université Libre de Bruxelles, Brussels, Belgium
| | - Jochen K Lennerz
- Center for Integrated Diagnostics, Massachusetts General Hospital/Harvard Medical School, Boston, MA, USA
| | - Marvin Lerousseau
- Centre for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France
- Institut Curie, PSL University, Paris, France
- INSERM U900, Paris, France
| | - Xiaoxian Li
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Radiology and Imaging Sciences, Biomedical Informatics, Pathology, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | - Sai K Maley
- NRG Oncology/NSABP Foundation, Pittsburgh, PA, USA
| | | | - Douglas K Marks
- Perlmutter Cancer Center, NYU Langone Health, New York, NY, USA
| | - Elizabeth S McDonald
- Breast Cancer Translational Research Group, University of Pennsylvania, Philadelphia, PA, USA
| | - Ravi Mehrotra
- Indian Cancer Genomic Atlas, Pune, India
- Centre for Health, Innovation and Policy Foundation, Noida, India
| | - Stefan Michiels
- Office of Biostatistics and Epidemiology, Gustave Roussy, Oncostat U1018, Inserm, University Paris-Saclay, Ligue Contre le Cancer labeled Team, Villejuif France
| | - Durga Kharidehal
- Department of Pathology, Narayana Medical College and Hospital, Nellore, India
| | - Fayyaz ul Amir Afsar Minhas
- Tissue Image Analytics Centre, Warwick Cancer Research Centre, PathLAKE Consortium, Department of Computer Science, University of Warwick, Coventry, UK
| | - Shachi Mittal
- Department of Chemical Engineering, Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
| | - David A Moore
- CRUK Lung Cancer Centre of Excellence, UCL and Cellular Pathology Department UCLH, London, UK
| | - Shamim Mushtaq
- Department of Biochemistry, Ziauddin University, Karachi, Pakistan
| | - Hussain Nighat
- Pathology and Laboratory Medicine, All India Institute of Medical Sciences, Raipur, India
| | - Thomas Papathomas
- Institute of Metabolism and Systems Research, University of Birmingham, Birmingham, UK
- Department of Clinical Pathology, Drammen Sykehus, Vestre Viken HF, Drammen, Norway
| | - Frederique Penault-Llorca
- Service de Pathologie et Biopathologie, Centre Jean PERRIN, INSERM U1240 Imagerie Moléculaire et Stratégies Théranostiques (IMoST), Université Clermont Auvergne, Clermont-Ferrand, France
| | - Rashindrie D Perera
- School of Electrical, Mechanical and Infrastructure Engineering, University of Melbourne, Melbourne, Victoria, Australia
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
| | - Christopher J Pinard
- Radiogenomics Laboratory, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
- Department of Clinical Studies, Ontario Veterinary College, University of Guelph, Guelph, Ontario, Canada
- Department of Oncology, Lakeshore Animal Health Partners, Mississauga, Ontario, Canada
- Centre for Advancing Responsible and Ethical Artificial Intelligence (CARE-AI), University of Guelph, Guelph, Ontario, Canada
| | | | - Giancarlo Pruneri
- Department of Pathology and Laboratory Medicine, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
- Faculty of Medicine and Surgery, University of Milan, Milan, Italy
| | - Lajos Pusztai
- Yale Cancer Center, Yale University, New Haven, CT, USA
- Department of Medical Oncology, Yale School of Medicine, Yale University, New Haven, CT, USA
| | | | - Bernardo Leon Rapoport
- The Medical Oncology Centre of Rosebank Johannesburg South Africa
- Department of Immunology, Faculty of Health Sciences, University of Pretoria, Pretoria, South Africa
| | - Tilman T Rau
- Institute of Pathology, University Hospital Düsseldorf and Heinrich-Heine-University, Düsseldorf Germany
| | | | - David Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
- Department of Medicine, Yale University School of Medicine, New Haven, CT, USA
| | - Anne Vincent-Salomon
- Department of Diagnostic and Theranostic Medicine, Institut Curie, University Paris-Sciences et Lettres, Paris, France
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook Medicine, New York NY, USA
| | - Shahin Sayed
- Department of Pathology, Aga Khan University, Nairobi, Kenya
| | - Evangelos Hytopoulos
- Department of Pathology, Aga Khan University, Nairobi, Kenya
- iRhythm Technologies Inc., San Francisco, CA, USA
| | - Sarah Mahon
- Mater Misericordiae University Hospital, Dublin, Ireland
| | - Kalliopi P Siziopikou
- Department of Pathology, Section of Breast Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory J.-C. Heuson, Institut Jules Bordet Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
- Medical Oncology Department Institut Jules Bordet Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Albrecht Stenzinger
- Institute of Pathology, University Hospital Heidelberg, Centers for Personalized Medicine (ZPM), Heidelberg, Germany
| | | | - Daniel Sur
- Department of Medical Oncology, University of Medicine and Pharmacy “luliu Hatieganu ”, Cluj-Napoca, Romania
| | - Fraser Symmans
- University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | | | - Sabine Tejpar
- Digestive Oncology, Department of Oncology, KU Leuven, Leuven, Belgium
| | - Jonas Teuwen
- Al for Oncology Lab, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Trine Tramm
- Department of Pathology, Institute of Clinical Medicine, Aarhus University Hospital, Aarhus, Denmark
| | - William T Tran
- Department of Radiation Oncology, University of Toronto and Sunnybrook Health Sciences Centre, Toronto, Canada
| | - Jeroen van der Laak
- Head of Integrative Genomics Analysis in Clinical Trials, ICR-CTSU, Division of Clinical Studies, The Institute of Cancer Research, London, UK
| | - Gregory E Verghese
- Cancer Bioinformatics, Faculty of Life Sciences and Medicine, School of Cancer & Pharmaceutical Sciences, King’s College London, London, UK
- The Breast Cancer Now Research Unit Faculty of Life Sciences and Medicine, School of Cancer and Pharmaceutical Sciences, King’s College London, London, UK
| | - Giuseppe Viale
- Department of Pathology, European Institute of Oncology & University of Milan, Milan, Italy
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Wanwick Coventry, UK
| | - Thomas Walter
- Centre for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France
- Institut Curie, PSL University, Paris, France
- INSERM U900, Paris, France
| | | | - Hannah Y Wen
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Wentao Yang
- Fudan Medical University Shanghai Cancer Center, Shanghai, PR China
| | - Yinyin Yuan
- Department of Translational Molecular Pathology, Division of Pathology and Laboratory Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | - Sibylle Loibl
- Department of Medicine and Research, German Breast Group, Neu-lsenburg Germany
| | - Carsten Denkert
- Institut für Pathologie, Philipps-Universität Marburg und Universitätsklinikum Marburg, Marburg, Germany
| | - Peter Savas
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
- The Sir Peter MacCallum Department of Medical Oncology, University of Melbourne, Melbourne, Victoria, Australia
| | - Sherene Loi
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
- Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, Victoria, Australia
| | | | - Roberto Salgado
- Department of Pathology PA, GZA-ZNA Hospitals, Antwerp, Belgium
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Arman Rahman
- UCD School of Medicine, UCD Conway Institute, University College Dublin, Dublin, Ireland
| |
Collapse
|
38
|
Piga I, Magni F, Smith A. The journey towards clinical adoption of MALDI-MS-based imaging proteomics: from current challenges to future expectations. FEBS Lett 2024; 598:621-634. [PMID: 38140823 DOI: 10.1002/1873-3468.14795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 12/06/2023] [Accepted: 12/09/2023] [Indexed: 12/24/2023]
Abstract
Among the spatial omics techniques available, mass spectrometry imaging (MSI) represents one of the most promising owing to its capability to map the distribution of hundreds of peptides and proteins, as well as other classes of biomolecules, within a complex sample background in a multiplexed and relatively high-throughput manner. In particular, matrix-assisted laser desorption/ionisation (MALDI-MSI) has come to the fore and established itself as the most widely used technique in clinical research. However, the march of this technique towards clinical utility has been hindered by issues related to method reproducibility, appropriate biocomputational tools, and data storage. Notwithstanding these challenges, significant progress has been achieved in recent years regarding multiple facets of the technology and has rendered it more suitable for a possible clinical role. As such, there is now more robust and extensive evidence to suggest that the technology has the potential to support clinical decision-making processes under appropriate circumstances. In this review, we will discuss some of the recent developments that have facilitated this progress and outline some of the more promising clinical proteomics applications which have been developed with a clear goal towards implementation in mind.
Collapse
Affiliation(s)
- Isabella Piga
- Department of Medicine and Surgery, Proteomics and Metabolomics Unit, University of Milano-Bicocca, Vedano al Lambro, Italy
| | - Fulvio Magni
- Department of Medicine and Surgery, Proteomics and Metabolomics Unit, University of Milano-Bicocca, Vedano al Lambro, Italy
| | - Andrew Smith
- Department of Medicine and Surgery, Proteomics and Metabolomics Unit, University of Milano-Bicocca, Vedano al Lambro, Italy
| |
Collapse
|
39
|
Jung M, Song SG, Cho SI, Shin S, Lee T, Jung W, Lee H, Park J, Song S, Park G, Song H, Park S, Lee J, Kang M, Park J, Pereira S, Yoo D, Chung K, Ali SM, Kim SW. Augmented interpretation of HER2, ER, and PR in breast cancer by artificial intelligence analyzer: enhancing interobserver agreement through a reader study of 201 cases. Breast Cancer Res 2024; 26:31. [PMID: 38395930 PMCID: PMC10885430 DOI: 10.1186/s13058-024-01784-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Accepted: 02/11/2024] [Indexed: 02/25/2024] Open
Abstract
BACKGROUND Accurate classification of breast cancer molecular subtypes is crucial in determining treatment strategies and predicting clinical outcomes. This classification largely depends on the assessment of human epidermal growth factor receptor 2 (HER2), estrogen receptor (ER), and progesterone receptor (PR) status. However, variability in interpretation among pathologists pose challenges to the accuracy of this classification. This study evaluates the role of artificial intelligence (AI) in enhancing the consistency of these evaluations. METHODS AI-powered HER2 and ER/PR analyzers, consisting of cell and tissue models, were developed using 1,259 HER2, 744 ER, and 466 PR-stained immunohistochemistry (IHC) whole-slide images of breast cancer. External validation cohort comprising HER2, ER, and PR IHCs of 201 breast cancer cases were analyzed with these AI-powered analyzers. Three board-certified pathologists independently assessed these cases without AI annotation. Then, cases with differing interpretations between pathologists and the AI analyzer were revisited with AI assistance, focusing on evaluating the influence of AI assistance on the concordance among pathologists during the revised evaluation compared to the initial assessment. RESULTS Reevaluation was required in 61 (30.3%), 42 (20.9%), and 80 (39.8%) of HER2, in 15 (7.5%), 17 (8.5%), and 11 (5.5%) of ER, and in 26 (12.9%), 24 (11.9%), and 28 (13.9%) of PR evaluations by the pathologists, respectively. Compared to initial interpretations, the assistance of AI led to a notable increase in the agreement among three pathologists on the status of HER2 (from 49.3 to 74.1%, p < 0.001), ER (from 93.0 to 96.5%, p = 0.096), and PR (from 84.6 to 91.5%, p = 0.006). This improvement was especially evident in cases of HER2 2+ and 1+, where the concordance significantly increased from 46.2 to 68.4% and from 26.5 to 70.7%, respectively. Consequently, a refinement in the classification of breast cancer molecular subtypes (from 58.2 to 78.6%, p < 0.001) was achieved with AI assistance. CONCLUSIONS This study underscores the significant role of AI analyzers in improving pathologists' concordance in the classification of breast cancer molecular subtypes.
Collapse
Affiliation(s)
- Minsun Jung
- Department of Pathology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Seung Geun Song
- Department of Pathology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - So-Woon Kim
- Department of Pathology, Kyung Hee University Hospital, Kyung Hee University College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
40
|
Nejat P, Alsaafin A, Alabtah G, Comfere NI, Mangold AR, Murphree DH, Zot P, Yasir S, Garcia JJ, Tizhoosh HR. Creating an atlas of normal tissue for pruning WSI patching through anomaly detection. Sci Rep 2024; 14:3932. [PMID: 38366094 PMCID: PMC10873359 DOI: 10.1038/s41598-024-54489-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 02/13/2024] [Indexed: 02/18/2024] Open
Abstract
Patching whole slide images (WSIs) is an important task in computational pathology. While most of them are designed to classify or detect the presence of pathological lesions in a WSI, the confounding role and redundant nature of normal histology are generally overlooked. In this paper, we propose and validate the concept of an "atlas of normal tissue" solely using samples of WSIs obtained from normal biopsies. Such atlases can be employed to eliminate normal fragments of tissue samples and hence increase the representativeness of the remaining patches. We tested our proposed method by establishing a normal atlas using 107 normal skin WSIs and demonstrated how established search engines like Yottixel can be improved. We used 553 WSIs of cutaneous squamous cell carcinoma to demonstrate the advantage. We also validated our method applied to an external dataset of 451 breast WSIs. The number of selected WSI patches was reduced by 30% to 50% after utilizing the proposed normal atlas while maintaining the same indexing and search performance in leave-one-patient-out validation for both datasets. We show that the proposed concept of establishing and using a normal atlas shows promise for unsupervised selection of the most representative patches of the abnormal WSI patches.
Collapse
Affiliation(s)
- Peyman Nejat
- KIMIA Lab, Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Areej Alsaafin
- KIMIA Lab, Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Ghazal Alabtah
- KIMIA Lab, Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
| | | | | | - Dennis H Murphree
- Department of Dermatology, Mayo Clinic, Rochester, MN, USA
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Patricija Zot
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Saba Yasir
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Joaquin J Garcia
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - H R Tizhoosh
- KIMIA Lab, Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
41
|
Greenberg A, Samueli B, Farkash S, Zohar Y, Ish-Shalom S, Hagege RR, Hershkovitz D. Algorithm-assisted diagnosis of Hirschsprung's disease - evaluation of robustness and comparative image analysis on data from various labs and slide scanners. Diagn Pathol 2024; 19:26. [PMID: 38321431 PMCID: PMC10845737 DOI: 10.1186/s13000-024-01452-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 01/25/2024] [Indexed: 02/08/2024] Open
Abstract
BACKGROUND Differences in the preparation, staining and scanning of digital pathology slides create significant pre-analytic variability. Algorithm-assisted tools must be able to contend with this variability in order to be applicable in clinical practice. In a previous study, a decision support algorithm was developed to assist in the diagnosis of Hirschsprung's disease. In the current study, we tested the robustness of this algorithm while assessing for pre-analytic factors which may affect its performance. METHODS The decision support algorithm was used on digital pathology slides obtained from four different medical centers (A-D) and scanned by three different scanner models (by Philips, Hamamatsu and 3DHISTECH). A total of 192 cases and 1782 slides were used in this study. RGB histograms were constructed to compare images from the various medical centers and scanner models and highlight the differences in color and contrast. RESULTS The algorithm was able to correctly identify ganglion cells in 99.2% of cases, from all medical centers (All scanned by the Philips slide scanner) as well as 95.5% and 100% of the slides scanned by the 3DHISTECH and Hamamatsu brand slide scanners, respectively. The total error rate for center D was lower than the other medical centers (3.9% vs 7.1%, 10.8% and 6% for centers A-C, respectively), the vast majority of errors being false positives (3.45% vs 0.45% false negatives). The other medical centers showed a higher rate of false negatives in relation to false positives (6.81% vs 0.29%, 9.8% vs 1.2% and 5.37% vs 0.63% for centers A-C, respectively). The total error rates for the Philips, Hamamatsu and 3DHISTECH brand scanners were 3.9%, 3.2% and 9.8%, respectively. RGB histograms demonstrated significant differences in pixel value distribution between the four medical centers, as well as between the 3DHISTECH brand scanner when compared to the Philips and Hamamatsu brand scanners. CONCLUSIONS The results reported in this paper suggest that the algorithm-based decision support system has sufficient robustness to be applicable for clinical practice. In addition, the novel method used in its development - Hierarchial-Contexual Analysis (HCA) may be applicable to the development of algorithm-assisted tools in other diseases, for which available datasets are limited. Validation of any given algorithm-assisted support system should nonetheless include data from as many medical centers and scanner models as possible.
Collapse
Affiliation(s)
- Ariel Greenberg
- Institute of Pathology, Tel-Aviv Sourasky Medical Center, 6 Weizmann Street, 6423906, Tel Aviv, Israel.
| | - Benzion Samueli
- Department of Pathology, Soroka University Medical Center, 76 Wingate Street, 8486614, Be'er Sheva, Israel
| | - Shai Farkash
- Department of Pathology, Emek Medical Center, Yitshak Rabin Boulevard 21, 1834111, Afula, Israel
| | - Yaniv Zohar
- Department of Pathology, Rambam Medical Center, 8 Haalia Hashnia, 3525408, Haifa, Israel
| | - Shahar Ish-Shalom
- Department of Pathology, Kaplan Medical Center, Pasternak St. P.O.B. 1, 76100, Rehovot, Israel
| | - Rami R Hagege
- Institute of Pathology, Tel-Aviv Sourasky Medical Center, 6 Weizmann Street, 6423906, Tel Aviv, Israel
| | - Dov Hershkovitz
- Institute of Pathology, Tel-Aviv Sourasky Medical Center, 6 Weizmann Street, 6423906, Tel Aviv, Israel
- Sackler Faculty of Medicine, Tel-Aviv University, Ramat Aviv 69978, Tel-Aviv, Israel
| |
Collapse
|
42
|
Brancato V, Esposito G, Coppola L, Cavaliere C, Mirabelli P, Scapicchio C, Borgheresi R, Neri E, Salvatore M, Aiello M. Standardizing digital biobanks: integrating imaging, genomic, and clinical data for precision medicine. J Transl Med 2024; 22:136. [PMID: 38317237 PMCID: PMC10845786 DOI: 10.1186/s12967-024-04891-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 01/14/2024] [Indexed: 02/07/2024] Open
Abstract
Advancements in data acquisition and computational methods are generating a large amount of heterogeneous biomedical data from diagnostic domains such as clinical imaging, pathology, and next-generation sequencing (NGS), which help characterize individual differences in patients. However, this information needs to be available and suitable to promote and support scientific research and technological development, supporting the effective adoption of the precision medicine approach in clinical practice. Digital biobanks can catalyze this process, facilitating the sharing of curated and standardized imaging data, clinical, pathological and molecular data, crucial to enable the development of a comprehensive and personalized data-driven diagnostic approach in disease management and fostering the development of computational predictive models. This work aims to frame this perspective, first by evaluating the state of standardization of individual diagnostic domains and then by identifying challenges and proposing a possible solution towards an integrative approach that can guarantee the suitability of information that can be shared through a digital biobank. Our analysis of the state of the art shows the presence and use of reference standards in biobanks and, generally, digital repositories for each specific domain. Despite this, standardization to guarantee the integration and reproducibility of the numerical descriptors generated by each domain, e.g. radiomic, pathomic and -omic features, is still an open challenge. Based on specific use cases and scenarios, an integration model, based on the JSON format, is proposed that can help address this problem. Ultimately, this work shows how, with specific standardization and promotion efforts, the digital biobank model can become an enabling technology for the comprehensive study of diseases and the effective development of data-driven technologies at the service of precision medicine.
Collapse
Affiliation(s)
| | - Giuseppina Esposito
- Bio Check Up S.R.L, 80121, Naples, Italy
- Department of Advanced Biomedical Sciences, University of Naples Federico II, 80131, Naples, Italy
| | | | | | - Peppino Mirabelli
- UOS Laboratori di Ricerca e Biobanca, AORN Santobono-Pausilipon, Via Teresa Ravaschieri, 8, 80122, Naples, Italy
| | - Camilla Scapicchio
- Academic Radiology, Department of Translational Research, University of Pisa, via Roma, 67, 56126, Pisa, Italy
| | - Rita Borgheresi
- Academic Radiology, Department of Translational Research, University of Pisa, via Roma, 67, 56126, Pisa, Italy
| | - Emanuele Neri
- Academic Radiology, Department of Translational Research, University of Pisa, via Roma, 67, 56126, Pisa, Italy
| | | | | |
Collapse
|
43
|
Ratziu V, Hompesch M, Petitjean M, Serdjebi C, Iyer JS, Parwani AV, Tai D, Bugianesi E, Cusi K, Friedman SL, Lawitz E, Romero-Gómez M, Schuppan D, Loomba R, Paradis V, Behling C, Sanyal AJ. Artificial intelligence-assisted digital pathology for non-alcoholic steatohepatitis: current status and future directions. J Hepatol 2024; 80:335-351. [PMID: 37879461 DOI: 10.1016/j.jhep.2023.10.015] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 08/28/2023] [Accepted: 10/09/2023] [Indexed: 10/27/2023]
Abstract
The worldwide prevalence of non-alcoholic steatohepatitis (NASH) is increasing, causing a significant medical burden, but no approved therapeutics are currently available. NASH drug development requires histological analysis of liver biopsies by expert pathologists for trial enrolment and efficacy assessment, which can be hindered by multiple issues including sample heterogeneity, inter-reader and intra-reader variability, and ordinal scoring systems. Consequently, there is a high unmet need for accurate, reproducible, quantitative, and automated methods to assist pathologists with histological analysis to improve the precision around treatment and efficacy assessment. Digital pathology (DP) workflows in combination with artificial intelligence (AI) have been established in other areas of medicine and are being actively investigated in NASH to assist pathologists in the evaluation and scoring of NASH histology. DP/AI models can be used to automatically detect, localise, quantify, and score histological parameters and have the potential to reduce the impact of scoring variability in NASH clinical trials. This narrative review provides an overview of DP/AI tools in development for NASH, highlights key regulatory considerations, and discusses how these advances may impact the future of NASH clinical management and drug development. This should be a high priority in the NASH field, particularly to improve the development of safe and effective therapeutics.
Collapse
Affiliation(s)
- Vlad Ratziu
- Sorbonne Université, ICAN Institute for Cardiometabolism and Nutrition, Hospital Pitié-Salpêtrière, INSERM UMRS 1138 CRC, Paris, France.
| | | | | | | | | | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | | | | | - Kenneth Cusi
- Division of Endocrinology, Diabetes and Metabolism, University of Florida, Gainesville, FL, USA
| | - Scott L Friedman
- Division of Liver Diseases, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Eric Lawitz
- Texas Liver Institute, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Manuel Romero-Gómez
- Hospital Universitario Virgen del Rocío, CiberEHD, Insituto de Biomedicina de Sevilla (HUVR/CSIC/US), Universidad de Sevilla, Seville, Spain
| | - Detlef Schuppan
- Institute of Translational Immunology and Department of Medicine, University Medical Center, Mainz, Germany; Department of Hepatology and Gastroenterology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Rohit Loomba
- NAFLD Research Center, University of California at San Diego, San Diego, CA, USA
| | - Valérie Paradis
- Université Paris Cité, Service d'Anatomie Pathologique, Hôpital Beaujon, Paris, France
| | | | - Arun J Sanyal
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| |
Collapse
|
44
|
Amgad M, Hodge JM, Elsebaie MAT, Bodelon C, Puvanesarajah S, Gutman DA, Siziopikou KP, Goldstein JA, Gaudet MM, Teras LR, Cooper LAD. A population-level digital histologic biomarker for enhanced prognosis of invasive breast cancer. Nat Med 2024; 30:85-97. [PMID: 38012314 DOI: 10.1038/s41591-023-02643-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 10/13/2023] [Indexed: 11/29/2023]
Abstract
Breast cancer is a heterogeneous disease with variable survival outcomes. Pathologists grade the microscopic appearance of breast tissue using the Nottingham criteria, which are qualitative and do not account for noncancerous elements within the tumor microenvironment. Here we present the Histomic Prognostic Signature (HiPS), a comprehensive, interpretable scoring of the survival risk incurred by breast tumor microenvironment morphology. HiPS uses deep learning to accurately map cellular and tissue structures to measure epithelial, stromal, immune, and spatial interaction features. It was developed using a population-level cohort from the Cancer Prevention Study-II and validated using data from three independent cohorts, including the Prostate, Lung, Colorectal, and Ovarian Cancer trial, Cancer Prevention Study-3, and The Cancer Genome Atlas. HiPS consistently outperformed pathologists in predicting survival outcomes, independent of tumor-node-metastasis stage and pertinent variables. This was largely driven by stromal and immune features. In conclusion, HiPS is a robustly validated biomarker to support pathologists and improve patient prognosis.
Collapse
Affiliation(s)
- Mohamed Amgad
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - James M Hodge
- Department of Population Science, American Cancer Society, Atlanta, GA, USA
| | - Maha A T Elsebaie
- Department of Medicine, John H. Stroger, Jr. Hospital of Cook County, Chicago, IL, USA
| | - Clara Bodelon
- Department of Population Science, American Cancer Society, Atlanta, GA, USA
| | | | - David A Gutman
- Department of Pathology, Emory University School of Medicine, Atlanta, GA, USA
| | - Kalliopi P Siziopikou
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Jeffery A Goldstein
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Mia M Gaudet
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD, USA
| | - Lauren R Teras
- Department of Population Science, American Cancer Society, Atlanta, GA, USA
| | - Lee A D Cooper
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
| |
Collapse
|
45
|
Kveton M, Hudec L, Vykopal I, Halinkovic M, Laco M, Felsoova A, Benesova W, Fabian O. Digital pathology in cardiac transplant diagnostics: from biopsies to algorithms. Cardiovasc Pathol 2024; 68:107587. [PMID: 37926351 DOI: 10.1016/j.carpath.2023.107587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 10/03/2023] [Accepted: 10/30/2023] [Indexed: 11/07/2023] Open
Abstract
In the field of heart transplantation, the ability to accurately and promptly diagnose cardiac allograft rejection is crucial. This comprehensive review explores the transformative role of digital pathology and computational pathology, especially through machine learning, in this critical domain. These methodologies harness large datasets to extract subtle patterns and valuable information that extend beyond human perceptual capabilities, potentially enhancing diagnostic outcomes. Current research indicates that these computer-based systems could offer accuracy and performance matching, or even exceeding, that of expert pathologists, thereby introducing more objectivity and reducing observer variability. Despite promising results, several challenges such as limited sample sizes, diverse data sources, and the absence of standardized protocols pose significant barriers to the widespread adoption of these techniques. The future of digital pathology in heart transplantation diagnostics depends on utilizing larger, more diverse patient cohorts, standardizing data collection, processing, and evaluation protocols, and fostering collaborative research efforts. The integration of various data types, including clinical, demographic, and imaging information, could further refine diagnostic precision. As researchers address these challenges and promote collaborative efforts, digital pathology has the potential to become an integral part of clinical practice, ultimately improving patient care in heart transplantation.
Collapse
Affiliation(s)
- Martin Kveton
- Third Faculty of Medicine, Charles University, Prague, Czech Republic; Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic.
| | - Lukas Hudec
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Ivan Vykopal
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Matej Halinkovic
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Miroslav Laco
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Andrea Felsoova
- Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic; Department of Histology and Embryology, Second Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Wanda Benesova
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Ondrej Fabian
- Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic; Department of Pathology and Molecular Medicine, Third Faculty of Medicine, Charles University and Thomayer Hospital, Prague, Czech Republic
| |
Collapse
|
46
|
Gallo M, Krajňanský V, Nenutil R, Holub P, Brázdil T. Shedding light on the black box of a neural network used to detect prostate cancer in whole slide images by occlusion-based explainability. N Biotechnol 2023; 78:52-67. [PMID: 37793603 DOI: 10.1016/j.nbt.2023.09.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 08/29/2023] [Accepted: 09/30/2023] [Indexed: 10/06/2023]
Abstract
Diagnostic histopathology faces increasing demands due to aging populations and expanding healthcare programs. Semi-automated diagnostic systems employing deep learning methods are one approach to alleviate this pressure. The learning models for histopathology are inherently complex and opaque from the user's perspective. Hence different methods have been developed to interpret their behavior. However, relatively limited attention has been devoted to the connection between interpretation methods and the knowledge of experienced pathologists. The main contribution of this paper is a method for comparing morphological patterns used by expert pathologists to detect cancer with the patterns identified as important for inference of learning models. Given the patch-based nature of processing large-scale histopathological imaging, we have been able to show statistically that the VGG16 model could utilize all the structures that are observable by the pathologist, given the patch size and scan resolution. The results show that the neural network approach to recognizing prostatic cancer is similar to that of a pathologist at medium optical resolution. The saliency maps identified several prevailing histomorphological features characterizing carcinoma, e.g., single-layered epithelium, small lumina, and hyperchromatic nuclei with halo. A convincing finding was the recognition of their mimickers in non-neoplastic tissue. The method can also identify differences, i.e., standard patterns not used by the learning models and new patterns not yet used by pathologists. Saliency maps provide added value for automated digital pathology to analyze and fine-tune deep learning systems and improve trust in computer-based decisions.
Collapse
Affiliation(s)
- Matej Gallo
- Faculty of Informatics, Masaryk University, Botanická 68a, 602 00 Brno, Czech Republic.
| | - Vojtěch Krajňanský
- Faculty of Informatics, Masaryk University, Botanická 68a, 602 00 Brno, Czech Republic
| | - Rudolf Nenutil
- Department of Pathology, Masaryk Memorial Cancer Institute, Žlutý kopec 7, 656 53 Brno, Czech Republic
| | - Petr Holub
- Institute of Computer Science, Masaryk University, Šumavská 416/15, 602 00 Brno, Czech Republic
| | - Tomáš Brázdil
- Faculty of Informatics, Masaryk University, Botanická 68a, 602 00 Brno, Czech Republic
| |
Collapse
|
47
|
Wibawa MS, Zhou JY, Wang R, Huang YY, Zhan Z, Chen X, Lv X, Young LS, Rajpoot N. AI-Based Risk Score from Tumour-Infiltrating Lymphocyte Predicts Locoregional-Free Survival in Nasopharyngeal Carcinoma. Cancers (Basel) 2023; 15:5789. [PMID: 38136336 PMCID: PMC10742296 DOI: 10.3390/cancers15245789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 11/28/2023] [Accepted: 12/08/2023] [Indexed: 12/24/2023] Open
Abstract
BACKGROUND Locoregional recurrence of nasopharyngeal carcinoma (NPC) occurs in 10% to 50% of cases following primary treatment. However, the current main prognostic markers for NPC, both stage and plasma Epstein-Barr virus DNA, are not sensitive to locoregional recurrence. METHODS We gathered 385 whole-slide images (WSIs) from haematoxylin and eosin (H&E)-stained NPC sections (n = 367 cases), which were collected from Sun Yat-sen University Cancer Centre. We developed a deep learning algorithm to detect tumour nuclei and lymphocyte nuclei in WSIs, followed by density-based clustering to quantify the tumour-infiltrating lymphocytes (TILs) into 12 scores. The Random Survival Forest model was then trained on the TILs to generate risk score. RESULTS Based on Kaplan-Meier analysis, the proposed methods were able to stratify low- and high-risk NPC cases in a validation set of locoregional recurrence with a statically significant result (p < 0.001). This finding was also found in distant metastasis-free survival (p < 0.001), progression-free survival (p < 0.001), and regional recurrence-free survival (p < 0.05). Furthermore, in both univariate analysis (HR: 1.58, CI: 1.13-2.19, p < 0.05) and multivariate analysis (HR:1.59, CI: 1.11-2.28, p < 0.05), we also found that our methods demonstrated a strong prognostic value for locoregional recurrence. CONCLUSION The proposed novel digital markers could potentially be utilised to assist treatment decisions in cases of NPC.
Collapse
Affiliation(s)
- Made Satria Wibawa
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK; (M.S.W.); (R.W.)
| | - Jia-Yu Zhou
- State Key Laboratory of Oncology in South China, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China; (J.-Y.Z.); (Y.-Y.H.); (Z.Z.); (X.C.); (X.L.)
- Department of Nasopharyngeal Carcinoma, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China
| | - Ruoyu Wang
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK; (M.S.W.); (R.W.)
| | - Ying-Ying Huang
- State Key Laboratory of Oncology in South China, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China; (J.-Y.Z.); (Y.-Y.H.); (Z.Z.); (X.C.); (X.L.)
- Department of Nasopharyngeal Carcinoma, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China
| | - Zejiang Zhan
- State Key Laboratory of Oncology in South China, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China; (J.-Y.Z.); (Y.-Y.H.); (Z.Z.); (X.C.); (X.L.)
- Department of Nasopharyngeal Carcinoma, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China
| | - Xi Chen
- State Key Laboratory of Oncology in South China, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China; (J.-Y.Z.); (Y.-Y.H.); (Z.Z.); (X.C.); (X.L.)
- Department of Nasopharyngeal Carcinoma, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China
| | - Xing Lv
- State Key Laboratory of Oncology in South China, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China; (J.-Y.Z.); (Y.-Y.H.); (Z.Z.); (X.C.); (X.L.)
- Department of Nasopharyngeal Carcinoma, Sun Yat-Sen University Cancer Center, Guangzhou 510060, China
| | - Lawrence S. Young
- Warwick Medical School, University of Warwick, Coventry CV4 7AL, UK;
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK; (M.S.W.); (R.W.)
- The Alan Turing Institute, London NW1 2DB, UK
| |
Collapse
|
48
|
Tavolara TE, Su Z, Gurcan MN, Niazi MKK. One label is all you need: Interpretable AI-enhanced histopathology for oncology. Semin Cancer Biol 2023; 97:70-85. [PMID: 37832751 DOI: 10.1016/j.semcancer.2023.09.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 09/06/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023]
Abstract
Artificial Intelligence (AI)-enhanced histopathology presents unprecedented opportunities to benefit oncology through interpretable methods that require only one overall label per hematoxylin and eosin (H&E) slide with no tissue-level annotations. We present a structured review of these methods organized by their degree of verifiability and by commonly recurring application areas in oncological characterization. First, we discuss morphological markers (tumor presence/absence, metastases, subtypes, grades) in which AI-identified regions of interest (ROIs) within whole slide images (WSIs) verifiably overlap with pathologist-identified ROIs. Second, we discuss molecular markers (gene expression, molecular subtyping) that are not verified via H&E but rather based on overlap with positive regions on adjacent tissue. Third, we discuss genetic markers (mutations, mutational burden, microsatellite instability, chromosomal instability) that current technologies cannot verify if AI methods spatially resolve specific genetic alterations. Fourth, we discuss the direct prediction of survival to which AI-identified histopathological features quantitatively correlate but are nonetheless not mechanistically verifiable. Finally, we discuss in detail several opportunities and challenges for these one-label-per-slide methods within oncology. Opportunities include reducing the cost of research and clinical care, reducing the workload of clinicians, personalized medicine, and unlocking the full potential of histopathology through new imaging-based biomarkers. Current challenges include explainability and interpretability, validation via adjacent tissue sections, reproducibility, data availability, computational needs, data requirements, domain adaptability, external validation, dataset imbalances, and finally commercialization and clinical potential. Ultimately, the relative ease and minimum upfront cost with which relevant data can be collected in addition to the plethora of available AI methods for outcome-driven analysis will surmount these current limitations and achieve the innumerable opportunities associated with AI-driven histopathology for the benefit of oncology.
Collapse
Affiliation(s)
- Thomas E Tavolara
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - Ziyu Su
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - Metin N Gurcan
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - M Khalid Khan Niazi
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA.
| |
Collapse
|
49
|
Pinto DG, Bychkov A, Tsuyama N, Fukuoka J, Eloy C. Real-World Implementation of Digital Pathology: Results From an Intercontinental Survey. J Transl Med 2023; 103:100261. [PMID: 37839634 DOI: 10.1016/j.labinv.2023.100261] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2022] [Revised: 09/29/2023] [Accepted: 10/07/2023] [Indexed: 10/17/2023] Open
Abstract
The past 70 years have been characterized by rapid advancements in computer technology, and the health care system has not been immune to this trend. However, anatomical pathology has remained largely an analog discipline. In recent years, this has been changing with the growing adoption of digital pathology, partly driven by the potential of computer-aided diagnosis. As part of an international collaboration, we conducted a comprehensive survey to gain a deeper understanding of the status of digital pathology implementation in Europe and Asia. A total of 127 anatomical pathology laboratories participated in the survey, including 75 from Europe and 52 from Asia, with 72 laboratories having established digital pathology workflow and 55 without digital pathology. Laboratories using digital pathology for diagnostic (n = 29) and nondiagnostic (n = 43) purposes were thoroughly questioned about their implementation strategies and institutional experiences, including details on equipment, storage, integration with laboratory information system, computer-aided diagnosis, and the costs of going digital. The impact of the digital pathology workflow was also evaluated, focusing on turnaround time, specimen traceability, quality control, and overall satisfaction. Laboratories without access to digital pathology were asked to provide insights into their perceptions of the technology, expectations, barriers to adoption, and potential facilitators. Our findings indicate that although digital pathology is still the future for many, it is already the present for some. This decade may be a time when anatomical pathology finally embraces digital revolution on a larger scale.
Collapse
Affiliation(s)
- Daniel Gomes Pinto
- Serviço de Anatomia Patológica, Hospital Garcia de Orta, EPE, Almada, Portugal; NOVA Medical School, Lisboa, Portugal; IPATIMUP - Instituto de Patologia e Imunologia Molecular da Universidade do Porto, Porto, Portugal
| | - Andrey Bychkov
- Department of Pathology, Kameda Medical Center, Kamogawa, Chiba, Japan
| | - Naoko Tsuyama
- Division of Pathology, Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
| | - Junya Fukuoka
- Department of Pathology, Kameda Medical Center, Kamogawa, Chiba, Japan; Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan
| | - Catarina Eloy
- IPATIMUP - Instituto de Patologia e Imunologia Molecular da Universidade do Porto, Porto, Portugal; Instituto de Investigação e Inovação Em Saúde (i3S) and Faculty of Medicine, University of Porto (FMUP), Porto, Portugal.
| |
Collapse
|
50
|
Waqas A, Bui MM, Glassy EF, El Naqa I, Borkowski P, Borkowski AA, Rasool G. Revolutionizing Digital Pathology With the Power of Generative Artificial Intelligence and Foundation Models. J Transl Med 2023; 103:100255. [PMID: 37757969 DOI: 10.1016/j.labinv.2023.100255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 09/06/2023] [Accepted: 09/19/2023] [Indexed: 09/29/2023] Open
Abstract
Digital pathology has transformed the traditional pathology practice of analyzing tissue under a microscope into a computer vision workflow. Whole-slide imaging allows pathologists to view and analyze microscopic images on a computer monitor, enabling computational pathology. By leveraging artificial intelligence (AI) and machine learning (ML), computational pathology has emerged as a promising field in recent years. Recently, task-specific AI/ML (eg, convolutional neural networks) has risen to the forefront, achieving above-human performance in many image-processing and computer vision tasks. The performance of task-specific AI/ML models depends on the availability of many annotated training datasets, which presents a rate-limiting factor for AI/ML development in pathology. Task-specific AI/ML models cannot benefit from multimodal data and lack generalization, eg, the AI models often struggle to generalize to new datasets or unseen variations in image acquisition, staining techniques, or tissue types. The 2020s are witnessing the rise of foundation models and generative AI. A foundation model is a large AI model trained using sizable data, which is later adapted (or fine-tuned) to perform different tasks using a modest amount of task-specific annotated data. These AI models provide in-context learning, can self-correct mistakes, and promptly adjust to user feedback. In this review, we provide a brief overview of recent advances in computational pathology enabled by task-specific AI, their challenges and limitations, and then introduce various foundation models. We propose to create a pathology-specific generative AI based on multimodal foundation models and present its potentially transformative role in digital pathology. We describe different use cases, delineating how it could serve as an expert companion of pathologists and help them efficiently and objectively perform routine laboratory tasks, including quantifying image analysis, generating pathology reports, diagnosis, and prognosis. We also outline the potential role that foundation models and generative AI can play in standardizing the pathology laboratory workflow, education, and training.
Collapse
Affiliation(s)
- Asim Waqas
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; Department of Electrical Engineering, University of South Florida, Tampa, Florida.
| | - Marilyn M Bui
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; Department of Pathology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; University of South Florida, Morsani College of Medicine, Tampa, Florida
| | - Eric F Glassy
- Affiliated Pathologists Medical Group, Inc., Rancho Dominguez, California
| | - Issam El Naqa
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida
| | - Piotr Borkowski
- Quest Diagnostics/Ameripath, Tampa, Florida; Center of Excellence for Digital and AI-Empowered Pathology, Quest Diagnostics, Tampa, Florida
| | - Andrew A Borkowski
- University of South Florida, Morsani College of Medicine, Tampa, Florida; James A. Haley Veterans' Hospital, Tampa, Florida; National Artificial Intelligence Institute, Washington, District of Columbia
| | - Ghulam Rasool
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; Department of Electrical Engineering, University of South Florida, Tampa, Florida; University of South Florida, Morsani College of Medicine, Tampa, Florida; Department of Neuro-Oncology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida
| |
Collapse
|