1
|
Tafavvoghi M, Bongo LA, Shvetsov N, Busund LTR, Møllersen K. Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review. J Pathol Inform 2024; 15:100363. [PMID: 38405160 PMCID: PMC10884505 DOI: 10.1016/j.jpi.2024.100363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 01/23/2024] [Indexed: 02/27/2024] Open
Abstract
Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.
Collapse
Affiliation(s)
- Masoud Tafavvoghi
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| | - Lars Ailo Bongo
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | - Nikita Shvetsov
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | | | - Kajsa Møllersen
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
2
|
Schmidt A, Morales-Alvarez P, Molina R. Probabilistic Attention Based on Gaussian Processes for Deep Multiple Instance Learning. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:10909-10922. [PMID: 37027623 DOI: 10.1109/tnnls.2023.3245329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Multiple instance learning (MIL) is a weakly supervised learning paradigm that is becoming increasingly popular because it requires less labeling effort than fully supervised methods. This is especially interesting for areas where the creation of large annotated datasets remains challenging, as in medicine. Although recent deep learning MIL approaches have obtained state-of-the-art results, they are fully deterministic and do not provide uncertainty estimations for the predictions. In this work, we introduce the attention Gaussian process (AGP) model, a novel probabilistic attention mechanism based on Gaussian processes (GPs) for deep MIL. AGP provides accurate bag-level predictions as well as instance-level explainability and can be trained end-to-end. Moreover, its probabilistic nature guarantees robustness to overfit on small datasets and uncertainty estimations for the predictions. The latter is especially important in medical applications, where decisions have a direct impact on the patient's health. The proposed model is validated experimentally as follows. First, its behavior is illustrated in two synthetic MIL experiments based on the well-known MNIST and CIFAR-10 datasets, respectively. Then, it is evaluated in three different real-world cancer detection experiments. AGP outperforms state-of-the-art MIL approaches, including deterministic deep learning ones. It shows a strong performance even on a small dataset with less than 100 labels and generalizes better than competing methods on an external test set. Moreover, we experimentally show that predictive uncertainty correlates with the risk of wrong predictions, and therefore it is a good indicator of reliability in practice. Our code is publicly available.
Collapse
|
3
|
Wei Y, Deng Y, Sun C, Lin M, Jiang H, Peng Y. Deep learning with noisy labels in medical prediction problems: a scoping review. J Am Med Inform Assoc 2024; 31:1596-1607. [PMID: 38814164 PMCID: PMC11187424 DOI: 10.1093/jamia/ocae108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Revised: 04/27/2024] [Accepted: 05/03/2024] [Indexed: 05/31/2024] Open
Abstract
OBJECTIVES Medical research faces substantial challenges from noisy labels attributed to factors like inter-expert variability and machine-extracted labels. Despite this, the adoption of label noise management remains limited, and label noise is largely ignored. To this end, there is a critical need to conduct a scoping review focusing on the problem space. This scoping review aims to comprehensively review label noise management in deep learning-based medical prediction problems, which includes label noise detection, label noise handling, and evaluation. Research involving label uncertainty is also included. METHODS Our scoping review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched 4 databases, including PubMed, IEEE Xplore, Google Scholar, and Semantic Scholar. Our search terms include "noisy label AND medical/healthcare/clinical," "uncertainty AND medical/healthcare/clinical," and "noise AND medical/healthcare/clinical." RESULTS A total of 60 papers met inclusion criteria between 2016 and 2023. A series of practical questions in medical research are investigated. These include the sources of label noise, the impact of label noise, the detection of label noise, label noise handling techniques, and their evaluation. Categorization of both label noise detection methods and handling techniques are provided. DISCUSSION From a methodological perspective, we observe that the medical community has been up to date with the broader deep-learning community, given that most techniques have been evaluated on medical data. We recommend considering label noise as a standard element in medical research, even if it is not dedicated to handling noisy labels. Initial experiments can start with easy-to-implement methods, such as noise-robust loss functions, weighting, and curriculum learning.
Collapse
Affiliation(s)
- Yishu Wei
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, United States
- Reddit Inc., San Francisco, CA 16093, United States
| | - Yu Deng
- Center for Health Information Partnerships, Northwestern University, Chicago, IL 10611, United States
| | - Cong Sun
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, United States
| | - Mingquan Lin
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, United States
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
| | - Hongmei Jiang
- Department of Statistics and Data Science, Northwestern University, Evanston, IL 60208, United States
| | - Yifan Peng
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, United States
| |
Collapse
|
4
|
Lalvani S, Katsaggelos A. Crowdsourcing with the drift diffusion model of decision making. Sci Rep 2024; 14:11311. [PMID: 38760397 DOI: 10.1038/s41598-024-61687-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Accepted: 05/08/2024] [Indexed: 05/19/2024] Open
Abstract
Crowdsourcing involves the use of annotated labels with unknown reliability to estimate ground truth labels in datasets. A common task in crowdsourcing involves estimating reliabilities of annotators (such as through the sensitivities and specificities of annotators in the binary label setting). In the literature, beta or dirichlet distributions are typically imposed as priors on annotator reliability. In this study, we investigated the use of a neuroscientifically validated model of decision making, known as the drift-diffusion model, as a prior on the annotator labeling process. Two experiments were conducted on synthetically generated data with non-linear (sinusoidal) decision boundaries. Variational inference was used to predict ground truth labels and annotator related parameters. Our method performed similarly to a state-of-the-art technique (SVGPCR) in prediction of crowdsourced data labels and prediction through a crowdsourced-generated Gaussian process classifier. By relying on a neuroscientifically validated model of decision making to model annotator behavior, our technique opens the avenue of predicting neuroscientific biomarkers of annotators, expanding the scope of what may be learnt about annotators in crowdsourcing tasks.
Collapse
Affiliation(s)
- Shamal Lalvani
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, 60201, USA.
| | - Aggelos Katsaggelos
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, 60201, USA
| |
Collapse
|
5
|
López-Pérez M, Morales-Álvarez P, Cooper LAD, Felicelli C, Goldstein J, Vadasz B, Molina R, Katsaggelos AK. Learning from crowds for automated histopathological image segmentation. Comput Med Imaging Graph 2024; 112:102327. [PMID: 38194768 DOI: 10.1016/j.compmedimag.2024.102327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 10/20/2023] [Accepted: 12/12/2023] [Indexed: 01/11/2024]
Abstract
Automated semantic segmentation of histopathological images is an essential task in Computational Pathology (CPATH). The main limitation of Deep Learning (DL) to address this task is the scarcity of expert annotations. Crowdsourcing (CR) has emerged as a promising solution to reduce the individual (expert) annotation cost by distributing the labeling effort among a group of (non-expert) annotators. Extracting knowledge in this scenario is challenging, as it involves noisy annotations. Jointly learning the underlying (expert) segmentation and the annotators' expertise is currently a commonly used approach. Unfortunately, this approach is frequently carried out by learning a different neural network for each annotator, which scales poorly when the number of annotators grows. For this reason, this strategy cannot be easily applied to real-world CPATH segmentation. This paper proposes a new family of methods for CR segmentation of histopathological images. Our approach consists of two coupled networks: a segmentation network (for learning the expert segmentation) and an annotator network (for learning the annotators' expertise). We propose to estimate the annotators' behavior with only one network that receives the annotator ID as input, achieving scalability on the number of annotators. Our family is composed of three different models for the annotator network. Within this family, we propose a novel modeling of the annotator network in the CR segmentation literature, which considers the global features of the image. We validate our methods on a real-world dataset of Triple Negative Breast Cancer images labeled by several medical students. Our new CR modeling achieves a Dice coefficient of 0.7827, outperforming the well-known STAPLE (0.7039) and being competitive with the supervised method with expert labels (0.7723). The code is available at https://github.com/wizmik12/CRowd_Seg.
Collapse
Affiliation(s)
- Miguel López-Pérez
- Department of Computer Science and Artificial Intelligence, University of Granada, Spain.
| | | | - Lee A D Cooper
- Department of Pathology at Northwestern University, Chicago, USA; Center for Computational Imaging and Signal Analytics, Northwestern University, Chicago, USA.
| | | | | | - Brian Vadasz
- Department of Pathology at Northwestern University, Chicago, USA
| | - Rafael Molina
- Department of Computer Science and Artificial Intelligence, University of Granada, Spain.
| | - Aggelos K Katsaggelos
- Center for Computational Imaging and Signal Analytics, Northwestern University, Chicago, USA; Department of Electrical and Computer Engineering at Northwestern University, Chicago, USA.
| |
Collapse
|
6
|
Kanwal N, López-Pérez M, Kiraz U, Zuiverloon TCM, Molina R, Engan K. Are you sure it's an artifact? Artifact detection and uncertainty quantification in histological images. Comput Med Imaging Graph 2024; 112:102321. [PMID: 38199127 DOI: 10.1016/j.compmedimag.2023.102321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 11/08/2023] [Accepted: 12/12/2023] [Indexed: 01/12/2024]
Abstract
Modern cancer diagnostics involves extracting tissue specimens from suspicious areas and conducting histotechnical procedures to prepare a digitized glass slide, called Whole Slide Image (WSI), for further examination. These procedures frequently introduce different types of artifacts in the obtained WSI, and histological artifacts might influence Computational Pathology (CPATH) systems further down to a diagnostic pipeline if not excluded or handled. Deep Convolutional Neural Networks (DCNNs) have achieved promising results for the detection of some WSI artifacts, however, they do not incorporate uncertainty in their predictions. This paper proposes an uncertainty-aware Deep Kernel Learning (DKL) model to detect blurry areas and folded tissues, two types of artifacts that can appear in WSIs. The proposed probabilistic model combines a CNN feature extractor and a sparse Gaussian Processes (GPs) classifier, which improves the performance of current state-of-the-art artifact detection DCNNs and provides uncertainty estimates. We achieved 0.996 and 0.938 F1 scores for blur and folded tissue detection on unseen data, respectively. In extensive experiments, we validated the DKL model on unseen data from external independent cohorts with different staining and tissue types, where it outperformed DCNNs. Interestingly, the DKL model is more confident in the correct predictions and less in the wrong ones. The proposed DKL model can be integrated into the preprocessing pipeline of CPATH systems to provide reliable predictions and possibly serve as a quality control tool.
Collapse
Affiliation(s)
- Neel Kanwal
- Department of Electrical Engineering and Computer Science, University of Stavanger, 4021 Stavanger, Norway.
| | - Miguel López-Pérez
- Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain
| | - Umay Kiraz
- Department of Pathology, Stavanger University Hospital, 4011 Stavanger, Norway; Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, 4021 Stavanger, Norway
| | - Tahlita C M Zuiverloon
- Department of Urology, University Medical Center Rotterdam, Erasmus MC Cancer Institute, 1035 GD Rotterdam, The Netherlands
| | - Rafael Molina
- Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain
| | - Kjersti Engan
- Department of Electrical Engineering and Computer Science, University of Stavanger, 4021 Stavanger, Norway
| |
Collapse
|
7
|
Soliman A, Li Z, Parwani AV. Artificial intelligence's impact on breast cancer pathology: a literature review. Diagn Pathol 2024; 19:38. [PMID: 38388367 PMCID: PMC10882736 DOI: 10.1186/s13000-024-01453-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 01/26/2024] [Indexed: 02/24/2024] Open
Abstract
This review discusses the profound impact of artificial intelligence (AI) on breast cancer (BC) diagnosis and management within the field of pathology. It examines the various applications of AI across diverse aspects of BC pathology, highlighting key findings from multiple studies. Integrating AI into routine pathology practice stands to improve diagnostic accuracy, thereby contributing to reducing avoidable errors. Additionally, AI has excelled in identifying invasive breast tumors and lymph node metastasis through its capacity to process large whole-slide images adeptly. Adaptive sampling techniques and powerful convolutional neural networks mark these achievements. The evaluation of hormonal status, which is imperative for BC treatment choices, has also been enhanced by AI quantitative analysis, aiding interobserver concordance and reliability. Breast cancer grading and mitotic count evaluation also benefit from AI intervention. AI-based frameworks effectively classify breast carcinomas, even for moderately graded cases that traditional methods struggle with. Moreover, AI-assisted mitotic figures quantification surpasses manual counting in precision and sensitivity, fostering improved prognosis. The assessment of tumor-infiltrating lymphocytes in triple-negative breast cancer using AI yields insights into patient survival prognosis. Furthermore, AI-powered predictions of neoadjuvant chemotherapy response demonstrate potential for streamlining treatment strategies. Addressing limitations, such as preanalytical variables, annotation demands, and differentiation challenges, is pivotal for realizing AI's full potential in BC pathology. Despite the existing hurdles, AI's multifaceted contributions to BC pathology hold great promise, providing enhanced accuracy, efficiency, and standardization. Continued research and innovation are crucial for overcoming obstacles and fully harnessing AI's transformative capabilities in breast cancer diagnosis and assessment.
Collapse
Affiliation(s)
- Amr Soliman
- Department of Pathology, Ohio State University, Columbus, OH, USA
| | - Zaibo Li
- Department of Pathology, Ohio State University, Columbus, OH, USA
| | - Anil V Parwani
- Department of Pathology, Ohio State University, Columbus, OH, USA.
| |
Collapse
|
8
|
Schwarze S, Schaadt NS, Sobotta VMG, Spicher N, Skripuletz T, Esmaeilzadeh M, Krauss JK, Hartmann C, Deserno TM, Feuerhake F. Task design for crowdsourced glioma cell annotation in microscopy images. Sci Rep 2024; 14:1965. [PMID: 38263411 PMCID: PMC10805723 DOI: 10.1038/s41598-024-51995-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 01/12/2024] [Indexed: 01/25/2024] Open
Abstract
Crowdsourcing has been used in computational pathology to generate cell and cell nuclei annotations for machine learning. Herein, we broaden its scope to the previously unsolved challenging task of glioma cell detection. This requires multiplexed immunofluorescence microscopy due to diffuse invasiveness and exceptional similarity between glioma cells and reactive astrocytes. In four pilot experiments, we iteratively developed a task design enabling high-quality annotations by crowdworkers on Amazon Mechanical Turk. We applied majority or weighted vote and validated them against ground truth in the final setting. On the base of a YOLO convolutional neural network architecture, we used these consensus labels for training with different image representations regarding colors, intensities, and immmunohistochemical marker combinations. A crowd of 712 workers defined aggregated point annotations in 235 images with an average [Formula: see text] score of 0.627 for majority vote. The networks resulted in acceptable [Formula: see text] scores up to 0.69 for YOLOv8 on average and indicated first evidence for transferability to images lacking tumor markers, especially in IDH-wildtype glioblastoma. Our work confirms feasibility of crowdsourcing to generate labels suitable for training of machine learning tools in the challenging and clinically relevant use case of glioma microenvironment.
Collapse
Affiliation(s)
- Svea Schwarze
- Department of Neuropathology, Institute for Pathology, Hannover Medical School, Hannover, Germany
| | - Nadine S Schaadt
- Department of Neuropathology, Institute for Pathology, Hannover Medical School, Hannover, Germany
| | - Viktor M G Sobotta
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Braunschweig, Germany
| | - Nicolai Spicher
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Braunschweig, Germany
| | | | | | - Joachim K Krauss
- Department of Neurosurgery, Hannover Medical School, Hannover, Germany
| | - Christian Hartmann
- Department of Neuropathology, Institute for Pathology, Hannover Medical School, Hannover, Germany
| | - Thomas M Deserno
- Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Braunschweig, Germany
| | - Friedrich Feuerhake
- Department of Neuropathology, Institute for Pathology, Hannover Medical School, Hannover, Germany.
- Institute of Neuropathology, University Clinic Freiburg, Freiburg, Germany.
| |
Collapse
|
9
|
Del Amor R, Pérez-Cano J, López-Pérez M, Terradez L, Aneiros-Fernandez J, Morales S, Mateos J, Molina R, Naranjo V. Annotation protocol and crowdsourcing multiple instance learning classification of skin histological images: The CR-AI4SkIN dataset. Artif Intell Med 2023; 145:102686. [PMID: 37925214 DOI: 10.1016/j.artmed.2023.102686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 10/11/2023] [Accepted: 10/13/2023] [Indexed: 11/06/2023]
Abstract
Digital Pathology (DP) has experienced a significant growth in recent years and has become an essential tool for diagnosing and prognosis of tumors. The availability of Whole Slide Images (WSIs) and the implementation of Deep Learning (DL) algorithms have paved the way for the appearance of Artificial Intelligence (AI) systems that support the diagnosis process. These systems require extensive and varied data for their training to be successful. However, creating labeled datasets in histopathology is laborious and time-consuming. We have developed a crowdsourcing-multiple instance labeling/learning protocol that is applied to the creation and use of the CR-AI4SkIN dataset.2 CR-AI4SkIN contains 271 WSIs of 7 Cutaneous Spindle Cell (CSC) neoplasms with expert and non-expert labels at region and WSI levels. It is the first dataset of these types of neoplasms made available. The regions selected by the experts are used to learn an automatic extractor of Regions of Interest (ROIs) from WSIs. To produce the embedding of each WSI, the representations of patches within the ROIs are obtained using a contrastive learning method, and then combined. Finally, they are fed to a Gaussian process-based crowdsourcing classifier, which utilizes the noisy non-expert WSI labels. We validate our crowdsourcing-multiple instance learning method in the CR-AI4SkIN dataset, addressing a binary classification problem (malign vs. benign). The proposed method obtains an F1 score of 0.7911 on the test set, outperforming three widely used aggregation methods for crowdsourcing tasks. Furthermore, our crowdsourcing method also outperforms the supervised model with expert labels on the test set (F1-score = 0.6035). The promising results support the proposed crowdsourcing multiple instance learning annotation protocol. It also validates the automatic extraction of interest regions and the use of contrastive embedding and Gaussian process classification to perform crowdsourcing classification tasks.
Collapse
Affiliation(s)
- Rocío Del Amor
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València, Valencia, Spain
| | - Jose Pérez-Cano
- Department of Computer Science and Artificial Intelligence, University of Granada, 18010 Granada, Spain
| | - Miguel López-Pérez
- Department of Computer Science and Artificial Intelligence, University of Granada, 18010 Granada, Spain.
| | - Liria Terradez
- Pathology Department. Hospital Clínico Universitario de Valencia, Universidad de Valencia, Spain
| | | | - Sandra Morales
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València, Valencia, Spain
| | - Javier Mateos
- Department of Computer Science and Artificial Intelligence, University of Granada, 18010 Granada, Spain
| | - Rafael Molina
- Department of Computer Science and Artificial Intelligence, University of Granada, 18010 Granada, Spain
| | - Valery Naranjo
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València, Valencia, Spain
| |
Collapse
|
10
|
Yang Y, Sun K, Gao Y, Wang K, Yu G. Preparing Data for Artificial Intelligence in Pathology with Clinical-Grade Performance. Diagnostics (Basel) 2023; 13:3115. [PMID: 37835858 PMCID: PMC10572440 DOI: 10.3390/diagnostics13193115] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/27/2023] [Accepted: 09/28/2023] [Indexed: 10/15/2023] Open
Abstract
The pathology is decisive for disease diagnosis but relies heavily on experienced pathologists. In recent years, there has been growing interest in the use of artificial intelligence in pathology (AIP) to enhance diagnostic accuracy and efficiency. However, the impressive performance of deep learning-based AIP in laboratory settings often proves challenging to replicate in clinical practice. As the data preparation is important for AIP, the paper has reviewed AIP-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. An in-depth analysis of data preparation methods is conducted, encompassing the acquisition of pathological tissue slides, data cleaning, screening, and subsequent digitization. Expert review, image annotation, dataset division for model training and validation are also discussed. Furthermore, we delve into the reasons behind the challenges in reproducing the high performance of AIP in clinical settings and present effective strategies to enhance AIP's clinical performance. The robustness of AIP depends on a randomized collection of representative disease slides, incorporating rigorous quality control and screening, correction of digital discrepancies, reasonable annotation, and sufficient data volume. Digital pathology is fundamental in clinical-grade AIP, and the techniques of data standardization and weakly supervised learning methods based on whole slide image (WSI) are effective ways to overcome obstacles of performance reproduction. The key to performance reproducibility lies in having representative data, an adequate amount of labeling, and ensuring consistency across multiple centers. Digital pathology for clinical diagnosis, data standardization and the technique of WSI-based weakly supervised learning will hopefully build clinical-grade AIP.
Collapse
Affiliation(s)
- Yuanqing Yang
- Department of Biomedical Engineering, School of Basic Medical Sciences, Central South University, Changsha 410013, China; (Y.Y.); (K.S.)
- Department of Biomedical Engineering, School of Medical, Tsinghua University, Beijing 100084, China
| | - Kai Sun
- Department of Biomedical Engineering, School of Basic Medical Sciences, Central South University, Changsha 410013, China; (Y.Y.); (K.S.)
- Furong Laboratory, Changsha 410013, China
| | - Yanhua Gao
- Department of Ultrasound, Shaanxi Provincial People’s Hospital, Xi’an 710068, China;
| | - Kuansong Wang
- Department of Pathology, School of Basic Medical Sciences, Central South University, Changsha 410013, China;
- Department of Pathology, Xiangya Hospital, Central South University, Changsha 410013, China
| | - Gang Yu
- Department of Biomedical Engineering, School of Basic Medical Sciences, Central South University, Changsha 410013, China; (Y.Y.); (K.S.)
| |
Collapse
|
11
|
Ruiz-Fresneda MA, Gijón A, Morales-Álvarez P. Bibliometric analysis of the global scientific production on machine learning applied to different cancer types. ENVIRONMENTAL SCIENCE AND POLLUTION RESEARCH INTERNATIONAL 2023; 30:96125-96137. [PMID: 37566331 PMCID: PMC10482761 DOI: 10.1007/s11356-023-28576-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 06/29/2023] [Indexed: 08/12/2023]
Abstract
Cancer disease is one of the main causes of death in the world, with million annual cases in the last decades. The need to find a cure has stimulated the search for efficient treatments and diagnostic procedures. One of the most promising tools that has emerged against cancer in recent years is machine learning (ML), which has raised a huge number of scientific papers published in a relatively short period of time. The present study analyzes global scientific production on ML applied to the most relevant cancer types through various bibliometric indicators. We find that over 30,000 studies have been published so far and observe that cancers with the highest number of published studies using ML (breast, lung, and colon cancer) are those with the highest incidence, being the USA and China the main scientific producers on the subject. Interestingly, the role of China and Japan in stomach cancer is correlated with the number of cases of this cancer type in Asia (78% of the worldwide cases). Knowing the countries and institutions that most study each area can be of great help for improving international collaborations between research groups and countries. Our analysis shows that medical and computer science journals lead the number of publications on the subject and could be useful for researchers in the field. Finally, keyword co-occurrence analysis suggests that ML-cancer research trends are focused not only on the use of ML as an effective diagnostic method, but also for the improvement of radiotherapy- and chemotherapy-based treatments.
Collapse
Affiliation(s)
| | - Alfonso Gijón
- Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain
- Research Centre for Information and Communication Technologies (CITIC-UGR), University of Granada, Granada, Spain
| | - Pablo Morales-Álvarez
- Research Centre for Information and Communication Technologies (CITIC-UGR), University of Granada, Granada, Spain
- Department of Statistics and Operations Research, University of Granada, Granada, Spain
| |
Collapse
|
12
|
Verghese G, Li M, Liu F, Lohan A, Kurian NC, Meena S, Gazinska P, Shah A, Oozeer A, Chan T, Opdam M, Linn S, Gillett C, Alberts E, Hardiman T, Jones S, Thavaraj S, Jones JL, Salgado R, Pinder SE, Rane S, Sethi A, Grigoriadis A. Multiscale deep learning framework captures systemic immune features in lymph nodes predictive of triple negative breast cancer outcome in large-scale studies. J Pathol 2023; 260:376-389. [PMID: 37230111 PMCID: PMC10720675 DOI: 10.1002/path.6088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 02/27/2023] [Accepted: 04/11/2023] [Indexed: 05/27/2023]
Abstract
The suggestion that the systemic immune response in lymph nodes (LNs) conveys prognostic value for triple-negative breast cancer (TNBC) patients has not previously been investigated in large cohorts. We used a deep learning (DL) framework to quantify morphological features in haematoxylin and eosin-stained LNs on digitised whole slide images. From 345 breast cancer patients, 5,228 axillary LNs, cancer-free and involved, were assessed. Generalisable multiscale DL frameworks were developed to capture and quantify germinal centres (GCs) and sinuses. Cox regression proportional hazard models tested the association between smuLymphNet-captured GC and sinus quantifications and distant metastasis-free survival (DMFS). smuLymphNet achieved a Dice coefficient of 0.86 and 0.74 for capturing GCs and sinuses, respectively, and was comparable to an interpathologist Dice coefficient of 0.66 (GC) and 0.60 (sinus). smuLymphNet-captured sinuses were increased in LNs harbouring GCs (p < 0.001). smuLymphNet-captured GCs retained clinical relevance in LN-positive TNBC patients whose cancer-free LNs had on average ≥2 GCs, had longer DMFS (hazard ratio [HR] = 0.28, p = 0.02) and extended GCs' prognostic value to LN-negative TNBC patients (HR = 0.14, p = 0.002). Enlarged smuLymphNet-captured sinuses in involved LNs were associated with superior DMFS in LN-positive TNBC patients in a cohort from Guy's Hospital (multivariate HR = 0.39, p = 0.039) and with distant recurrence-free survival in 95 LN-positive TNBC patients of the Dutch-N4plus trial (HR = 0.44, p = 0.024). Heuristic scoring of subcapsular sinuses in LNs of LN-positive Tianjin TNBC patients (n = 85) cross-validated the association of enlarged sinuses with shorter DMFS (involved LNs: HR = 0.33, p = 0.029 and cancer-free LNs: HR = 0.21 p = 0.01). Morphological LN features reflective of cancer-associated responses are robustly quantifiable by smuLymphNet. Our findings further strengthen the value of assessment of LN properties beyond the detection of metastatic deposits for prognostication of TNBC patients. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Gregory Verghese
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Breast Cancer Now Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Mengyuan Li
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Fangfang Liu
- Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center for Cancer, Tianjin's Clinical Research Center for Cancer, Key Laboratory of Breast Cancer Prevention and Therapy, Tianjin Medical University, Ministry of EducationKey Laboratory of Cancer Prevention and TherapyTianjinPR China
| | - Amit Lohan
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Nikhil Cherian Kurian
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Swati Meena
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Patrycja Gazinska
- Breast Cancer Now Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Biobank Research GroupLukasiewicz Research Network, PORT Polish Center for Technology DevelopmentWroclawPoland
| | - Aekta Shah
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Department of PathologyTata Memorial Centre, Tata Memorial Hospital, Homi Bhabha National InstituteMumbaiIndia
| | - Aasiyah Oozeer
- King's Health Partners Cancer Biobank, King's College LondonLondonUK
| | - Terry Chan
- Division of Molecular PathologyThe Netherlands Cancer InstituteAmsterdamThe Netherlands
| | - Mark Opdam
- Division of Molecular PathologyThe Netherlands Cancer InstituteAmsterdamThe Netherlands
| | - Sabine Linn
- Division of Molecular PathologyThe Netherlands Cancer InstituteAmsterdamThe Netherlands
- Department of Medical OncologyThe Netherlands Cancer Institute, Antoni van LeeuwenhoekAmsterdamThe Netherlands
- Department of PathologyUniversity Medical CentreUtrechtThe Netherlands
| | - Cheryl Gillett
- King's Health Partners Cancer Biobank, King's College LondonLondonUK
| | - Elena Alberts
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Thomas Hardiman
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Samantha Jones
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of LondonLondonUK
| | - Selvam Thavaraj
- Faculty of Dentistry, Oral & Craniofacial ScienceKing's College LondonLondonUK
- Head and Neck PathologyGuy's & St Thomas' NHS Foundation TrustLondonUK
| | - J Louise Jones
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of LondonLondonUK
| | - Roberto Salgado
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Division of ResearchPeter Mac Callum Cancer CentreMelbourneAustralia
| | - Sarah E Pinder
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Swapnil Rane
- Department of PathologyTata Memorial Centre‐ACTREC, HBNIMumbaiIndia
| | - Amit Sethi
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Anita Grigoriadis
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Breast Cancer Now Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| |
Collapse
|
13
|
Goldstein JA, Nateghi R, Irmakci I, Cooper LAD. Machine learning classification of placental villous infarction, perivillous fibrin deposition, and intervillous thrombus. Placenta 2023; 135:43-50. [PMID: 36958179 PMCID: PMC10156426 DOI: 10.1016/j.placenta.2023.03.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 03/09/2023] [Accepted: 03/11/2023] [Indexed: 03/17/2023]
Abstract
INTRODUCTION Placental parenchymal lesions are commonly encountered and carry significant clinical associations. However, they are frequently missed or misclassified by general practice pathologists. Interpretation of pathology slides has emerged as one of the most successful applications of machine learning (ML) in medicine with applications ranging from cancer detection and prognostication to transplant medicine. The goal of this study was to use a whole-slide learning model to identify and classify placental parenchymal lesions including villous infarctions, intervillous thrombi (IVT), and perivillous fibrin deposition (PVFD). METHODS We generated whole slide images from placental discs examined at our institution with infarct, IVT, PVFD, or no macroscopic lesion. Slides were analyzed as a set of overlapping patches. We extracted feature vectors from each patch using a pretrained convolutional neural network (EfficientNetV2L). We trained a model to assign attention to each vector and used the attentions as weights to produce a pooled feature vector. The pooled vector was classified as normal or 1 of 3 lesions using a fully connected network. Patch attention was plotted to highlight informative areas of the slide. RESULTS Overall balanced accuracy in a test set of held-out slides was 0.86 with receiver-operator characteristic areas under the curve of 0.917-0.993. Cases of PVFD were frequently miscalled as normal or infarcts, the latter possibly due to the perivillous fibrin found at the periphery of infarctions. We used attention maps to further understand some errors, including one most likely due to poor tissue fixation and processing. DISCUSSION We used a whole-slide learning paradigm to train models to recognize three of the most common placental parenchymal lesions. We used attention maps to gain insight into model function, which differed from intuitive explanations.
Collapse
Affiliation(s)
| | - Ramin Nateghi
- Northwestern University, Department of Pathology, Chicago, IL, USA
| | - Ismail Irmakci
- Northwestern University, Department of Pathology, Chicago, IL, USA
| | - Lee A D Cooper
- Northwestern University, Department of Pathology, Chicago, IL, USA; Northwestern University, McCormick School of Engineering, Evanston, IL, USA
| |
Collapse
|
14
|
Probabilistic fusion of crowds and experts for the search of gravitational waves. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.110183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
15
|
Kim I, Kang K, Song Y, Kim TJ. Application of Artificial Intelligence in Pathology: Trends and Challenges. Diagnostics (Basel) 2022; 12:2794. [PMID: 36428854 PMCID: PMC9688959 DOI: 10.3390/diagnostics12112794] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 11/03/2022] [Accepted: 11/11/2022] [Indexed: 11/16/2022] Open
Abstract
Given the recent success of artificial intelligence (AI) in computer vision applications, many pathologists anticipate that AI will be able to assist them in a variety of digital pathology tasks. Simultaneously, tremendous advancements in deep learning have enabled a synergy with artificial intelligence (AI), allowing for image-based diagnosis on the background of digital pathology. There are efforts for developing AI-based tools to save pathologists time and eliminate errors. Here, we describe the elements in the development of computational pathology (CPATH), its applicability to AI development, and the challenges it faces, such as algorithm validation and interpretability, computing systems, reimbursement, ethics, and regulations. Furthermore, we present an overview of novel AI-based approaches that could be integrated into pathology laboratory workflows.
Collapse
Affiliation(s)
- Inho Kim
- College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Republic of Korea
| | - Kyungmin Kang
- College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Republic of Korea
| | - Youngjae Song
- College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Republic of Korea
| | - Tae-Jung Kim
- Department of Hospital Pathology, Yeouido St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, 10, 63-ro, Yeongdeungpo-gu, Seoul 07345, Republic of Korea
| |
Collapse
|