1
|
Tafavvoghi M, Bongo LA, Shvetsov N, Busund LTR, Møllersen K. Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review. J Pathol Inform 2024; 15:100363. [PMID: 38405160 PMCID: PMC10884505 DOI: 10.1016/j.jpi.2024.100363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 01/23/2024] [Indexed: 02/27/2024] Open
Abstract
Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.
Collapse
Affiliation(s)
- Masoud Tafavvoghi
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| | - Lars Ailo Bongo
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | - Nikita Shvetsov
- Department of Computer Science, Uit The Arctic University of Norway, Tromsø, Norway
| | | | - Kajsa Møllersen
- Department of Community Medicine, Uit The Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
2
|
Liu H, Xie X, Wang B. Deep learning infers clinically relevant protein levels and drug response in breast cancer from unannotated pathology images. NPJ Breast Cancer 2024; 10:18. [PMID: 38413598 PMCID: PMC10899601 DOI: 10.1038/s41523-024-00620-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 02/02/2024] [Indexed: 02/29/2024] Open
Abstract
The computational pathology has been demonstrated to effectively uncover tumor-related genomic alterations and transcriptomic patterns. Although proteomics has indeed shown great potential in the field of precision medicine, few studies have focused on the computational prediction of protein levels from pathology images. In this paper, we assume that deep learning-based pathological features imply the protein levels of tumor biomarkers that are indicative of prognosis and drug response. For this purpose, we propose wsi2rppa, a weakly supervised contrastive learning framework to infer the protein levels of tumor biomarkers from whole slide images (WSIs) in breast cancer. We first conducted contrastive learning-based pre-training on tessellated tiles to extract pathological features, which are then aggregated by attention pooling and adapted to downstream tasks. We conducted extensive evaluation experiments on the TCGA-BRCA cohort (1978 WSIs of 1093 patients with protein levels of 223 biomarkers) and the CPTAC-BRCA cohort (642 WSIs of 134 patients). The results showed that our method achieved state-of-the-art performance in tumor diagnostic tasks, and also performed well in predicting clinically relevant protein levels and drug response. To show the model interpretability, we spatially visualized the WSIs colored the tiles by their attention scores, and found that the regions with high scores were highly consistent with the tumor and necrotic regions annotated by a 10-year experienced pathologist. Moreover, spatial transcriptomic data further verified that the heatmap generated by attention scores agrees greatly with the spatial expression landscape of two typical tumor biomarker genes. In predicting the response to drug trastuzumab treatment, our method achieved a 0.79 AUC value which is much higher than the previous study reported 0.68. These findings showed the remarkable potential of computational pathology in the prediction of clinically relevant protein levels, drug response, and clinical outcomes.
Collapse
Affiliation(s)
- Hui Liu
- College of Computer and Information Engineering, Nanjing Tech University, 211816, Nanjing, Jiangsu, China
| | - Xiaodong Xie
- College of Computer and Information Engineering, Nanjing Tech University, 211816, Nanjing, Jiangsu, China
| | - Bin Wang
- Department of Cardiothoracic Surgery, The Third Affiliated Hospital of Soochow University, 213110, Changzhou, Jiangsu, China.
| |
Collapse
|
3
|
Ashurov A, Chelloug SA, Tselykh A, Muthanna MSA, Muthanna A, Al-Gaashani MSAM. Improved Breast Cancer Classification through Combining Transfer Learning and Attention Mechanism. Life (Basel) 2023; 13:1945. [PMID: 37763348 PMCID: PMC10532552 DOI: 10.3390/life13091945] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Revised: 09/16/2023] [Accepted: 09/17/2023] [Indexed: 09/29/2023] Open
Abstract
Breast cancer, a leading cause of female mortality worldwide, poses a significant health challenge. Recent advancements in deep learning techniques have revolutionized breast cancer pathology by enabling accurate image classification. Various imaging methods, such as mammography, CT, MRI, ultrasound, and biopsies, aid in breast cancer detection. Computer-assisted pathological image classification is of paramount importance for breast cancer diagnosis. This study introduces a novel approach to breast cancer histopathological image classification. It leverages modified pre-trained CNN models and attention mechanisms to enhance model interpretability and robustness, emphasizing localized features and enabling accurate discrimination of complex cases. Our method involves transfer learning with deep CNN models-Xception, VGG16, ResNet50, MobileNet, and DenseNet121-augmented with the convolutional block attention module (CBAM). The pre-trained models are finetuned, and the two CBAM models are incorporated at the end of the pre-trained models. The models are compared to state-of-the-art breast cancer diagnosis approaches and tested for accuracy, precision, recall, and F1 score. The confusion matrices are used to evaluate and visualize the results of the compared models. They help in assessing the models' performance. The test accuracy rates for the attention mechanism (AM) using the Xception model on the "BreakHis" breast cancer dataset are encouraging at 99.2% and 99.5%. The test accuracy for DenseNet121 with AMs is 99.6%. The proposed approaches also performed better than previous approaches examined in the related studies.
Collapse
Affiliation(s)
- Asadulla Ashurov
- School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
| | - Samia Allaoua Chelloug
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Alexey Tselykh
- Institute of Computer Technologies and Information Security, Southern Federal University, Taganrog 347922, Russia; (A.T.); (M.S.A.M.)
| | - Mohammed Saleh Ali Muthanna
- Institute of Computer Technologies and Information Security, Southern Federal University, Taganrog 347922, Russia; (A.T.); (M.S.A.M.)
| | - Ammar Muthanna
- RUDN University, 6 Miklukho-Maklaya Street, Moscow 117198, Russia;
| | - Mehdhar S. A. M. Al-Gaashani
- College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
| |
Collapse
|
4
|
Nause N, Ispizua Yamati FR, Seidel M, Mahlein AK, Hoffmann CM. Workflow for phenotyping sugar beet roots by automated evaluation of cell characteristics and tissue arrangement using digital image processing. PLANT METHODS 2023; 19:35. [PMID: 37004019 PMCID: PMC10064576 DOI: 10.1186/s13007-023-01014-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 03/22/2023] [Indexed: 06/19/2023]
Abstract
BACKGROUND Cell characteristics, including cell type, size, shape, packing, cell-to-cell-adhesion, intercellular space, and cell wall thickness, influence the physical characteristics of plant tissues. Genotypic differences were found concerning damage susceptibility related to beet texture for sugar beet (Beta vulgaris). Sugar beet storage roots are characterized by heterogeneous tissue with several cambium rings surrounded by small-celled vascular tissue and big-celled sugar-storing parenchyma between the rings. This study presents a procedure for phenotyping heterogeneous tissues like beetroots by imaging. RESULTS Ten Beta genotypes (nine sugar beet and one fodder beet) were included to establish a pipeline for the automated histologic evaluation of cell characteristics and tissue arrangement using digital image processing written in the programming language R. The identification of cells has been validated by comparison with manual cell identification. Cells are reliably discriminated from intercellular spaces, and cells with similar morphological features are assigned to biological tissue types. CONCLUSIONS Genotypic differences in cell diameter and cell arrangement can straightforwardly be phenotyped by the presented workflow. The presented routine can further identify genotypic differences in cell diameter and cell arrangement during early growth stages and between sugar storage capabilities.
Collapse
Affiliation(s)
- Nelia Nause
- Institute of Sugar Beet Research, Holtenser Landstraße 77, 37079, Göttingen, Germany
| | | | - Marion Seidel
- Bildungsakademie Der Universitätsmedizin Göttingen, Humboldtallee 11, 37073, Göttingen, Germany
| | - Anne-Katrin Mahlein
- Institute of Sugar Beet Research, Holtenser Landstraße 77, 37079, Göttingen, Germany
| | - Christa M Hoffmann
- Institute of Sugar Beet Research, Holtenser Landstraße 77, 37079, Göttingen, Germany
| |
Collapse
|
5
|
Ogundokun RO, Misra S, Akinrotimi AO, Ogul H. MobileNet-SVM: A Lightweight Deep Transfer Learning Model to Diagnose BCH Scans for IoMT-Based Imaging Sensors. SENSORS (BASEL, SWITZERLAND) 2023; 23:656. [PMID: 36679455 PMCID: PMC9863875 DOI: 10.3390/s23020656] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/15/2022] [Revised: 12/02/2022] [Accepted: 12/16/2022] [Indexed: 06/17/2023]
Abstract
Many individuals worldwide pass away as a result of inadequate procedures for prompt illness identification and subsequent treatment. A valuable life can be saved or at least extended with the early identification of serious illnesses, such as various cancers and other life-threatening conditions. The development of the Internet of Medical Things (IoMT) has made it possible for healthcare technology to offer the general public efficient medical services and make a significant contribution to patients' recoveries. By using IoMT to diagnose and examine BreakHis v1 400× breast cancer histology (BCH) scans, disorders may be quickly identified and appropriate treatment can be given to a patient. Imaging equipment having the capability of auto-analyzing acquired pictures can be used to achieve this. However, the majority of deep learning (DL)-based image classification approaches are of a large number of parameters and unsuitable for application in IoMT-centered imaging sensors. The goal of this study is to create a lightweight deep transfer learning (DTL) model suited for BCH scan examination and has a good level of accuracy. In this study, a lightweight DTL-based model "MobileNet-SVM", which is the hybridization of MobileNet and Support Vector Machine (SVM), for auto-classifying BreakHis v1 400× BCH images is presented. When tested against a real dataset of BreakHis v1 400× BCH images, the suggested technique achieved a training accuracy of 100% on the training dataset. It also obtained an accuracy of 91% and an F1-score of 91.35 on the test dataset. Considering how complicated BCH scans are, the findings are encouraging. The MobileNet-SVM model is ideal for IoMT imaging equipment in addition to having a high degree of precision. According to the simulation findings, the suggested model requires a small computation speed and time.
Collapse
Affiliation(s)
- Roseline Oluwaseun Ogundokun
- Department of Multimedia Engineering, Kaunas University of Technology, 44249 Kaunas, Lithuania
- Department of Computer Science, Landmark University, Omu Aran 251103, Kwara, Nigeria
| | - Sanjay Misra
- Department of Computer Science and Communication, Østfold University College, 1757 Halden, Norway
| | | | - Hasan Ogul
- Department of Computer Science and Communication, Østfold University College, 1757 Halden, Norway
| |
Collapse
|