1
|
DeVoe K, Takahashi G, Tarshizi E, Sacker A. Evaluation of the precision and accuracy in the classification of breast histopathology images using the MobileNetV3 model. J Pathol Inform 2024; 15:100377. [PMID: 38706514 PMCID: PMC11066512 DOI: 10.1016/j.jpi.2024.100377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Revised: 04/05/2024] [Accepted: 04/08/2024] [Indexed: 05/07/2024] Open
Abstract
Accurate surgical pathological assessment of breast biopsies is essential to the proper management of breast lesions. Identifying histological features, such as nuclear pleomorphism, increased mitotic activity, cellular atypia, patterns of architectural disruption, as well as invasion through basement membranes into surrounding stroma and normal structures, including invasion of vascular and lymphatic spaces, help to classify lesions as malignant. This visual assessment is repeated on numerous slides taken at various sections through the resected tumor, each at different magnifications. Computer vision models have been proposed to assist human pathologists in classification tasks such as these. Using MobileNetV3, a convolutional architecture designed to achieve high accuracy with a compact parameter footprint, we attempted to classify breast cancer images in the BreakHis_v1 breast pathology dataset to determine the performance of this model out-of-the-box. Using transfer learning to take advantage of ImageNet embeddings without special feature extraction, we were able to correctly classify histopathology images broadly as benign or malignant with 0.98 precision, 0.97 recall, and an F1 score of 0.98. The ability to classify into histological subcategories was varied, with the greatest success being with classifying ductal carcinoma (accuracy 0.95), and the lowest success being with lobular carcinoma (accuracy 0.59). Multiclass ROC assessment of performance as a multiclass classifier yielded AUC values ≥0.97 in both benign and malignant subsets. In comparison with previous efforts, using older and larger convolutional network architectures with feature extraction pre-processing, our work highlights that modern, resource-efficient architectures can classify histopathological images with accuracy that at least matches that of previous efforts, without the need for labor-intensive feature extraction protocols. Suggestions to further refine the model are discussed.
Collapse
Affiliation(s)
- Kenneth DeVoe
- Shiley-Marcos School of Engineering, Applied Artificial Intelligence MS Program, University of San Diego, 5998 Alcalá Park, San Diego, CA 92110, USA
| | - Gary Takahashi
- Shiley-Marcos School of Engineering, Applied Artificial Intelligence MS Program, University of San Diego, 5998 Alcalá Park, San Diego, CA 92110, USA
| | - Ebrahim Tarshizi
- Shiley-Marcos School of Engineering, Applied Artificial Intelligence MS Program, University of San Diego, 5998 Alcalá Park, San Diego, CA 92110, USA
| | - Allan Sacker
- Department of Pathology, Providence St. Vincent Medical Center, 9205 SW Barnes Road, Portland, OR 97225, USA
| |
Collapse
|
2
|
Barcellona L, Nicolè L, Cappellesso R, Dei Tos AP, Ghidoni S. SlideTiler: A dataset creator software for boosting deep learning on histological whole slide images. J Pathol Inform 2024; 15:100356. [PMID: 38222323 PMCID: PMC10787253 DOI: 10.1016/j.jpi.2023.100356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Revised: 11/15/2023] [Accepted: 12/04/2023] [Indexed: 01/16/2024] Open
Abstract
The introduction of deep learning caused a significant breakthrough in digital pathology. Thanks to its capability of mining hidden data patterns in digitised histological slides to resolve diagnostic tasks and extract prognostic and predictive information. However, the high performance achieved in classification tasks depends on the availability of large datasets, whose collection and preprocessing are still time-consuming processes. Therefore, strategies to make these steps more efficient are worth investigation. This work introduces SlideTiler, an open-source software with a user-friendly graphical interface. SlideTiler can manage several image preprocessing phases through an intuitive workflow that does not require specific coding skills. The software was designed to provide direct access to virtual slides, allowing custom tiling of specific regions of interest drawn by the user, tile labelling, quality assessment, and direct export to dataset directories. To illustrate the functions and the scalability of SlideTiler, a deep learning-based classifier was implemented to classify 4 different tumour histotypes available in the TCGA repository. The results demonstrate the effectiveness of SlideTiler in facilitating data preprocessing and promoting accessibility to digitised pathology images for research purposes. Considering the increasing interest in deep learning applications of digital pathology, SlideTiler has a positive impact on this field. Moreover, SlideTiler has been conceived as a dynamic tool in constant evolution, and more updated and efficient versions will be released in the future.
Collapse
Affiliation(s)
- Leonardo Barcellona
- Department of Information Engineering, University of Padua, Padua, Italy
- Polytechnic University of Turin, Turin, Italy
| | - Lorenzo Nicolè
- Unit of Pathology and Cytopathology, Ospedale dell’Angelo, Mestre, Italy
- Department of Medicine, DIMED, University of Padua, Padua, Italy
| | | | - Angelo Paolo Dei Tos
- Department of Medicine, DIMED, University of Padua, Padua, Italy
- Department of Integrated diagnostics, Azienda Ospedale-Università, Padua, Italy
| | - Stefano Ghidoni
- Department of Information Engineering, University of Padua, Padua, Italy
| |
Collapse
|
3
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
4
|
Ciaparrone C, Maffei E, L'Imperio V, Pisapia P, Eloy C, Fraggetta F, Zeppa P, Caputo A. Computer-assisted urine cytology: Faster, cheaper, better? Cytopathology 2024; 35:634-641. [PMID: 38894608 DOI: 10.1111/cyt.13412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 06/05/2024] [Accepted: 06/07/2024] [Indexed: 06/21/2024]
Abstract
Recent advancements in computer-assisted diagnosis (CAD) have catalysed significant progress in pathology, particularly in the realm of urine cytopathology. This review synthesizes the latest developments and challenges in CAD for diagnosing urothelial carcinomas, addressing the limitations of traditional urinary cytology. Through a literature review, we identify and analyse CAD models and algorithms developed for urine cytopathology, highlighting their methodologies and performance metrics. We discuss the potential of CAD to improve diagnostic accuracy, efficiency and patient outcomes, emphasizing its role in streamlining workflow and reducing errors. Furthermore, CAD tools have shown potential in exploring pathological conditions, uncovering novel biomarkers and prognostic/predictive features previously unknown or unseen. Finally, we examine the practical issues surrounding the integration of CAD into clinical practice, including regulatory approval, validation and training for pathologists. Despite the promising results, challenges remain, necessitating further research and validation efforts. Overall, CAD presents a transformative opportunity to revolutionize diagnostic practices in urine cytopathology, paving the way for enhanced patient care and outcomes.
Collapse
Affiliation(s)
- Chiara Ciaparrone
- Department of Pathology, University Hospital of Salerno, Salerno, Italy
| | - Elisabetta Maffei
- Department of Pathology, University Hospital of Salerno, Salerno, Italy
| | - Vincenzo L'Imperio
- Department of Medicine and Surgery, Pathology, IRCCS Fondazione San Gerardo dei Tintori, University of Milano-Bicocca, Milan, Italy
| | - Pasquale Pisapia
- Department of Public Health, University of Naples "Federico II", Naples, Italy
| | - Catarina Eloy
- Pathology Laboratory, Institute of Molecular Pathology and Immunology of University of Porto (IPATIMUP), Porto, Portugal
| | | | - Pio Zeppa
- Department of Pathology, University Hospital of Salerno, Salerno, Italy
- Department of Medicine and Surgery, University of Salerno, Baronissi, Italy
| | - Alessandro Caputo
- Department of Pathology, University Hospital of Salerno, Salerno, Italy
- Department of Medicine and Surgery, University of Salerno, Baronissi, Italy
| |
Collapse
|
5
|
Glass M, Ji Z, Davis R, Pavlisko EN, DiBernardo L, Carney J, Fishbein G, Luthringer D, Miller D, Mitchell R, Larsen B, Butt Y, Bois M, Maleszewski J, Halushka M, Seidman M, Lin CY, Buja M, Stone J, Dov D, Carin L, Glass C. A machine learning algorithm improves the diagnostic accuracy of the histologic component of antibody mediated rejection (AMR-H) in cardiac transplant endomyocardial biopsies. Cardiovasc Pathol 2024; 72:107646. [PMID: 38677634 DOI: 10.1016/j.carpath.2024.107646] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 04/15/2024] [Accepted: 04/16/2024] [Indexed: 04/29/2024] Open
Abstract
BACKGROUND Pathologic antibody mediated rejection (pAMR) remains a major driver of graft failure in cardiac transplant patients. The endomyocardial biopsy remains the primary diagnostic tool but presents with challenges, particularly in distinguishing the histologic component (pAMR-H) defined by 1) intravascular macrophage accumulation in capillaries and 2) activated endothelial cells that expand the cytoplasm to narrow or occlude the vascular lumen. Frequently, pAMR-H is difficult to distinguish from acute cellular rejection (ACR) and healing injury. With the advent of digital slide scanning and advances in machine deep learning, artificial intelligence technology is widely under investigation in the areas of oncologic pathology, but in its infancy in transplant pathology. For the first time, we determined if a machine learning algorithm could distinguish pAMR-H from normal myocardium, healing injury and ACR. MATERIALS AND METHODS A total of 4,212 annotations (1,053 regions of normal, 1,053 pAMR-H, 1,053 healing injury and 1,053 ACR) were completed from 300 hematoxylin and eosin slides scanned using a Leica Aperio GT450 digital whole slide scanner at 40X magnification. All regions of pAMR-H were annotated from patients confirmed with a previous diagnosis of pAMR2 (>50% positive C4d immunofluorescence and/or >10% CD68 positive intravascular macrophages). Annotations were imported into a Python 3.7 development environment using the OpenSlide™ package and a convolutional neural network approach utilizing transfer learning was performed. RESULTS The machine learning algorithm showed 98% overall validation accuracy and pAMR-H was correctly distinguished from specific categories with the following accuracies: normal myocardium (99.2%), healing injury (99.5%) and ACR (99.5%). CONCLUSION Our novel deep learning algorithm can reach acceptable, and possibly surpass, performance of current diagnostic standards of identifying pAMR-H. Such a tool may serve as an adjunct diagnostic aid for improving the pathologist's accuracy and reproducibility, especially in difficult cases with high inter-observer variability. This is one of the first studies that provides evidence that an artificial intelligence machine learning algorithm can be trained and validated to diagnose pAMR-H in cardiac transplant patients. Ongoing studies include multi-institutional verification testing to ensure generalizability.
Collapse
Affiliation(s)
- Matthew Glass
- Duke Division of Artificial Intelligence and Computational Pathology, Duke University Medical Center, Durham NC, USA; Department of Anesthesiology, Duke University Medical Center, Durham NC, USA
| | - Zhicheng Ji
- Department of Biostatistics and Bioinformatics, Duke School of Medicine, Durham NC, USA
| | - Richard Davis
- Department of Pathology, Duke University Medical Center, Durham NC, USA
| | - Elizabeth N Pavlisko
- Duke Division of Artificial Intelligence and Computational Pathology, Duke University Medical Center, Durham NC, USA; Department of Pathology, Duke University Medical Center, Durham NC, USA
| | - Louis DiBernardo
- Department of Pathology, Duke University Medical Center, Durham NC, USA
| | - John Carney
- Department of Pathology, Duke University Medical Center, Durham NC, USA
| | - Gregory Fishbein
- Department of Pathology, University of California at Los Angeles, Los Angeles CA, USA
| | - Daniel Luthringer
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles CA, USA
| | - Dylan Miller
- Department of Pathology, Intermountain Healthcare, Salt Lake City UT, USA
| | - Richard Mitchell
- Department of Pathology, Brigham and Women's Hospital, Boston MA, USA
| | - Brandon Larsen
- Department of Pathology and Laboratory Medicine, Mayo Clinic, Phoenix AZ, USA
| | - Yasmeen Butt
- Department of Pathology and Laboratory Medicine, Mayo Clinic, Phoenix AZ, USA
| | - Melanie Bois
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester MN, USA
| | - Joseph Maleszewski
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester MN, USA
| | - Marc Halushka
- Department of Pathology, Johns Hopkins University School of Medicine, Baltimore MD, USA
| | - Michael Seidman
- Department of Pathology, University Health Network, Toronto ON, CA
| | - Chieh-Yu Lin
- Department of Pathology and Immunology, Washington University, St. Louis MO, USA
| | - Maximilian Buja
- Department of Pathology and Laboratory Medicine, The University of Texas Health Science Center at Houston, Houston TX, USA
| | - James Stone
- Department of Pathology, Massachusetts General Hospital, Boston MA, USA
| | - David Dov
- Duke Division of Artificial Intelligence and Computational Pathology, Duke University Medical Center, Durham NC, USA; Pratt School of Engineering, Department of Electrical and Computer Engineering, Duke University, Durham NC, USA
| | - Lawrence Carin
- Duke Division of Artificial Intelligence and Computational Pathology, Duke University Medical Center, Durham NC, USA; Pratt School of Engineering, Department of Electrical and Computer Engineering, Duke University, Durham NC, USA
| | - Carolyn Glass
- Duke Division of Artificial Intelligence and Computational Pathology, Duke University Medical Center, Durham NC, USA; Department of Pathology, Duke University Medical Center, Durham NC, USA.
| |
Collapse
|
6
|
Gambella A, Salvi M, Molinari F. Reply to: "Application of digital pathology in liver transplantation". J Hepatol 2024; 81:e114-e115. [PMID: 38759888 DOI: 10.1016/j.jhep.2024.05.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/01/2024] [Accepted: 05/06/2024] [Indexed: 05/19/2024]
Affiliation(s)
- Alessandro Gambella
- Pathology Unit, Department of Medical Sciences, University of Turin, Turin, Italy; Division of Liver and Transplant Pathology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
| | - Massimo Salvi
- Department of Electronics and Telecommunications, PolitoBIOMed Lab, Politecnico di Torino, Biolab, Corso Duca degli Abruzzi 24, 10129 Turin, Italy
| | - Filippo Molinari
- Department of Electronics and Telecommunications, PolitoBIOMed Lab, Politecnico di Torino, Biolab, Corso Duca degli Abruzzi 24, 10129 Turin, Italy
| |
Collapse
|
7
|
Wen C, Bai X, Yang J, Li S, Wang X, Yang D. Deep learning based approach: automated gingival inflammation grading model using gingival removal strategy. Sci Rep 2024; 14:19780. [PMID: 39187553 PMCID: PMC11347620 DOI: 10.1038/s41598-024-70311-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Accepted: 08/14/2024] [Indexed: 08/28/2024] Open
Abstract
Gingival inflammation grade serves as a well-established index in periodontitis. The aim of this study was to develop a deep learning network utilizing a novel feature extraction method for the automatic assessment of gingival inflammation. T-distributed Stochastic Neighbor Embedding (t-SNE) was utilized for dimensionality reduction. A convolutional neural network (CNN) model based on DenseNet was developed for the identification and evaluation of gingival inflammation. To enhance the performance of the deep learning (DL) model, a novel teeth removal algorithm was implemented. Additionally, a Grad-CAM + + encoder was applied to generate heatmaps for computer visual attention analysis. The mean Intersection over Union (MIoU) for the identification of gingivitis was 0.727 ± 0.117. The accuracy rates for the five inflammatory degrees were 77.09%, 77.25%, 74.38%, 73.68% and 79.22%. The Area Under the Receiver Operating Characteristic (AUROC) values were 0.83, 0.80, 0.81, 0.81 and 0.84, respectively. The attention ratio towards gingival tissue increased from 37.73% to 62.20%, and within 8 mm of the gingival margin, it rose from 21.11% to 38.23%. On the gingiva, the overall attention ratio increased from 51.82% to 78.21%. The proposed DL model with novel feature extraction method provides high accuracy and sensitivity for identifying and grading gingival inflammation.
Collapse
Affiliation(s)
- Chang Wen
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, #237 Luoyu Road, Hongshan District, Wuhan, China
- Center for Orthodontics and Pediatric Dentistry at Optics Valley Branch, School & Hospital of Stomatology, Wuhan University, Wuhan, China
| | - Xueying Bai
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, #237 Luoyu Road, Hongshan District, Wuhan, China
| | - Jiaxin Yang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, #237 Luoyu Road, Hongshan District, Wuhan, China
| | - Sihong Li
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, #237 Luoyu Road, Hongshan District, Wuhan, China
| | - Xiaoxuan Wang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, #237 Luoyu Road, Hongshan District, Wuhan, China
- Department of Periodontology, School & Hospital of Stomatology, Wuhan University, Wuhan, China
| | - Dong Yang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, #237 Luoyu Road, Hongshan District, Wuhan, China.
- Department of Periodontology, School & Hospital of Stomatology, Wuhan University, Wuhan, China.
| |
Collapse
|
8
|
Elshamy R, Abu-Elnasr O, Elhoseny M, Elmougy S. Enhancing colorectal cancer histology diagnosis using modified deep neural networks optimizer. Sci Rep 2024; 14:19534. [PMID: 39174564 PMCID: PMC11341685 DOI: 10.1038/s41598-024-69193-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 08/01/2024] [Indexed: 08/24/2024] Open
Abstract
Optimizers are the bottleneck of the training process of any Convolutionolution neural networks (CNN) model. One of the critical steps when work on CNN model is choosing the optimal optimizer to solve a specific problem. Recent challenge in nowadays researches is building new versions of traditional CNN optimizers that can work more efficient than the traditional optimizers. Therefore, this work proposes a novel enhanced version of Adagrad optimizer called SAdagrad that avoids the drawbacks of Adagrad optimizer in dealing with tuning the learning rate value for each step of the training process. In order to evaluate SAdagrad, this paper builds a CNN model that combines a fine- tuning technique and a weight decay technique together. It trains the proposed CNN model on Kather colorectal cancer histology dataset which is one of the most challenging datasets in recent researches of Diagnose of Colorectal Cancer (CRC). In fact, recently, there have been plenty of deep learning models achieving successful results with regard to CRC classification experiments. However, the enhancement of these models remains challenging. To train our proposed model, a learning transfer process, which is adopted from a pre-complicated defined model is applied to the proposed model and combined it with a regularization technique that helps in avoiding overfitting. The experimental results show that SAdagrad reaches a remarkable accuracy (98%), when compared with Adaptive momentum optimizer (Adam) and Adagrad optimizer. The experiments also reveal that the proposed model has a more stable training and testing processes, can reduce the overfitting problem in multiple epochs and can achieve a higher accuracy compared with previous researches on Diagnosis CRC using the same Kather colorectal cancer histology dataset.
Collapse
Affiliation(s)
- Reham Elshamy
- Department of Computer Science, Faculty of Computers and Information, Mansoura University, Mansoura, 35516, Egypt.
| | - Osama Abu-Elnasr
- Department of Computer Science, Faculty of Computers and Information, Mansoura University, Mansoura, 35516, Egypt
| | - Mohamed Elhoseny
- Department of Information Systems, Faculty of Computers and Information, Mansoura University, Mansoura, 35516, Egypt
| | - Samir Elmougy
- Department of Computer Science, Faculty of Computers and Information, Mansoura University, Mansoura, 35516, Egypt
| |
Collapse
|
9
|
Dunenova G, Kalmataeva Z, Kaidarova D, Dauletbaev N, Semenova Y, Mansurova M, Grjibovski A, Kassymbekova F, Sarsembayev A, Semenov D, Glushkova N. The Performance and Clinical Applicability of HER2 Digital Image Analysis in Breast Cancer: A Systematic Review. Cancers (Basel) 2024; 16:2761. [PMID: 39123488 PMCID: PMC11311684 DOI: 10.3390/cancers16152761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2024] [Revised: 07/28/2024] [Accepted: 07/30/2024] [Indexed: 08/12/2024] Open
Abstract
This systematic review aims to address the research gap in the performance of computational algorithms for the digital image analysis of HER2 images in clinical settings. While numerous studies have explored various aspects of these algorithms, there is a lack of comprehensive evaluation regarding their effectiveness in real-world clinical applications. We conducted a search of the Web of Science and PubMed databases for studies published from 31 December 2013 to 30 June 2024, focusing on performance effectiveness and components such as dataset size, diversity and source, ground truth, annotation, and validation methods. The study was registered with PROSPERO (CRD42024525404). Key questions guiding this review include the following: How effective are current computational algorithms at detecting HER2 status in digital images? What are the common validation methods and dataset characteristics used in these studies? Is there standardization of algorithm evaluations of clinical applications that can improve the clinical utility and reliability of computational tools for HER2 detection in digital image analysis? We identified 6833 publications, with 25 meeting the inclusion criteria. The accuracy rate with clinical datasets varied from 84.19% to 97.9%. The highest accuracy was achieved on the publicly available Warwick dataset at 98.8% in synthesized datasets. Only 12% of studies used separate datasets for external validation; 64% of studies used a combination of accuracy, precision, recall, and F1 as a set of performance measures. Despite the high accuracy rates reported in these studies, there is a notable absence of direct evidence supporting their clinical application. To facilitate the integration of these technologies into clinical practice, there is an urgent need to address real-world challenges and overreliance on internal validation. Standardizing study designs on real clinical datasets can enhance the reliability and clinical applicability of computational algorithms in improving the detection of HER2 cancer.
Collapse
Affiliation(s)
- Gauhar Dunenova
- Department of Epidemiology, Biostatistics and Evidence-Based Medicine, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
| | - Zhanna Kalmataeva
- Rector Office, Asfendiyarov Kazakh National Medical University, Almaty 050000, Kazakhstan;
| | - Dilyara Kaidarova
- Kazakh Research Institute of Oncology and Radiology, Almaty 050022, Kazakhstan;
| | - Nurlan Dauletbaev
- Department of Internal, Respiratory and Critical Care Medicine, Philipps University of Marburg, 35037 Marburg, Germany;
- Department of Pediatrics, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC H4A 3J1, Canada
- Faculty of Medicine and Health Care, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
| | - Yuliya Semenova
- School of Medicine, Nazarbayev University, Astana 010000, Kazakhstan;
| | - Madina Mansurova
- Department of Artificial Intelligence and Big Data, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan;
| | - Andrej Grjibovski
- Central Scientific Research Laboratory, Northern State Medical University, Arkhangelsk 163000, Russia;
- Department of Epidemiology and Modern Vaccination Technologies, I.M. Sechenov First Moscow State Medical University, Moscow 105064, Russia
- Department of Biology, Ecology and Biotechnology, Northern (Arctic) Federal University, Arkhangelsk 163000, Russia
- Department of Health Policy and Management, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
| | - Fatima Kassymbekova
- Department of Public Health and Social Sciences, Kazakhstan Medical University “KSPH”, Almaty 050060, Kazakhstan;
| | - Aidos Sarsembayev
- School of Digital Technologies, Almaty Management University, Almaty 050060, Kazakhstan;
- Health Research Institute, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan;
| | - Daniil Semenov
- Computer Science and Engineering Program, Astana IT University, Astana 020000, Kazakhstan;
| | - Natalya Glushkova
- Department of Epidemiology, Biostatistics and Evidence-Based Medicine, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan
- Health Research Institute, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan;
| |
Collapse
|
10
|
Silva AB, Martins AS, Tosta TAA, Loyola AM, Cardoso SV, Neves LA, de Faria PR, do Nascimento MZ. OralEpitheliumDB: A Dataset for Oral Epithelial Dysplasia Image Segmentation and Classification. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1691-1710. [PMID: 38409608 DOI: 10.1007/s10278-024-01041-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 02/03/2024] [Accepted: 02/06/2024] [Indexed: 02/28/2024]
Abstract
Early diagnosis of potentially malignant disorders, such as oral epithelial dysplasia, is the most reliable way to prevent oral cancer. Computational algorithms have been used as an auxiliary tool to aid specialists in this process. Usually, experiments are performed on private data, making it difficult to reproduce the results. There are several public datasets of histological images, but studies focused on oral dysplasia images use inaccessible datasets. This prevents the improvement of algorithms aimed at this lesion. This study introduces an annotated public dataset of oral epithelial dysplasia tissue images. The dataset includes 456 images acquired from 30 mouse tongues. The images were categorized among the lesion grades, with nuclear structures manually marked by a trained specialist and validated by a pathologist. Also, experiments were carried out in order to illustrate the potential of the proposed dataset in classification and segmentation processes commonly explored in the literature. Convolutional neural network (CNN) models for semantic and instance segmentation were employed on the images, which were pre-processed with stain normalization methods. Then, the segmented and non-segmented images were classified with CNN architectures and machine learning algorithms. The data obtained through these processes is available in the dataset. The segmentation stage showed the F1-score value of 0.83, obtained with the U-Net model using the ResNet-50 as a backbone. At the classification stage, the most expressive result was achieved with the Random Forest method, with an accuracy value of 94.22%. The results show that the segmentation contributed to the classification results, but studies are needed for the improvement of these stages of automated diagnosis. The original, gold standard, normalized, and segmented images are publicly available and may be used for the improvement of clinical applications of CAD methods on oral epithelial dysplasia tissue images.
Collapse
Affiliation(s)
- Adriano Barbosa Silva
- Faculty of Computer Science (FACOM) - Federal University of Uberlândia (UFU), Av. João Naves de Ávila 2121, BLB, 38400-902, Uberlândia, MG, Brazil.
| | - Alessandro Santana Martins
- Federal Institute of Triângulo Mineiro (IFTM), R. Belarmino Vilela Junqueira, S/N, 38305-200, Ituiutaba, MG, Brazil
| | - Thaína Aparecida Azevedo Tosta
- Science and Technology Institute, Federal University of São Paulo (UNIFESP), Av. Cesare Mansueto Giulio Lattes, 1201, 12247-014, São José dos Campos, SP, Brazil
| | - Adriano Mota Loyola
- School of Dentistry, Federal University of Uberlândia (UFU), Av. Pará - 1720, 38405-320, Uberlândia, MG, Brazil
| | - Sérgio Vitorino Cardoso
- School of Dentistry, Federal University of Uberlândia (UFU), Av. Pará - 1720, 38405-320, Uberlândia, MG, Brazil
| | - Leandro Alves Neves
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), R. Cristóvão Colombo, 2265, 38305-200, São José do Rio Preto, SP, Brazil
| | - Paulo Rogério de Faria
- Department of Histology and Morphology, Institute of Biomedical Science, Federal University of Uberlândia (UFU), Av. Amazonas, S/N, 38405-320, Uberlândia, MG, Brazil
| | - Marcelo Zanchetta do Nascimento
- Faculty of Computer Science (FACOM) - Federal University of Uberlândia (UFU), Av. João Naves de Ávila 2121, BLB, 38400-902, Uberlândia, MG, Brazil
| |
Collapse
|
11
|
Kim A, Yoshida K, Kovacs GG, Forrest SL. Computer-Based Evaluation of α-Synuclein Pathology in Multiple System Atrophy as a Novel Tool to Recognize Disease Subtypes. Mod Pathol 2024; 37:100533. [PMID: 38852813 DOI: 10.1016/j.modpat.2024.100533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Revised: 04/11/2024] [Accepted: 05/27/2024] [Indexed: 06/11/2024]
Abstract
Multiple system atrophy (MSA) is a neurodegenerative disorder with variable disease course and distinct constellations of clinical (cerebellar [MSA-C] or parkinsonism [MSA-P]) and pathological phenotypes, suggestive of distinct α-synuclein (αSyn) strains. Neuropathologically, MSA is characterized by the accumulation of αSyn in oligodendrocytic glial cytoplasmic inclusions (GCI). Using a novel computer-based method, this study quantified the size of GCIs, density of all αSyn pathology, density of only the GCIs, and number of GCIs in MSA cases (n = 20). The putamen and cerebellar white matter were immunostained with the disease-associated 5G4 anti-αSyn antibody. Following digital scanning and image processing, total 5G4-immunoreactive pathology (ie, neuronal, neuritic, and glial) and GCIs were optically dissected for inclusion size and density measurement and then evaluated applying a novel computer-based method using ImageJ. GCI size varied between cases and brain regions (P < .0001), and heterogeneity in the density of all αSyn pathology including the density and number of GCIs were observed between regions and across cases, where MSA-C cases had a significantly higher density of all αSyn pathology in the cerebellar white matter (P = .049). Some region-specific morphologic variables inversely correlated with the age of onset and death, suggestive of an underlying aging-related cellular mechanism. Unsupervised K-means cluster analysis classified MSA cases into 3 distinct groups based on region-specific morphologic variables. In conclusion, we developed a novel computer-based method that is easily accessible, providing a first step to developing artificial intelligence-based evaluation strategies for large scale comparative studies. Our observations on the variability of morphologic variables between brain regions and cases highlight (1) the importance of computer-based approaches to detect features not considered in the routine diagnostic practice, and (2) novel aspects for the identification of previously unrecognized MSA subtypes that do not necessarily reflect the current clinical classification of MSA-C or MSA-P.
Collapse
Affiliation(s)
- Ain Kim
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada; Tanz Centre for Research in Neurodegenerative Disease, University of Toronto, Toronto, Ontario, Canada
| | - Koji Yoshida
- Tanz Centre for Research in Neurodegenerative Disease, University of Toronto, Toronto, Ontario, Canada; Department of Legal Medicine, Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama, Toyama, Toyama, Japan
| | - Gabor G Kovacs
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada; Tanz Centre for Research in Neurodegenerative Disease, University of Toronto, Toronto, Ontario, Canada; Krembil Brain Institute, University Health Network, Toronto, Ontario, Canada
| | - Shelley L Forrest
- Tanz Centre for Research in Neurodegenerative Disease, University of Toronto, Toronto, Ontario, Canada; Krembil Brain Institute, University Health Network, Toronto, Ontario, Canada.
| |
Collapse
|
12
|
Liu W, Zhang Q, Li Q, Wang S. Contrastive and uncertainty-aware nuclei segmentation and classification. Comput Biol Med 2024; 178:108667. [PMID: 38850962 DOI: 10.1016/j.compbiomed.2024.108667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 04/18/2024] [Accepted: 05/26/2024] [Indexed: 06/10/2024]
Abstract
Nuclei segmentation and classification play a crucial role in pathology diagnosis, enabling pathologists to analyze cellular characteristics accurately. Overlapping cluster nuclei, misdetection of small-scale nuclei, and pleomorphic nuclei-induced misclassification have always been major challenges in the nuclei segmentation and classification tasks. To this end, we introduce an auxiliary task of nuclei boundary-guided contrastive learning to enhance the representativeness and discriminative power of visual features, particularly for addressing the challenge posed by the unclear contours of adherent nuclei and small nuclei. In addition, misclassifications resulting from pleomorphic nuclei often exhibit low classification confidence, indicating a high level of uncertainty. To mitigate misclassification, we capitalize on the characteristic clustering of similar cells to propose a locality-aware class embedding module, offering a regional perspective to capture category information. Moreover, we address uncertain classification in densely aggregated nuclei by designing a top-k uncertainty attention module that leverages deep features to enhance shallow features, thereby improving the learning of contextual semantic information. We demonstrate that the proposed network outperforms the off-the-shelf methods in both nuclei segmentation and classification experiments, achieving the state-of-the-art performance.
Collapse
Affiliation(s)
- Wenxi Liu
- College of Computer and Data Science, Fuzhou University, Fuzhou, 350108, China.
| | - Qing Zhang
- College of Computer and Data Science, Fuzhou University, Fuzhou, 350108, China.
| | - Qi Li
- College of Computer and Data Science, Fuzhou University, Fuzhou, 350108, China.
| | - Shu Wang
- College of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China.
| |
Collapse
|
13
|
Yuan L, Shen Z, Shan Y, Zhu J, Wang Q, Lu Y, Shi H. Unveiling the landscape of pathomics in personalized immunotherapy for lung cancer: a bibliometric analysis. Front Oncol 2024; 14:1432212. [PMID: 39040448 PMCID: PMC11260632 DOI: 10.3389/fonc.2024.1432212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2024] [Accepted: 06/19/2024] [Indexed: 07/24/2024] Open
Abstract
Background Pathomics has emerged as a promising biomarker that could facilitate personalized immunotherapy in lung cancer. It is essential to elucidate the global research trends and emerging prospects in this domain. Methods The annual distribution, journals, authors, countries, institutions, and keywords of articles published between 2018 and 2023 were visualized and analyzed using CiteSpace and other bibliometric tools. Results A total of 109 relevant articles or reviews were included, demonstrating an overall upward trend; The terms "deep learning", "tumor microenvironment", "biomarkers", "image analysis", "immunotherapy", and "survival prediction", etc. are hot keywords in this field. Conclusion In future research endeavors, advanced methodologies involving artificial intelligence and pathomics will be deployed for the digital analysis of tumor tissues and the tumor microenvironment in lung cancer patients, leveraging histopathological tissue sections. Through the integration of comprehensive multi-omics data, this strategy aims to enhance the depth of assessment, characterization, and understanding of the tumor microenvironment, thereby elucidating a broader spectrum of tumor features. Consequently, the development of a multimodal fusion model will ensue, enabling precise evaluation of personalized immunotherapy efficacy and prognosis for lung cancer patients, potentially establishing a pivotal frontier in this domain of investigation.
Collapse
Affiliation(s)
- Lei Yuan
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| | - Zhiming Shen
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| | - Yibo Shan
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| | - Jianwei Zhu
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| | - Qi Wang
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| | - Yi Lu
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| | - Hongcan Shi
- Department of Thoracic Surgery, Northern Jiangsu People’s Hospital Affiliated to Yangzhou University, Yangzhou, China
- Institute of Translational Medicine, Medical College, Yangzhou University, Yangzhou, China
- Jiangsu Key Laboratory of Integrated Traditional Chinese and Western Medicine for Prevention and Treatment of Senile Diseases, Yangzhou University, Yangzhou, China
| |
Collapse
|
14
|
Ye Y, Xia L, Yang S, Luo Y, Tang Z, Li Y, Han L, Xie H, Ren Y, Na N. Deep learning-enabled classification of kidney allograft rejection on whole slide histopathologic images. Front Immunol 2024; 15:1438247. [PMID: 39034991 PMCID: PMC11257957 DOI: 10.3389/fimmu.2024.1438247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2024] [Accepted: 06/21/2024] [Indexed: 07/23/2024] Open
Abstract
Background Diagnosis of kidney transplant rejection currently relies on manual histopathological assessment, which is subjective and susceptible to inter-observer variability, leading to limited reproducibility. We aim to develop a deep learning system for automated assessment of whole-slide images (WSIs) from kidney allograft biopsies to enable detection and subtyping of rejection and to predict the prognosis of rejection. Method We collected H&E-stained WSIs of kidney allograft biopsies at 400x magnification from January 2015 to September 2023 at two hospitals. These biopsy specimens were classified as T cell-mediated rejection, antibody-mediated rejection, and other lesions based on the consensus reached by two experienced transplant pathologists. To achieve feature extraction, feature aggregation, and global classification, we employed multi-instance learning and common convolution neural networks (CNNs). The performance of the developed models was evaluated using various metrics, including confusion matrix, receiver operating characteristic curves, the area under the curve (AUC), classification map, heat map, and pathologist-machine confrontations. Results In total, 906 WSIs from 302 kidney allograft biopsies were included for analysis. The model based on multi-instance learning enables detection and subtyping of rejection, named renal rejection artificial intelligence model (RRAIM), with the overall 3-category AUC of 0.798 in the independent test set, which is superior to that of three transplant pathologists under nearly routine assessment conditions. Moreover, the prognosis models accurately predicted graft loss within 1 year following rejection and treatment response for rejection, achieving AUC of 0.936 and 0.756, respectively. Conclusion We first developed deep-learning models utilizing multi-instance learning for the detection and subtyping of rejection and prediction of rejection prognosis in kidney allograft biopsies. These models performed well and may be useful in assisting the pathological diagnosis.
Collapse
Affiliation(s)
- Yongrong Ye
- Department of Kidney Transplantation, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Liubing Xia
- Department of Kidney Transplantation, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Shicong Yang
- Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - You Luo
- Department of Kidney Transplantation, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Zuofu Tang
- Department of Kidney Transplantation, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Yuanqing Li
- School of Automation Science and Engineering, South China University of Technology, Guangzhou, China
- Research Center for Brain-Computer Interface, Pazhou Lab, Guangzhou, China
| | - Lanqing Han
- Center for Artificial Intelligence in Medicine, Research Institute of Tsinghua, Pearl River Delta, Guangzhou, China
| | - Hanbin Xie
- Department of Anesthesiology, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Yong Ren
- Scientific Research Project Department, Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou), Pazhou Lab, Guangzhou, China
- Shensi lab, Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China (UESTC), Shenzhen, China
- The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, China
| | - Ning Na
- Department of Kidney Transplantation, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
15
|
Janowczyk A, Zlobec I, Walker C, Berezowska S, Huschauer V, Tinguely M, Kupferschmid J, Mallet T, Merkler D, Kreutzfeldt M, Gasic R, Rau TT, Mazzucchelli L, Eyberg I, Cathomas G, Mertz KD, Koelzer VH, Soldini D, Jochum W, Rössle M, Henkel M, Grobholz R. Swiss digital pathology recommendations: results from a Delphi process conducted by the Swiss Digital Pathology Consortium of the Swiss Society of Pathology. Virchows Arch 2024; 485:13-30. [PMID: 38112792 PMCID: PMC11271322 DOI: 10.1007/s00428-023-03712-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 10/23/2023] [Accepted: 11/04/2023] [Indexed: 12/21/2023]
Abstract
Integration of digital pathology (DP) into clinical diagnostic workflows is increasingly receiving attention as new hardware and software become available. To facilitate the adoption of DP, the Swiss Digital Pathology Consortium (SDiPath) organized a Delphi process to produce a series of recommendations for DP integration within Swiss clinical environments. This process saw the creation of 4 working groups, focusing on the various components of a DP system (1) scanners, quality assurance and validation of scans, (2) integration of Whole Slide Image (WSI)-scanners and DP systems into the Pathology Laboratory Information System, (3) digital workflow-compliance with general quality guidelines, and (4) image analysis (IA)/artificial intelligence (AI), with topic experts for each recruited for discussion and statement generation. The work product of the Delphi process is 83 consensus statements presented here, forming the basis for "SDiPath Recommendations for Digital Pathology". They represent an up-to-date resource for national and international hospitals, researchers, device manufacturers, algorithm developers, and all supporting fields, with the intent of providing expectations and best practices to help ensure safe and efficient DP usage.
Collapse
Affiliation(s)
- Andrew Janowczyk
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, USA.
- Department of Oncology, Division of Precision Oncology, Geneva University Hospitals, Geneva, Switzerland.
- Department of Diagnostics, Division of Clinical Pathology, Geneva University Hospitals, Geneva, Switzerland.
| | - Inti Zlobec
- Institute for Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Cedric Walker
- Institute of Animal Pathology, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| | - Sabina Berezowska
- Institute of Pathology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | | | - Marianne Tinguely
- Institute of Pathology Enge, Zurich, Switzerland
- Medical Faculty, University of Zürich, Zurich, Switzerland
| | | | - Thomas Mallet
- Division of Clinical Pathology, Geneva University Hospital, Geneva, Switzerland
| | - Doron Merkler
- Division of Clinical Pathology, Geneva University Hospital, Geneva, Switzerland
- Department of Pathology and Immunology, University of Geneva, Geneva, Switzerland
| | - Mario Kreutzfeldt
- Division of Clinical Pathology, Geneva University Hospital, Geneva, Switzerland
- Department of Pathology and Immunology, University of Geneva, Geneva, Switzerland
| | | | - Tilman T Rau
- Institute of Pathology, University Hospital and Heinrich-Heine University, Düsseldorf, Germany
- Institute of Pathology, University of Bern, Bern, Switzerland
| | | | - Isgard Eyberg
- Institute of Pathology, Cantonal Hospital Aarau, Aarau, Switzerland
| | - Gieri Cathomas
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Kirsten D Mertz
- Institute of Pathology, Cantonal Hospital Baselland, Liestal, Switzerland
| | - Viktor H Koelzer
- Department of Pathology and Molecular Pathology, University and University Hospital of Zürich, Zurich, Switzerland
| | | | - Wolfram Jochum
- Institute of Pathology, Cantonal Hospital St. Gallen, St. Gallen, Switzerland
| | - Matthias Rössle
- Pathologie Luzerner Kantonsspital (Pathology Cantonal Hospital Lucerne), Spitalstrasse, Switzerland
| | - Maurice Henkel
- Research & Analytic Services University Hospital Basel, Basel, Switzerland
- Institute of Radiology, University Hospital Basel, Basel, Switzerland
- University of Basel, Basel, Switzerland
| | - Rainer Grobholz
- Medical Faculty, University of Zürich, Zurich, Switzerland
- Institute of Pathology, Cantonal Hospital Aarau, Aarau, Switzerland
| |
Collapse
|
16
|
Singh M, Kumar A, Khanna NN, Laird JR, Nicolaides A, Faa G, Johri AM, Mantella LE, Fernandes JFE, Teji JS, Singh N, Fouda MM, Singh R, Sharma A, Kitas G, Rathore V, Singh IM, Tadepalli K, Al-Maini M, Isenovic ER, Chaturvedi S, Garg D, Paraskevas KI, Mikhailidis DP, Viswanathan V, Kalra MK, Ruzsa Z, Saba L, Laine AF, Bhatt DL, Suri JS. Artificial intelligence for cardiovascular disease risk assessment in personalised framework: a scoping review. EClinicalMedicine 2024; 73:102660. [PMID: 38846068 PMCID: PMC11154124 DOI: 10.1016/j.eclinm.2024.102660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Revised: 04/25/2024] [Accepted: 05/08/2024] [Indexed: 06/09/2024] Open
Abstract
Background The field of precision medicine endeavors to transform the healthcare industry by advancing individualised strategies for diagnosis, treatment modalities, and predictive assessments. This is achieved by utilizing extensive multidimensional biological datasets encompassing diverse components, such as an individual's genetic makeup, functional attributes, and environmental influences. Artificial intelligence (AI) systems, namely machine learning (ML) and deep learning (DL), have exhibited remarkable efficacy in predicting the potential occurrence of specific cancers and cardiovascular diseases (CVD). Methods We conducted a comprehensive scoping review guided by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. Our search strategy involved combining key terms related to CVD and AI using the Boolean operator AND. In August 2023, we conducted an extensive search across reputable scholarly databases including Google Scholar, PubMed, IEEE Xplore, ScienceDirect, Web of Science, and arXiv to gather relevant academic literature on personalised medicine for CVD. Subsequently, in January 2024, we extended our search to include internet search engines such as Google and various CVD websites. These searches were further updated in March 2024. Additionally, we reviewed the reference lists of the final selected research articles to identify any additional relevant literature. Findings A total of 2307 records were identified during the process of conducting the study, consisting of 564 entries from external sites like arXiv and 1743 records found through database searching. After 430 duplicate articles were eliminated, 1877 items that remained were screened for relevancy. In this stage, 1241 articles remained for additional review after 158 irrelevant articles and 478 articles with insufficient data were removed. 355 articles were eliminated for being inaccessible, 726 for being written in a language other than English, and 281 for not having undergone peer review. Consequently, 121 studies were deemed suitable for inclusion in the qualitative synthesis. At the intersection of CVD, AI, and precision medicine, we found important scientific findings in our scoping review. Intricate pattern extraction from large, complicated genetic datasets is a skill that AI algorithms excel at, allowing for accurate disease diagnosis and CVD risk prediction. Furthermore, these investigations have uncovered unique genetic biomarkers linked to CVD, providing insight into the workings of the disease and possible treatment avenues. The construction of more precise predictive models and personalised treatment plans based on the genetic profiles of individual patients has been made possible by the revolutionary advancement of CVD risk assessment through the integration of AI and genomics. Interpretation The systematic methodology employed ensured the thorough examination of available literature and the inclusion of relevant studies, contributing to the robustness and reliability of the study's findings. Our analysis stresses a crucial point in terms of the adaptability and versatility of AI solutions. AI algorithms designed in non-CVD domains such as in oncology, often include ideas and tactics that might be modified to address cardiovascular problems. Funding No funding received.
Collapse
Affiliation(s)
- Manasvi Singh
- Stroke Monitoring and Diagnostic Division, AtheroPoint™, Roseville, CA, 95661, USA
- Bennett University, 201310, Greater Noida, India
| | - Ashish Kumar
- Bennett University, 201310, Greater Noida, India
| | - Narendra N. Khanna
- Department of Cardiology, Indraprastha APOLLO Hospitals, New Delhi, 110001, India
| | - John R. Laird
- Heart and Vascular Institute, Adventist Health St. Helena, St Helena, CA, 94574, USA
| | - Andrew Nicolaides
- Vascular Screening and Diagnostic Centre and University of Nicosia Medical School, Cyprus
| | - Gavino Faa
- Department of Pathology, University of Cagliari, Cagliari, Italy
| | - Amer M. Johri
- Department of Medicine, Division of Cardiology, Queen's University, Kingston, Canada
| | - Laura E. Mantella
- Department of Medicine, Division of Cardiology, University of Toronto, Toronto, Canada
| | | | - Jagjit S. Teji
- Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, 60611, USA
| | - Narpinder Singh
- Department of Food Science and Technology, Graphic Era Deemed to Be University, Dehradun, Uttarakhand, 248002, India
| | - Mostafa M. Fouda
- Department of Electrical and Computer Engineering, Idaho State University, Pocatello, ID, 83209, USA
| | - Rajesh Singh
- Department of Research and Innovation, Uttaranchal Institute of Technology, Uttaranchal University, Dehradun, 248007, India
| | - Aditya Sharma
- Division of Cardiovascular Medicine, University of Virginia, Charlottesville, 22901, VA, USA
| | - George Kitas
- Academic Affairs, Dudley Group NHS Foundation Trust, DY1, Dudley, UK
| | - Vijay Rathore
- Nephrology Department, Kaiser Permanente, Sacramento, CA, 95823, USA
| | - Inder M. Singh
- Stroke Monitoring and Diagnostic Division, AtheroPoint™, Roseville, CA, 95661, USA
| | | | - Mustafa Al-Maini
- Allergy, Clinical Immunology and Rheumatology Institute, Toronto, ON, L4Z 4C4, Canada
| | - Esma R. Isenovic
- Department of Radiobiology and Molecular Genetics, National Institute of The Republic of Serbia, University of Belgrade, 110010, Serbia
| | - Seemant Chaturvedi
- Department of Neurology & Stroke Program, University of Maryland, Baltimore, MD, USA
| | | | | | - Dimitri P. Mikhailidis
- Department of Clinical Biochemistry, Royal Free Hospital Campus, University College London Medical School, University College London (UCL), London, UK
| | | | | | - Zoltan Ruzsa
- Invasive Cardiology Division, University of Szeged, Szeged, Hungary
| | - Luca Saba
- Department of Radiology, Azienda Ospedaliero Universitaria, 40138, Cagliari, Italy
| | - Andrew F. Laine
- Departments of Biomedical and Radiology, Columbia University, New York, NY, USA
| | | | - Jasjit S. Suri
- Stroke Monitoring and Diagnostic Division, AtheroPoint™, Roseville, CA, 95661, USA
- Department of Electrical and Computer Engineering, Idaho State University, Pocatello, ID, 83209, USA
- Department of Computer Science, Graphic Era Deemed to Be University, Dehradun, Uttarakhand, 248002, India
| |
Collapse
|
17
|
Chen J, Yang J, Wang J, Zhao Z, Wang M, Sun C, Song N, Feng S. Study on an Automatic Classification Method for Determining the Malignancy Grade of Glioma Pathological Sections Based on Hyperspectral Multi-Scale Spatial-Spectral Fusion Features. SENSORS (BASEL, SWITZERLAND) 2024; 24:3803. [PMID: 38931588 PMCID: PMC11207485 DOI: 10.3390/s24123803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/12/2024] [Revised: 06/05/2024] [Accepted: 06/11/2024] [Indexed: 06/28/2024]
Abstract
This study describes a novel method for grading pathological sections of gliomas. Our own integrated hyperspectral imaging system was employed to characterize 270 bands of cancerous tissue samples from microarray slides of gliomas. These samples were then classified according to the guidelines developed by the World Health Organization, which define the subtypes and grades of diffuse gliomas. We explored a hyperspectral feature extraction model called SMLMER-ResNet using microscopic hyperspectral images of brain gliomas of different malignancy grades. The model combines the channel attention mechanism and multi-scale image features to automatically learn the pathological organization of gliomas and obtain hierarchical feature representations, effectively removing the interference of redundant information. It also completes multi-modal, multi-scale spatial-spectral feature extraction to improve the automatic classification of glioma subtypes. The proposed classification method demonstrated high average classification accuracy (>97.3%) and a Kappa coefficient (0.954), indicating its effectiveness in improving the automatic classification of hyperspectral gliomas. The method is readily applicable in a wide range of clinical settings, offering valuable assistance in alleviating the workload of clinical pathologists. Furthermore, the study contributes to the development of more personalized and refined treatment plans, as well as subsequent follow-up and treatment adjustment, by providing physicians with insights into the underlying pathological organization of gliomas.
Collapse
Affiliation(s)
- Jiaqi Chen
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
- University of Chinese Academy of Sciences, Beijing 130033, China
| | - Jin Yang
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
| | - Jinyu Wang
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
- University of Chinese Academy of Sciences, Beijing 130033, China
| | - Zitong Zhao
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
- University of Chinese Academy of Sciences, Beijing 130033, China
| | - Mingjia Wang
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
| | - Ci Sun
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
| | - Nan Song
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
| | - Shulong Feng
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; (J.C.)
| |
Collapse
|
18
|
Ruiz-Casado JL, Molina-Cabello MA, Luque-Baena RM. Enhancing Histopathological Image Classification Performance through Synthetic Data Generation with Generative Adversarial Networks. SENSORS (BASEL, SWITZERLAND) 2024; 24:3777. [PMID: 38931561 PMCID: PMC11207853 DOI: 10.3390/s24123777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2024] [Revised: 06/08/2024] [Accepted: 06/09/2024] [Indexed: 06/28/2024]
Abstract
Breast cancer is the second most common cancer worldwide, primarily affecting women, while histopathological image analysis is one of the possibile methods used to determine tumor malignancy. Regarding image analysis, the application of deep learning has become increasingly prevalent in recent years. However, a significant issue is the unbalanced nature of available datasets, with some classes having more images than others, which may impact the performance of the models due to poorer generalizability. A possible strategy to avoid this problem is downsampling the class with the most images to create a balanced dataset. Nevertheless, this approach is not recommended for small datasets as it can lead to poor model performance. Instead, techniques such as data augmentation are traditionally used to address this issue. These techniques apply simple transformations such as translation or rotation to the images to increase variability in the dataset. Another possibility is using generative adversarial networks (GANs), which can generate images from a relatively small training set. This work aims to enhance model performance in classifying histopathological images by applying data augmentation using GANs instead of traditional techniques.
Collapse
Affiliation(s)
- Jose L. Ruiz-Casado
- ITIS Software, University of Málaga, C/ Arquitecto Francisco Peñalosa, 18, 29010 Malaga, Spain; (J.L.R.-C.); (M.A.M.-C.)
| | - Miguel A. Molina-Cabello
- ITIS Software, University of Málaga, C/ Arquitecto Francisco Peñalosa, 18, 29010 Malaga, Spain; (J.L.R.-C.); (M.A.M.-C.)
- Instituto de Investigación Biomédica de Málaga y Plataforma en Nanomedicina-IBIMA Plataforma BIONAND, Avenida Severo Ochoa, 35, 29590 Malaga, Spain
| | - Rafael M. Luque-Baena
- ITIS Software, University of Málaga, C/ Arquitecto Francisco Peñalosa, 18, 29010 Malaga, Spain; (J.L.R.-C.); (M.A.M.-C.)
- Instituto de Investigación Biomédica de Málaga y Plataforma en Nanomedicina-IBIMA Plataforma BIONAND, Avenida Severo Ochoa, 35, 29590 Malaga, Spain
| |
Collapse
|
19
|
Ou DX, Lu CW, Chen LW, Lee WY, Hu HW, Chuang JH, Lin MW, Chen KY, Chiu LY, Chen JS, Chen CM, Hsieh MS. Deep Learning Analysis for Predicting Tumor Spread through Air Space in Early-Stage Lung Adenocarcinoma Pathology Images. Cancers (Basel) 2024; 16:2132. [PMID: 38893251 PMCID: PMC11172106 DOI: 10.3390/cancers16112132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Revised: 05/25/2024] [Accepted: 06/01/2024] [Indexed: 06/21/2024] Open
Abstract
The presence of spread through air spaces (STASs) in early-stage lung adenocarcinoma is a significant prognostic factor associated with disease recurrence and poor outcomes. Although current STAS detection methods rely on pathological examinations, the advent of artificial intelligence (AI) offers opportunities for automated histopathological image analysis. This study developed a deep learning (DL) model for STAS prediction and investigated the correlation between the prediction results and patient outcomes. To develop the DL-based STAS prediction model, 1053 digital pathology whole-slide images (WSIs) from the competition dataset were enrolled in the training set, and 227 WSIs from the National Taiwan University Hospital were enrolled for external validation. A YOLOv5-based framework comprising preprocessing, candidate detection, false-positive reduction, and patient-based prediction was proposed for STAS prediction. The model achieved an area under the curve (AUC) of 0.83 in predicting STAS presence, with 72% accuracy, 81% sensitivity, and 63% specificity. Additionally, the DL model demonstrated a prognostic value in disease-free survival compared to that of pathological evaluation. These findings suggest that DL-based STAS prediction could serve as an adjunctive screening tool and facilitate clinical decision-making in patients with early-stage lung adenocarcinoma.
Collapse
Affiliation(s)
- De-Xiang Ou
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, Taipei 10617, Taiwan; (D.-X.O.); (L.-W.C.); (K.-Y.C.)
| | - Chao-Wen Lu
- Division of Thoracic Surgery, Department of Surgery, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei 100, Taiwan; (C.-W.L.); (J.-H.C.)
- Graduate Institute of Pathology, National Taiwan University College of Medicine, Taipei 100, Taiwan
| | - Li-Wei Chen
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, Taipei 10617, Taiwan; (D.-X.O.); (L.-W.C.); (K.-Y.C.)
| | - Wen-Yao Lee
- Division of Thoracic Surgery, Department of Surgery, Fu Jen Catholic University Hospital, No. 69, Guizi Road, Taishan District, New Taipei City 24352, Taiwan;
| | - Hsiang-Wei Hu
- Department of Pathology, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei 100, Taiwan;
| | - Jen-Hao Chuang
- Division of Thoracic Surgery, Department of Surgery, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei 100, Taiwan; (C.-W.L.); (J.-H.C.)
| | - Mong-Wei Lin
- Division of Thoracic Surgery, Department of Surgery, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei 100, Taiwan; (C.-W.L.); (J.-H.C.)
| | - Kuan-Yu Chen
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, Taipei 10617, Taiwan; (D.-X.O.); (L.-W.C.); (K.-Y.C.)
| | - Ling-Ying Chiu
- Institute of Medicine, Chung Shan Medical University, Taichung 40201, Taiwan;
| | - Jin-Shing Chen
- Division of Thoracic Surgery, Department of Surgery, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei 100, Taiwan; (C.-W.L.); (J.-H.C.)
| | - Chung-Ming Chen
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, Taipei 10617, Taiwan; (D.-X.O.); (L.-W.C.); (K.-Y.C.)
| | - Min-Shu Hsieh
- Graduate Institute of Pathology, National Taiwan University College of Medicine, Taipei 100, Taiwan
- Department of Pathology, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei 100, Taiwan;
| |
Collapse
|
20
|
Yüksel B, Özveren N, Yeşil Ç. Evaluation of Dental Plaque Area with Artificial Intelligence Model. Niger J Clin Pract 2024; 27:759-765. [PMID: 38943301 DOI: 10.4103/njcp.njcp_862_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Accepted: 03/06/2024] [Indexed: 07/01/2024]
Abstract
OBJECTIVES This study aims to assess the diagnostic accuracy of an artificial intelligence (AI) system employing deep learning for identifying dental plaque, utilizing a dataset comprising photographs of permanent teeth. MATERIALS AND METHODS In this study, photographs of 168 teeth belonging to 20 patients aged between 10 and 15 years, who met our criteria, were included. Intraoral photographs were taken of the patients in two stages, before and after the application of the plaque staining agent. To train the AI system to identify plaque on teeth with dental plaque that is not discolored, plaque and teeth were marked on photos with exposed dental plaque. One hundred forty teeth were used to construct the training group, while 28 teeth were used to create the test group. Another dentist reviewed images of teeth with dental plaque that was not discolored, and the effectiveness of AI in detecting plaque was evaluated using pertinent performance indicators. To compare the AI model and the dentist's evaluation outcomes, the mean intersection over union (IoU) values were evaluated by the Wilcoxon test. RESULTS The AI system showed higher performance in our study with a precision of 82% accuracy, 84% sensitivity, 83% F1 score, 87% accuracy, and 89% specificity in plaque detection. The area under the curve (AUC) value was found to be 0.922, and the IoU value was 76%. Subsequently, the dentist's plaque diagnosis performance was also evaluated. The IoU value was 0.71, and the AUC was 0.833. The AI model showed statistically significantly higher performance than the dentist (P < 0.05). CONCLUSIONS The AI algorithm that we developed has achieved promising results and demonstrated clinically acceptable performance in detecting dental plaque compared to a dentist.
Collapse
Affiliation(s)
- B Yüksel
- Department of Pediatric Dentistry, Faculty of Dentistry, Trakya University, Edirne, Turkey
| | - N Özveren
- Department of Pediatric Dentistry, Faculty of Dentistry, Trakya University, Edirne, Turkey
| | - Ç Yeşil
- Department of Computer Engineering, Faculty of Engineering, Yeditepe University, İstanbul, Turkey
| |
Collapse
|
21
|
Williams DKA, Graifman G, Hussain N, Amiel M, Tran P, Reddy A, Haider A, Kavitesh BK, Li A, Alishahian L, Perera N, Efros C, Babu M, Tharakan M, Etienne M, Babu BA. Digital pathology, deep learning, and cancer: a narrative review. Transl Cancer Res 2024; 13:2544-2560. [PMID: 38881914 PMCID: PMC11170525 DOI: 10.21037/tcr-23-964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Accepted: 03/24/2024] [Indexed: 06/18/2024]
Abstract
Background and Objective Cancer is a leading cause of morbidity and mortality worldwide. The emergence of digital pathology and deep learning technologies signifies a transformative era in healthcare. These technologies can enhance cancer detection, streamline operations, and bolster patient care. A substantial gap exists between the development phase of deep learning models in controlled laboratory environments and their translations into clinical practice. This narrative review evaluates the current landscape of deep learning and digital pathology, analyzing the factors influencing model development and implementation into clinical practice. Methods We searched multiple databases, including Web of Science, Arxiv, MedRxiv, BioRxiv, Embase, PubMed, DBLP, Google Scholar, IEEE Xplore, Semantic Scholar, and Cochrane, targeting articles on whole slide imaging and deep learning published from 2014 and 2023. Out of 776 articles identified based on inclusion criteria, we selected 36 papers for the analysis. Key Content and Findings Most articles in this review focus on the in-laboratory phase of deep learning model development, a critical stage in the deep learning lifecycle. Challenges arise during model development and their integration into clinical practice. Notably, lab performance metrics may not always match real-world clinical outcomes. As technology advances and regulations evolve, we expect more clinical trials to bridge this performance gap and validate deep learning models' effectiveness in clinical care. High clinical accuracy is vital for informed decision-making throughout a patient's cancer care. Conclusions Deep learning technology can enhance cancer detection, clinical workflows, and patient care. Challenges may arise during model development. The deep learning lifecycle involves data preprocessing, model development, and clinical implementation. Achieving health equity requires including diverse patient groups and eliminating bias during implementation. While model development is integral, most articles focus on the pre-deployment phase. Future longitudinal studies are crucial for validating models in real-world settings post-deployment. A collaborative approach among computational pathologists, technologists, industry, and healthcare providers is essential for driving adoption in clinical settings.
Collapse
Affiliation(s)
| | | | - Nowair Hussain
- Department of Internal Medicine, Overlook Medical Center, Summit, NJ, USA
| | | | | | - Arjun Reddy
- Applied Mathematics & Statistics Stony Brook University, Stony Brook, NY, USA
| | - Ali Haider
- Department of Artificial Intelligence, Yeshiva University, New York, NY, USA
| | - Bali Kumar Kavitesh
- Centre for Frontier AI Research (CFAR), Agency for Science, Technology, and Research (A*STAR), Singapore, Singapore
| | - Austin Li
- New York Medical College, Valhalla, NY, USA
| | | | | | | | - Myoungmee Babu
- Artificial Intelligence and Mathematics, New York City Department of Education, New York, NY, USA
| | | | - Mill Etienne
- Department of Neurology, New York Medical College, Valhalla, NY, USA
| | - Benson A Babu
- New York Medical College, Valhalla, NY, USA
- Department of Hospital Medicine, Wyckoff, Medical Center, New York, NY, USA
| |
Collapse
|
22
|
Fiorin A, López Pablo C, Lejeune M, Hamza Siraj A, Della Mea V. Enhancing AI Research for Breast Cancer: A Comprehensive Review of Tumor-Infiltrating Lymphocyte Datasets. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01043-8. [PMID: 38806950 DOI: 10.1007/s10278-024-01043-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 01/19/2024] [Accepted: 02/07/2024] [Indexed: 05/30/2024]
Abstract
The field of immunology is fundamental to our understanding of the intricate dynamics of the tumor microenvironment. In particular, tumor-infiltrating lymphocyte (TIL) assessment emerges as essential aspect in breast cancer cases. To gain comprehensive insights, the quantification of TILs through computer-assisted pathology (CAP) tools has become a prominent approach, employing advanced artificial intelligence models based on deep learning techniques. The successful recognition of TILs requires the models to be trained, a process that demands access to annotated datasets. Unfortunately, this task is hampered not only by the scarcity of such datasets, but also by the time-consuming nature of the annotation phase required to create them. Our review endeavors to examine publicly accessible datasets pertaining to the TIL domain and thereby become a valuable resource for the TIL community. The overall aim of the present review is thus to make it easier to train and validate current and upcoming CAP tools for TIL assessment by inspecting and evaluating existing publicly available online datasets.
Collapse
Affiliation(s)
- Alessio Fiorin
- Oncological Pathology and Bioinformatics Research Group, Institut d'Investigació Sanitària Pere Virgili (IISPV), C/Esplanetes no 14, 43500, Tortosa, Spain.
- Department of Pathology, Hospital de Tortosa Verge de la Cinta (HTVC), Institut Català de la Salut (ICS), C/Esplanetes no 14, 43500, Tortosa, Spain.
- Department of Computer Engineering and Mathematics, Universitat Rovira i Virgili (URV), Tarragona, Spain.
| | - Carlos López Pablo
- Oncological Pathology and Bioinformatics Research Group, Institut d'Investigació Sanitària Pere Virgili (IISPV), C/Esplanetes no 14, 43500, Tortosa, Spain.
- Department of Pathology, Hospital de Tortosa Verge de la Cinta (HTVC), Institut Català de la Salut (ICS), C/Esplanetes no 14, 43500, Tortosa, Spain.
- Department of Computer Engineering and Mathematics, Universitat Rovira i Virgili (URV), Tarragona, Spain.
| | - Marylène Lejeune
- Oncological Pathology and Bioinformatics Research Group, Institut d'Investigació Sanitària Pere Virgili (IISPV), C/Esplanetes no 14, 43500, Tortosa, Spain
- Department of Pathology, Hospital de Tortosa Verge de la Cinta (HTVC), Institut Català de la Salut (ICS), C/Esplanetes no 14, 43500, Tortosa, Spain
- Department of Computer Engineering and Mathematics, Universitat Rovira i Virgili (URV), Tarragona, Spain
| | - Ameer Hamza Siraj
- Department of Mathematics, Computer Science and Physics, University of Udine, Udine, Italy
| | - Vincenzo Della Mea
- Department of Mathematics, Computer Science and Physics, University of Udine, Udine, Italy
| |
Collapse
|
23
|
Baul S, Tanvir Ahmed K, Jiang Q, Wang G, Li Q, Yong J, Zhang W. Integrating spatial transcriptomics and bulk RNA-seq: predicting gene expression with enhanced resolution through graph attention networks. Brief Bioinform 2024; 25:bbae316. [PMID: 38960406 PMCID: PMC11221891 DOI: 10.1093/bib/bbae316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Revised: 06/04/2024] [Accepted: 06/17/2024] [Indexed: 07/05/2024] Open
Abstract
Spatial transcriptomics data play a crucial role in cancer research, providing a nuanced understanding of the spatial organization of gene expression within tumor tissues. Unraveling the spatial dynamics of gene expression can unveil key insights into tumor heterogeneity and aid in identifying potential therapeutic targets. However, in many large-scale cancer studies, spatial transcriptomics data are limited, with bulk RNA-seq and corresponding Whole Slide Image (WSI) data being more common (e.g. TCGA project). To address this gap, there is a critical need to develop methodologies that can estimate gene expression at near-cell (spot) level resolution from existing WSI and bulk RNA-seq data. This approach is essential for reanalyzing expansive cohort studies and uncovering novel biomarkers that have been overlooked in the initial assessments. In this study, we present STGAT (Spatial Transcriptomics Graph Attention Network), a novel approach leveraging Graph Attention Networks (GAT) to discern spatial dependencies among spots. Trained on spatial transcriptomics data, STGAT is designed to estimate gene expression profiles at spot-level resolution and predict whether each spot represents tumor or non-tumor tissue, especially in patient samples where only WSI and bulk RNA-seq data are available. Comprehensive tests on two breast cancer spatial transcriptomics datasets demonstrated that STGAT outperformed existing methods in accurately predicting gene expression. Further analyses using the TCGA breast cancer dataset revealed that gene expression estimated from tumor-only spots (predicted by STGAT) provides more accurate molecular signatures for breast cancer sub-type and tumor stage prediction, and also leading to improved patient survival and disease-free analysis. Availability: Code is available at https://github.com/compbiolabucf/STGAT.
Collapse
Affiliation(s)
- Sudipto Baul
- Department of Computer Science, University of Central Florida, Orlando, FL 32816, United States
| | - Khandakar Tanvir Ahmed
- Department of Computer Science, University of Central Florida, Orlando, FL 32816, United States
| | - Qibing Jiang
- Department of Computer Science, University of Central Florida, Orlando, FL 32816, United States
| | - Guangyu Wang
- Houston Methodist Research Institute, Weill Cornell Medical College, Houston, TX 77030, United States
| | - Qian Li
- Department of Biostatistics, St. Jude Children’s Research Hospital, Memphis, TN 38105, United States
| | - Jeongsik Yong
- Department of Biochemistry, Molecular Biology and Biophysics, University of Minnesota Twin Cities, Minneapolis, MN 55455, United States
| | - Wei Zhang
- Department of Computer Science, University of Central Florida, Orlando, FL 32816, United States
| |
Collapse
|
24
|
Baheti B, Innani S, Nasrallah M, Bakas S. Prognostic stratification of glioblastoma patients by unsupervised clustering of morphology patterns on whole slide images furthering our disease understanding. Front Neurosci 2024; 18:1304191. [PMID: 38831756 PMCID: PMC11146603 DOI: 10.3389/fnins.2024.1304191] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 04/25/2024] [Indexed: 06/05/2024] Open
Abstract
Introduction Glioblastoma (GBM) is a highly aggressive malignant tumor of the central nervous system that displays varying molecular and morphological profiles, leading to challenging prognostic assessments. Stratifying GBM patients according to overall survival (OS) from H&E-stained whole slide images (WSI) using advanced computational methods is challenging, but with direct clinical implications. Methods This work is focusing on GBM (IDH-wildtype, CNS WHO Gr.4) cases, identified from the TCGA-GBM and TCGA-LGG collections after considering the 2021 WHO classification criteria. The proposed approach starts with patch extraction in each WSI, followed by comprehensive patch-level curation to discard artifactual content, i.e., glass reflections, pen markings, dust on the slide, and tissue tearing. Each patch is then computationally described as a feature vector defined by a pre-trained VGG16 convolutional neural network. Principal component analysis provides a feature representation of reduced dimensionality, further facilitating identification of distinct groups of morphology patterns, via unsupervised k-means clustering. Results The optimal number of clusters, according to cluster reproducibility and separability, is automatically determined based on the rand index and silhouette coefficient, respectively. Our proposed approach achieved prognostic stratification accuracy of 83.33% on a multi-institutional independent unseen hold-out test set with sensitivity and specificity of 83.33%. Discussion We hypothesize that the quantification of these clusters of morphology patterns, reflect the tumor's spatial heterogeneity and yield prognostic relevant information to distinguish between short and long survivors using a decision tree classifier. The interpretability analysis of the obtained results can contribute to furthering and quantifying our understanding of GBM and potentially improving our diagnostic and prognostic predictions.
Collapse
Affiliation(s)
- Bhakti Baheti
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, United States
- Center for Artificial Intelligence and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, United States
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Shubham Innani
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, United States
- Center for Artificial Intelligence and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, United States
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - MacLean Nasrallah
- Center for Artificial Intelligence and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, United States
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Spyridon Bakas
- Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, United States
- Center for Artificial Intelligence and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, United States
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
- Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, Indianapolis, IN, United States
| |
Collapse
|
25
|
Chen J, Xue Y, Ren L, Lv K, Du P, Cheng H, Sun S, Hua L, Xie Q, Wu R, Gong Y. Predicting meningioma grades and pathologic marker expression via deep learning. Eur Radiol 2024; 34:2997-3008. [PMID: 37853176 DOI: 10.1007/s00330-023-10258-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Revised: 07/05/2023] [Accepted: 07/15/2023] [Indexed: 10/20/2023]
Abstract
OBJECTIVES To establish a deep learning (DL) model for predicting tumor grades and expression of pathologic markers of meningioma. METHODS A total of 1192 meningioma patients from two centers who underwent surgical resection between September 2018 and December 2021 were retrospectively included. The pathological data and post-contrast T1-weight images for each patient were collected. The patients from institute I were subdivided into training, validation, and testing sets, while the patients from institute II served as the external testing cohort. The fine-tuned ResNet50 model based on transfer learning was adopted to classify WHO grade in the whole cohort and predict Ki-67 index, H3K27me3, and progesterone receptor (PR) status of grade 1 meningiomas. The predictive performance was evaluated by the accuracy and loss curve, confusion matrix, receiver operating characteristic curve (ROC), and area under curve (AUC). RESULTS The DL prediction model for each label achieved high predictive performance in two cohorts. For WHO grade prediction, the area under the curve (AUC) was 0.966 (95%CI 0.957-0.975) in the internal testing set and 0.669 (95%CI 0.643-0.695) in the external validation cohort. The AUC in predicting Ki-67 index, H3K27me3, and PR status were 0.905 (95%CI 0.895-0.915), 0.773 (95%CI 0.760-0.786), and 0.771 (95%CI 0.750-0.792) in the internal testing set and 0.591 (95%CI 0.562-0.620), 0.658 (95%CI 0.648-0.668), and 0.703 (95%CI 0.674-0.732) in the external validation cohort, respectively. CONCLUSION DL models can preoperatively predict meningioma grades and pathologic marker expression with favorable predictive performance. CLINICAL RELEVANCE STATEMENT Our DL model could predict meningioma grades and expression of pathologic markers and identify high-risk patients with WHO grade 1 meningioma, which would suggest a more aggressive operative intervention preoperatively and a more frequent follow-up schedule postoperatively. KEY POINTS WHO grades and some pathologic markers of meningioma were associated with therapeutic strategies and clinical outcomes. A deep learning-based approach was employed to develop a model for predicting meningioma grades and the expression of pathologic markers. Preoperative prediction of meningioma grades and the expression of pathologic markers was beneficial for clinical decision-making.
Collapse
Affiliation(s)
- Jiawei Chen
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China
| | - Yanping Xue
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China
| | - Leihao Ren
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China
| | - Kun Lv
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, China
| | - Peng Du
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, China
| | - Haixia Cheng
- Department of Pathology, Huashan Hospital, Shanghai Medical College, Fudan University, Shanghai, China
| | - Shuchen Sun
- Department of Neurosurgery, Shanghai International Hospital, Shanghai, China
- Department of Neurosurgery, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Lingyang Hua
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China
| | - Qing Xie
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China.
| | - Ruiqi Wu
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China.
| | - Ye Gong
- Department of Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology, MOE Frontiers Center for Brain Science and Institutes of Brain Science, Fudan University, Shanghai, China.
- Department of Critical Care Medicine, Huashan Hospital, Shanghai Medical College, Fudan University, Shanghai, China.
| |
Collapse
|
26
|
Liu Q, Zhang X, Jiang X, Zhang C, Li J, Zhang X, Yang J, Yu N, Zhu Y, Liu J, Xie F, Li Y, Hao Y, Feng Y, Wang Q, Gao Q, Zhang W, Zhang T, Dong T, Cui B. A Histopathologic Image Analysis for the Classification of Endocervical Adenocarcinoma Silva Patterns Depend on Weakly Supervised Deep Learning. THE AMERICAN JOURNAL OF PATHOLOGY 2024; 194:735-746. [PMID: 38382842 DOI: 10.1016/j.ajpath.2024.01.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 12/25/2023] [Accepted: 01/18/2024] [Indexed: 02/23/2024]
Abstract
Twenty-five percent of cervical cancers are classified as endocervical adenocarcinomas (EACs), which comprise a highly heterogeneous group of tumors. A histopathologic risk stratification system known as the Silva pattern system was developed based on morphology. However, accurately classifying such patterns can be challenging. The study objective was to develop a deep learning pipeline (Silva3-AI) that automatically analyzes whole slide image-based histopathologic images and identifies Silva patterns with high accuracy. Initially, a total of 202 patients with EACs and histopathologic slides were obtained from Qilu Hospital of Shandong University for developing and internally testing the Silva3-AI model. Subsequently, an additional 161 patients and slides were collected from seven other medical centers for independent testing. The Silva3-AI model was developed using a vision transformer and recurrent neural network architecture, utilizing multi-magnification patches, and its performance was evaluated based on a class-specific area under the receiver-operating characteristic curve. Silva3-AI achieved a class-specific area under the receiver-operating characteristic curve of 0.947 for Silva A, 0.908 for Silva B, and 0.947 for Silva C on the independent test set. Notably, the performance of Silva3-AI was consistent with that of professional pathologists with 10 years' diagnostic experience. Furthermore, the visualization of prediction heatmaps facilitated the identification of tumor microenvironment heterogeneity, which is known to contribute to variations in Silva patterns.
Collapse
Affiliation(s)
- Qingqing Liu
- Cheeloo College of Medicine, Shandong University, Jinan City, China
| | - Xiaofang Zhang
- Department of Pathology, School of Basic Medical Sciences and Qilu Hospital, Shandong University, Jinan City, China
| | - Xuji Jiang
- Cheeloo College of Medicine, Shandong University, Jinan City, China
| | - Chunyan Zhang
- Department of Pathology, Affiliated Hospital of Jining Medical University of Shandong, Jining City, China
| | - Jiamei Li
- Department of Pathology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan City, China
| | - Xuedong Zhang
- Department of Pathology, Liaocheng People's Hospital, Liaocheng City, China
| | - Jingyan Yang
- Department of Pathology, The Second Hospital of Shandong University, Jinan City, China
| | - Ning Yu
- Department of Pathology, Binzhou Medical University Hospital, Binzhou City, China
| | - Yongcun Zhu
- Department of Pathology, Weihai Municipal Hospital of Shandong University, Weihai City, China
| | - Jing Liu
- Department of Pathology, Jining No. 1 People's Hospital, Jining City, China
| | - Fengxiang Xie
- Department of Pathology, KingMed Diagnostics, Jinan City, China
| | - Yawen Li
- Department of Pathology, School of Basic Medical Sciences and Qilu Hospital, Shandong University, Jinan City, China
| | - Yiping Hao
- Cheeloo College of Medicine, Shandong University, Jinan City, China
| | - Yuan Feng
- Cheeloo College of Medicine, Shandong University, Jinan City, China
| | - Qi Wang
- Department of Obstetrics and Gynecology, Shandong Provincial Qianfoshan Hospital, Shandong University, Jinan City, China
| | - Qun Gao
- Department of Obstetrics and Gynecology, The Affiliated Hospital of Qingdao University, Qingdao City, China
| | - Wenjing Zhang
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan City, China
| | - Teng Zhang
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan City, China
| | - Taotao Dong
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan City, China.
| | - Baoxia Cui
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan City, China.
| |
Collapse
|
27
|
Mu Y, Tizhoosh HR, Dehkharghanian T, Alfasly S, Campbell CJV. Model-Agnostic Binary Patch Grouping for Bone Marrow Whole Slide Image Representation. THE AMERICAN JOURNAL OF PATHOLOGY 2024; 194:721-734. [PMID: 38320631 DOI: 10.1016/j.ajpath.2024.01.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/28/2023] [Revised: 12/29/2023] [Accepted: 01/10/2024] [Indexed: 02/08/2024]
Abstract
Histopathology is the reference standard for pathology diagnosis, and has evolved with the digitization of glass slides [ie, whole slide images (WSIs)]. While trained histopathologists are able to diagnose diseases by examining WSIs visually, this process is time consuming and prone to variability. To address these issues, artificial intelligence models are being developed to generate slide-level representations of WSIs, summarizing the entire slide as a single vector. This enables various computational pathology applications, including interslide search, multimodal training, and slide-level classification. Achieving expressive and robust slide-level representations hinges on patch feature extraction and aggregation steps. This study proposed an additional binary patch grouping (BPG) step, a plugin that can be integrated into various slide-level representation pipelines, to enhance the quality of slide-level representation in bone marrow histopathology. BPG excludes patches with less clinical relevance through minimal interaction with the pathologist; a one-time human intervention for the entire process. This study further investigated domain-general versus domain-specific feature extraction models based on convolution and attention and examined two different feature aggregation methods, with and without BPG, showing BPG's generalizability. The results showed that using BPG boosts the performance of WSI retrieval (mean average precision at 10) by 4% and improves WSI classification (weighted-F1) by 5% compared to not using BPG. Additionally, domain-general large models and parameterized pooling produced the best-quality slide-level representations.
Collapse
Affiliation(s)
- Youqing Mu
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada; Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Hamid R Tizhoosh
- Rhazes Lab, Artificial Intelligence and Informatics, Mayo Clinic, Rochester, Minnesota
| | - Taher Dehkharghanian
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada; Department of Nephrology, University Health Network, Toronto, Ontario, Canada
| | - Saghir Alfasly
- Rhazes Lab, Artificial Intelligence and Informatics, Mayo Clinic, Rochester, Minnesota
| | - Clinton J V Campbell
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada.
| |
Collapse
|
28
|
Baroni GL, Rasotto L, Roitero K, Tulisso A, Di Loreto C, Della Mea V. Optimizing Vision Transformers for Histopathology: Pretraining and Normalization in Breast Cancer Classification. J Imaging 2024; 10:108. [PMID: 38786562 PMCID: PMC11121856 DOI: 10.3390/jimaging10050108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Revised: 04/26/2024] [Accepted: 04/27/2024] [Indexed: 05/25/2024] Open
Abstract
This paper introduces a self-attention Vision Transformer model specifically developed for classifying breast cancer in histology images. We examine various training strategies and configurations, including pretraining, dimension resizing, data augmentation and color normalization strategies, patch overlap, and patch size configurations, in order to evaluate their impact on the effectiveness of the histology image classification. Additionally, we provide evidence for the increase in effectiveness gathered through geometric and color data augmentation techniques. We primarily utilize the BACH dataset to train and validate our methods and models, but we also test them on two additional datasets, BRACS and AIDPATH, to verify their generalization capabilities. Our model, developed from a transformer pretrained on ImageNet, achieves an accuracy rate of 0.91 on the BACH dataset, 0.74 on the BRACS dataset, and 0.92 on the AIDPATH dataset. Using a model based on the prostate small and prostate medium HistoEncoder models, we achieve accuracy rates of 0.89 and 0.86, respectively. Our results suggest that pretraining on large-scale general datasets like ImageNet is advantageous. We also show the potential benefits of using domain-specific pretraining datasets, such as extensive histopathological image collections as in HistoEncoder, though not yet with clear advantages.
Collapse
Affiliation(s)
- Giulia Lucrezia Baroni
- Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy; (G.L.B.); (L.R.); (K.R.)
| | - Laura Rasotto
- Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy; (G.L.B.); (L.R.); (K.R.)
| | - Kevin Roitero
- Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy; (G.L.B.); (L.R.); (K.R.)
| | - Angelica Tulisso
- Istituto di Anatomia Patologica, Azienda Sanitaria Universitaria Friuli Centrale, 33100 Udine, Italy
| | - Carla Di Loreto
- Istituto di Anatomia Patologica, Azienda Sanitaria Universitaria Friuli Centrale, 33100 Udine, Italy
| | - Vincenzo Della Mea
- Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy; (G.L.B.); (L.R.); (K.R.)
| |
Collapse
|
29
|
Collazo C, Vargas I, Cara B, Weinheimer CJ, Grabau RP, Goldgof D, Hall L, Wickline SA, Pan H. Synergizing Deep Learning-Enabled Preprocessing and Human-AI Integration for Efficient Automatic Ground Truth Generation. Bioengineering (Basel) 2024; 11:434. [PMID: 38790302 PMCID: PMC11117745 DOI: 10.3390/bioengineering11050434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 04/20/2024] [Accepted: 04/26/2024] [Indexed: 05/26/2024] Open
Abstract
The progress of incorporating deep learning in the field of medical image interpretation has been greatly hindered due to the tremendous cost and time associated with generating ground truth for supervised machine learning, alongside concerns about the inconsistent quality of images acquired. Active learning offers a potential solution to these problems of expanding dataset ground truth by algorithmically choosing the most informative samples for ground truth labeling. Still, this effort incurs the costs of human labeling, which needs minimization. Furthermore, automatic labeling approaches employing active learning often exhibit overfitting tendencies while selecting samples closely aligned with the training set distribution and excluding out-of-distribution samples, which could potentially improve the model's effectiveness. We propose that the majority of out-of-distribution instances can be attributed to inconsistent cross images. Since the FDA approved the first whole-slide image system for medical diagnosis in 2017, whole-slide images have provided enriched critical information to advance the field of automated histopathology. Here, we exemplify the benefits of a novel deep learning strategy that utilizes high-resolution whole-slide microscopic images. We quantitatively assess and visually highlight the inconsistencies within the whole-slide image dataset employed in this study. Accordingly, we introduce a deep learning-based preprocessing algorithm designed to normalize unknown samples to the training set distribution, effectively mitigating the overfitting issue. Consequently, our approach significantly increases the amount of automatic region-of-interest ground truth labeling on high-resolution whole-slide images using active deep learning. We accept 92% of the automatic labels generated for our unlabeled data cohort, expanding the labeled dataset by 845%. Additionally, we demonstrate expert time savings of 96% relative to manual expert ground-truth labeling.
Collapse
Affiliation(s)
| | - Ian Vargas
- The Heart Institute, College of Medicine, University of South Florida, Tampa, FL 33602, USA (B.C.); (S.A.W.)
| | - Brendon Cara
- The Heart Institute, College of Medicine, University of South Florida, Tampa, FL 33602, USA (B.C.); (S.A.W.)
| | - Carla J. Weinheimer
- Department of Medicine, Washington University in St. Louis, St. Louis, MO 63110, USA
| | - Ryan P. Grabau
- The Heart Institute, College of Medicine, University of South Florida, Tampa, FL 33602, USA (B.C.); (S.A.W.)
| | - Dmitry Goldgof
- College of Engineering, University of South Florida, Tampa, FL 33620, USA
| | - Lawrence Hall
- College of Engineering, University of South Florida, Tampa, FL 33620, USA
| | - Samuel A. Wickline
- The Heart Institute, College of Medicine, University of South Florida, Tampa, FL 33602, USA (B.C.); (S.A.W.)
| | - Hua Pan
- Department of Medicine, Washington University in St. Louis, St. Louis, MO 63110, USA
- Department of Pathology & Immunology, Washington University in St. Louis, St. Louis, MO 63110, USA
- Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA
| |
Collapse
|
30
|
Huang Z, Wang L, Xu L. DRA-Net: Medical image segmentation based on adaptive feature extraction and region-level information fusion. Sci Rep 2024; 14:9714. [PMID: 38678063 DOI: 10.1038/s41598-024-60475-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2024] [Accepted: 04/23/2024] [Indexed: 04/29/2024] Open
Abstract
Medical image segmentation is a key task in computer aided diagnosis. In recent years, convolutional neural network (CNN) has made some achievements in medical image segmentation. However, the convolution operation can only extract features in a fixed size region at a time, which leads to the loss of some key features. The recently popular Transformer has global modeling capabilities, but it does not pay enough attention to local information and cannot accurately segment the edge details of the target area. Given these issues, we proposed dynamic regional attention network (DRA-Net). Different from the above methods, it first measures the similarity of features and concentrates attention on different dynamic regions. In this way, the network can adaptively select different modeling scopes for feature extraction, reducing information loss. Then, regional feature interaction is carried out to better learn local edge details. At the same time, we also design ordered shift multilayer perceptron (MLP) blocks to enhance communication within different regions, further enhancing the network's ability to learn local edge details. After several experiments, the results indicate that our network produces more accurate segmentation performance compared to other CNN and Transformer based networks.
Collapse
Affiliation(s)
- Zhongmiao Huang
- School of Computer Science and Technology, Xinjiang University, Urumqi, 830046, China
| | - Liejun Wang
- School of Computer Science and Technology, Xinjiang University, Urumqi, 830046, China.
| | - Lianghui Xu
- School of Computer Science and Technology, Xinjiang University, Urumqi, 830046, China
| |
Collapse
|
31
|
Ochi M, Komura D, Onoyama T, Shinbo K, Endo H, Odaka H, Kakiuchi M, Katoh H, Ushiku T, Ishikawa S. Registered multi-device/staining histology image dataset for domain-agnostic machine learning models. Sci Data 2024; 11:330. [PMID: 38570515 PMCID: PMC10991301 DOI: 10.1038/s41597-024-03122-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 03/04/2024] [Indexed: 04/05/2024] Open
Abstract
Variations in color and texture of histopathology images are caused by differences in staining conditions and imaging devices between hospitals. These biases decrease the robustness of machine learning models exposed to out-of-domain data. To address this issue, we introduce a comprehensive histopathology image dataset named PathoLogy Images of Scanners and Mobile phones (PLISM). The dataset consisted of 46 human tissue types stained using 13 hematoxylin and eosin conditions and captured using 13 imaging devices. Precisely aligned image patches from different domains allowed for an accurate evaluation of color and texture properties in each domain. Variation in PLISM was assessed and found to be significantly diverse across various domains, particularly between whole-slide images and smartphones. Furthermore, we assessed the improvement in domain shift using a convolutional neural network pre-trained on PLISM. PLISM is a valuable resource that facilitates the precise evaluation of domain shifts in digital pathology and makes significant contributions towards the development of robust machine learning models that can effectively address challenges of domain shift in histological image analysis.
Collapse
Affiliation(s)
- Mieko Ochi
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Daisuke Komura
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.
| | - Takumi Onoyama
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
- Division of Gastroenterology and Nephrology, Department of Multidisciplinary Internal Medicine, School of Medicine, Faculty of Medicine, Tottori University, 36-1 Nishicho, Yonago, Tottori, 683-8504, Japan
| | - Koki Shinbo
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Haruya Endo
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Hiroto Odaka
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Miwako Kakiuchi
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Hiroto Katoh
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Tetsuo Ushiku
- Department of Pathology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Shumpei Ishikawa
- Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.
- Division of Pathology, National Cancer Center Exploratory Oncology Research & Clinical Trial Center, 6-5-1 Kashiwanoha, Kashiwa, Chiba, 277-8577, Japan.
| |
Collapse
|
32
|
Shah SSH, Elmorsy E, Othman RQA, Syed A, Armaghan SU, Khalid Bokhari SU, Elmorsy ME, Bawadekji A. The Evaluation of Artificial Intelligence Technology for the Differentiation of Fresh Human Blood Cells From Other Species' Blood in the Investigation of Crime Scenes. Cureus 2024; 16:e58496. [PMID: 38765447 PMCID: PMC11101600 DOI: 10.7759/cureus.58496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/17/2024] [Indexed: 05/22/2024] Open
Abstract
OBJECTIVES The current study used the deep machine learning approach to differentiate human blood specimens from cow, goat, and chicken blood stains based on cell morphology. METHODS A total of 1,955 known Giemsa-stained digitized images were acquired from the blood of humans, cows, goats, and chickens. To train the deep learning models, the well-known VGG16, Resnet18, and Resnet34 algorithms were used. Based on the image analysis, confusion matrices were generated. RESULTS Findings showed that the F1 score for the chicken, cow, goat, and human classes were all equal to 1.0 for each of the three algorithms. The Matthews correlation coefficient (MCC) was 1 for chickens, cows, and humans in all three algorithms, while the MCC score was 0.989 for goats by ResNet18, and it was 0.994 for both ResNet34 and VGG16 algorithms. The three algorithms showed 100% sensitivity, specificity, and positive and negative predictive values for the human, cow, and chicken cells. For the goat cells, the data showed 100% sensitivity and negative predictive values with specificity and positive predictive values ranging from 98.5% to 99.6%. CONCLUSION These data showed the importance of deep learning as a potential tool for the differentiation of the species of origin of fresh crime scene blood stains.
Collapse
Affiliation(s)
| | - Ekramy Elmorsy
- Department of Pathology, Northern Border University, Arar, SAU
| | | | - Asmara Syed
- Department of Pathology, Northern Border University, Arar, SAU
| | - Syed Umar Armaghan
- Department of Research & Development - Robotic Section, Idrak AI Pvt. Ltd., Islamabad, PAK
| | | | - Mahmoud E Elmorsy
- Department of Computer Engineering, King Fahd University of Petroleum and Minerals, Dhahran, SAU
| | | |
Collapse
|
33
|
Zamanitajeddin N, Jahanifar M, Bilal M, Eastwood M, Rajpoot N. Social network analysis of cell networks improves deep learning for prediction of molecular pathways and key mutations in colorectal cancer. Med Image Anal 2024; 93:103071. [PMID: 38199068 DOI: 10.1016/j.media.2023.103071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 11/14/2023] [Accepted: 12/19/2023] [Indexed: 01/12/2024]
Abstract
Colorectal cancer (CRC) is a primary global health concern, and identifying the molecular pathways, genetic subtypes, and mutations associated with CRC is crucial for precision medicine. However, traditional measurement techniques such as gene sequencing are costly and time-consuming, while most deep learning methods proposed for this task lack interpretability. This study offers a new approach to enhance the state-of-the-art deep learning methods for molecular pathways and key mutation prediction by incorporating cell network information. We build cell graphs with nuclei as nodes and nuclei connections as edges of the network and leverage Social Network Analysis (SNA) measures to extract abstract, perceivable, and interpretable features that explicitly describe the cell network characteristics in an image. Our approach does not rely on precise nuclei segmentation or feature extraction, is computationally efficient, and is easily scalable. In this study, we utilize the TCGA-CRC-DX dataset, comprising 499 patients and 502 diagnostic slides from primary colorectal tumours, sourced from 36 distinct medical centres in the United States. By incorporating the SNA features alongside deep features in two multiple instance learning frameworks, we demonstrate improved performance for chromosomal instability (CIN), hypermutated tumour (HM), TP53 gene, BRAF gene, and Microsatellite instability (MSI) status prediction tasks (2.4%-4% and 7-8.8% improvement in AUROC and AUPRC on average). Additionally, our method achieves outstanding performance on MSI prediction in an external PAIP dataset (99% AUROC and 98% AUPRC), demonstrating its generalizability. Our findings highlight the discrimination power of SNA features and how they can be beneficial to deep learning models' performance and provide insights into the correlation of cell network profiles with molecular pathways and key mutations.
Collapse
Affiliation(s)
- Neda Zamanitajeddin
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK.
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mohsin Bilal
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mark Eastwood
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Histofy Ltd., Birmingham, UK.
| |
Collapse
|
34
|
Ghaderinia M, Abadijoo H, Mahdavian A, Kousha E, Shakibi R, Taheri SMR, Simaee H, Khatibi A, Moosavi-Movahedi AA, Khayamian MA. Smartphone-based device for point-of-care diagnostics of pulmonary inflammation using convolutional neural networks (CNNs). Sci Rep 2024; 14:6912. [PMID: 38519489 PMCID: PMC10959990 DOI: 10.1038/s41598-024-54939-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Accepted: 02/19/2024] [Indexed: 03/25/2024] Open
Abstract
In pulmonary inflammation diseases, like COVID-19, lung involvement and inflammation determine the treatment regime. Respiratory inflammation is typically arisen due to the cytokine storm and the leakage of the vessels for immune cells recruitment. Currently, such a situation is detected by the clinical judgment of a specialist or precisely by a chest CT scan. However, the lack of accessibility to the CT machines in many poor medical centers as well as its expensive service, demands more accessible methods for fast and cheap detection of lung inflammation. Here, we have introduced a novel method for tracing the inflammation and lung involvement in patients with pulmonary inflammation, such as COVID-19, by a simple electrolyte detection in their sputum samples. The presence of the electrolyte in the sputum sample results in the fern-like structures after air-drying. These fern patterns are different in the CT positive and negative cases that are detected by an AI application on a smartphone and using a low-cost and portable mini-microscope. Evaluating 160 patient-derived sputum sample images, this method demonstrated an interesting accuracy of 95%, as confirmed by CT-scan results. This finding suggests that the method has the potential to serve as a promising and reliable approach for recognizing lung inflammatory diseases, such as COVID-19.
Collapse
Affiliation(s)
- Mohammadreza Ghaderinia
- Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran
- Integrated Biophysics and Bioengineering Lab (iBL), Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran
- Nano Electronic Center of Excellence, Nano Bio Electronics Devices Lab, School of Electrical and Computer Engineering, University of Tehran, P.O. Box 14395/515, Tehran, Iran
| | - Hamed Abadijoo
- Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran
- Integrated Biophysics and Bioengineering Lab (iBL), Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran
- Nano Electronic Center of Excellence, Nano Bio Electronics Devices Lab, School of Electrical and Computer Engineering, University of Tehran, P.O. Box 14395/515, Tehran, Iran
| | - Ashkan Mahdavian
- Nano Electronic Center of Excellence, Nano Bio Electronics Devices Lab, School of Electrical and Computer Engineering, University of Tehran, P.O. Box 14395/515, Tehran, Iran
| | - Ebrahim Kousha
- Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran
- Integrated Biophysics and Bioengineering Lab (iBL), Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran
- Nano Electronic Center of Excellence, Nano Bio Electronics Devices Lab, School of Electrical and Computer Engineering, University of Tehran, P.O. Box 14395/515, Tehran, Iran
| | - Reyhaneh Shakibi
- Department of Biophysics, Faculty of Biological Sciences, Tarbiat Modares University, Tehran, Iran
| | - S Mohammad-Reza Taheri
- Groningen university, University medical center Groningen, Antonius Deusinglaan 1, 9713AW, Groningen, The Netherlands
- Condensed Matter National Laboratory, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
| | - Hossein Simaee
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Ali Khatibi
- Department of Biotechnology, Faculty of Biological Sciences, Alzahra University, Tehran, Iran
| | | | - Mohammad Ali Khayamian
- Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran.
- Integrated Biophysics and Bioengineering Lab (iBL), Institute of Biochemistry and Biophysics, University of Tehran, Tehran, 1417614335, Iran.
| |
Collapse
|
35
|
Koziarski M, Cyganek B, Niedziela P, Olborski B, Antosz Z, Żydak M, Kwolek B, Wąsowicz P, Bukała A, Swadźba J, Sitkowski P. DiagSet: a dataset for prostate cancer histopathological image classification. Sci Rep 2024; 14:6780. [PMID: 38514661 PMCID: PMC10958036 DOI: 10.1038/s41598-024-52183-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 01/15/2024] [Indexed: 03/23/2024] Open
Abstract
Cancer diseases constitute one of the most significant societal challenges. In this paper, we introduce a novel histopathological dataset for prostate cancer detection. The proposed dataset, consisting of over 2.6 million tissue patches extracted from 430 fully annotated scans, 4675 scans with assigned binary diagnoses, and 46 scans with diagnoses independently provided by a group of histopathologists can be found at https://github.com/michalkoziarski/DiagSet . Furthermore, we propose a machine learning framework for detection of cancerous tissue regions and prediction of scan-level diagnosis, utilizing thresholding to abstain from the decision in uncertain cases. The proposed approach, composed of ensembles of deep neural networks operating on the histopathological scans at different scales, achieves 94.6% accuracy in patch-level recognition and is compared in a scan-level diagnosis with 9 human histopathologists showing high statistical agreement.
Collapse
Affiliation(s)
- Michał Koziarski
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland.
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland.
- Mila - Quebec AI Institute, 6666 Rue Saint-Urbain, Montréal, QC H2S 3H1, Canada.
| | - Bogusław Cyganek
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Przemysław Niedziela
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Bogusław Olborski
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Zbigniew Antosz
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Marcin Żydak
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Bogdan Kwolek
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Paweł Wąsowicz
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| | - Andrzej Bukała
- AGH University of Science and Technology, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Jakub Swadźba
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
- Andrzej Frycz Modrzewski Krakow University, Gustawa Herlinga-Grudzińskiego 1, 30-705, Kraków, Poland
| | - Piotr Sitkowski
- Diagnostyka Consilio Sp. z o.o., Ul. Kosynierów Gdyńskich 61a, 93-357, Łódż, Poland
| |
Collapse
|
36
|
Mahbod A, Polak C, Feldmann K, Khan R, Gelles K, Dorffner G, Woitek R, Hatamikia S, Ellinger I. NuInsSeg: A fully annotated dataset for nuclei instance segmentation in H&E-stained histological images. Sci Data 2024; 11:295. [PMID: 38486039 PMCID: PMC10940572 DOI: 10.1038/s41597-024-03117-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 03/04/2024] [Indexed: 03/18/2024] Open
Abstract
In computational pathology, automatic nuclei instance segmentation plays an essential role in whole slide image analysis. While many computerized approaches have been proposed for this task, supervised deep learning (DL) methods have shown superior segmentation performances compared to classical machine learning and image processing techniques. However, these models need fully annotated datasets for training which is challenging to acquire, especially in the medical domain. In this work, we release one of the biggest fully manually annotated datasets of nuclei in Hematoxylin and Eosin (H&E)-stained histological images, called NuInsSeg. This dataset contains 665 image patches with more than 30,000 manually segmented nuclei from 31 human and mouse organs. Moreover, for the first time, we provide additional ambiguous area masks for the entire dataset. These vague areas represent the parts of the images where precise and deterministic manual annotations are impossible, even for human experts. The dataset and detailed step-by-step instructions to generate related segmentation masks are publicly available on the respective repositories.
Collapse
Affiliation(s)
- Amirreza Mahbod
- Research Center for Medical Image Analysis and Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, 3500, Austria.
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, 1090, Austria.
| | - Christine Polak
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, 1090, Austria
| | - Katharina Feldmann
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, 1090, Austria
| | - Rumsha Khan
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, 1090, Austria
| | - Katharina Gelles
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, 1090, Austria
| | - Georg Dorffner
- Institute of Artificial Intelligence, Medical University of Vienna, Vienna, 1090, Austria
| | - Ramona Woitek
- Research Center for Medical Image Analysis and Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, 3500, Austria
| | - Sepideh Hatamikia
- Research Center for Medical Image Analysis and Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, 3500, Austria
- Austrian Center for Medical Innovation and Technology, Wiener Neustadt, 2700, Austria
| | - Isabella Ellinger
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, 1090, Austria
| |
Collapse
|
37
|
Nagaishi M, Miyoshi H, Kugler M, Sato K, Kohno K, Takeuchi M, Yamada K, Furuta T, Hashimoto N, Takeuchi I, Hontani H, Ohshima K. The Detection of Neoplastic Cells Using Objective Cytomorphologic Parameters in Malignant Lymphoma. J Transl Med 2024; 104:100302. [PMID: 38092181 DOI: 10.1016/j.labinv.2023.100302] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 12/01/2023] [Accepted: 12/07/2023] [Indexed: 01/13/2024] Open
Abstract
Pathologic evaluation is the most crucial method for diagnosing malignant lymphomas. However, there are no established diagnostic criteria for evaluating pathologic morphology. We manually circled cell nuclei in the lesions of 10 patients with diffuse large B-cell lymphoma (DLBCL), follicular lymphoma, and reactive lymphadenitis. Seventeen parameters related to nuclear shape, color, and other characteristics were measured. We attempted to compare the statistical differences between these subtypes and extract distinctive disease-specific populations on the basis of these parameters. Statistically significant differences were observed between the different types of lymphoma for many of the 17 parameters. Through t-distributed stochastic neighbor embedding analysis, we extracted a cluster of cells that showed distinctive features of DLBCL and were not found in follicular lymphoma or reactive lymphadenitis. We created a decision tree to identify the characteristics of the cells within that cluster. Based on a 5-fold cross-validation study, the average sensitivity, specificity, and accuracy obtained were 84.1%, 98.4%, and 97.3%, respectively. A similar result was achieved using a validation experiment. Important parameters that indicate the features of DLBCL include Area, ConcaveCount, MaxGray, and ModeGray. By quantifying pathologic morphology, it was possible to objectively represent the cell morphology specific to each lymphoma subtype using quantitative indicators. The quantified morphologic information has the potential to serve as a reproducible and flexible diagnostic tool.
Collapse
Affiliation(s)
- Miharu Nagaishi
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| | - Hiroaki Miyoshi
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan.
| | - Mauricio Kugler
- Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan
| | - Kensaku Sato
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| | - Kei Kohno
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| | - Mai Takeuchi
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| | - Kyohei Yamada
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| | - Takuya Furuta
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| | | | - Ichiro Takeuchi
- Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan; Department of Mechanical Systems Engineering, Nagoya University, Nagoya, Japan
| | - Hidekata Hontani
- Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan
| | - Koichi Ohshima
- Department of Pathology, Kurume University, School of Medicine, Kurume, Japan
| |
Collapse
|
38
|
Rajadurai S, Perumal K, Ijaz MF, Chowdhary CL. PrecisionLymphoNet: Advancing Malignant Lymphoma Diagnosis via Ensemble Transfer Learning with CNNs. Diagnostics (Basel) 2024; 14:469. [PMID: 38472941 DOI: 10.3390/diagnostics14050469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2023] [Revised: 01/26/2024] [Accepted: 01/27/2024] [Indexed: 03/14/2024] Open
Abstract
Malignant lymphoma, which impacts the lymphatic system, presents diverse challenges in accurate diagnosis due to its varied subtypes-chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL). Lymphoma is a form of cancer that begins in the lymphatic system, impacting lymphocytes, which are a specific type of white blood cell. This research addresses these challenges by proposing ensemble and non-ensemble transfer learning models employing pre-trained weights from VGG16, VGG19, DenseNet201, InceptionV3, and Xception. For the ensemble technique, this paper adopts a stack-based ensemble approach. It is a two-level classification approach and best suited for accuracy improvement. Testing on a multiclass dataset of CLL, FL, and MCL reveals exceptional diagnostic accuracy, with DenseNet201, InceptionV3, and Xception exceeding 90% accuracy. The proposed ensemble model, leveraging InceptionV3 and Xception, achieves an outstanding 99% accuracy over 300 epochs, surpassing previous prediction methods. This study demonstrates the feasibility and efficiency of the proposed approach, showcasing its potential in real-world medical applications for precise lymphoma diagnosis.
Collapse
Affiliation(s)
- Sivashankari Rajadurai
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India
| | - Kumaresan Perumal
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India
| | - Muhammad Fazal Ijaz
- School of IT and Engineering, Melbourne Institute of Technology, Melbourne, VIC 3000, Australia
| | - Chiranji Lal Chowdhary
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India
| |
Collapse
|
39
|
Gifani P, Shalbaf A. Transfer Learning with Pretrained Convolutional Neural Network for Automated Gleason Grading of Prostate Cancer Tissue Microarrays. JOURNAL OF MEDICAL SIGNALS & SENSORS 2024; 14:4. [PMID: 38510670 PMCID: PMC10950311 DOI: 10.4103/jmss.jmss_42_22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Revised: 12/20/2022] [Accepted: 03/22/2023] [Indexed: 03/22/2024]
Abstract
Background The Gleason grading system has been the most effective prediction for prostate cancer patients. This grading system provides this possibility to assess prostate cancer's aggressiveness and then constitutes an important factor for stratification and therapeutic decisions. However, determining Gleason grade requires highly-trained pathologists and is time-consuming and tedious, and suffers from inter-pathologist variability. To remedy these limitations, this paper introduces an automatic methodology based on transfer learning with pretrained convolutional neural networks (CNNs) for automatic Gleason grading of prostate cancer tissue microarray (TMA). Methods Fifteen pretrained (CNNs): Efficient Nets (B0-B5), NasNetLarge, NasNetMobile, InceptionV3, ResNet-50, SeResnet 50, Xception, DenseNet121, ResNext50, and inception_resnet_v2 were fine-tuned on a dataset of prostate carcinoma TMA images. Six pathologists separately identified benign and cancerous areas for each prostate TMA image by allocating benign, 3, 4, or 5 Gleason grade for 244 patients. The dataset was labeled by these pathologists and majority vote was applied on pixel-wise annotations to obtain a unified label. Results Results showed the NasnetLarge architecture is the best model among them in the classification of prostate TMA images of 244 patients with accuracy of 0.93 and area under the curve of 0.98. Conclusion Our study can act as a highly trained pathologist to categorize the prostate cancer stages with more objective and reproducible results.
Collapse
Affiliation(s)
- Parisa Gifani
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Ahmad Shalbaf
- Cancer Research Center, Shahid Beheshti University of Medical Sciences, Tehran, Iran
- Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| |
Collapse
|
40
|
Mahbub T, Obeid A, Javed S, Dias J, Hassan T, Werghi N. Center-Focused Affinity Loss for Class Imbalance Histology Image Classification. IEEE J Biomed Health Inform 2024; 28:952-963. [PMID: 37999960 DOI: 10.1109/jbhi.2023.3336372] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2023]
Abstract
Early-stage cancer diagnosis potentially improves the chances of survival for many cancer patients worldwide. Manual examination of Whole Slide Images (WSIs) is a time-consuming task for analyzing tumor-microenvironment. To overcome this limitation, the conjunction of deep learning with computational pathology has been proposed to assist pathologists in efficiently prognosing the cancerous spread. Nevertheless, the existing deep learning methods are ill-equipped to handle fine-grained histopathology datasets. This is because these models are constrained via conventional softmax loss function, which cannot expose them to learn distinct representational embeddings of the similarly textured WSIs containing an imbalanced data distribution. To address this problem, we propose a novel center-focused affinity loss (CFAL) function that exhibits 1) constructing uniformly distributed class prototypes in the feature space, 2) penalizing difficult samples, 3) minimizing intra-class variations, and 4) placing greater emphasis on learning minority class features. We evaluated the performance of the proposed CFAL loss function on two publicly available breast and colon cancer datasets having varying levels of imbalanced classes. The proposed CFAL function shows better discrimination abilities as compared to the popular loss functions such as ArcFace, CosFace, and Focal loss. Moreover, it outperforms several SOTA methods for histology image classification across both datasets.
Collapse
|
41
|
Ratziu V, Hompesch M, Petitjean M, Serdjebi C, Iyer JS, Parwani AV, Tai D, Bugianesi E, Cusi K, Friedman SL, Lawitz E, Romero-Gómez M, Schuppan D, Loomba R, Paradis V, Behling C, Sanyal AJ. Artificial intelligence-assisted digital pathology for non-alcoholic steatohepatitis: current status and future directions. J Hepatol 2024; 80:335-351. [PMID: 37879461 DOI: 10.1016/j.jhep.2023.10.015] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 08/28/2023] [Accepted: 10/09/2023] [Indexed: 10/27/2023]
Abstract
The worldwide prevalence of non-alcoholic steatohepatitis (NASH) is increasing, causing a significant medical burden, but no approved therapeutics are currently available. NASH drug development requires histological analysis of liver biopsies by expert pathologists for trial enrolment and efficacy assessment, which can be hindered by multiple issues including sample heterogeneity, inter-reader and intra-reader variability, and ordinal scoring systems. Consequently, there is a high unmet need for accurate, reproducible, quantitative, and automated methods to assist pathologists with histological analysis to improve the precision around treatment and efficacy assessment. Digital pathology (DP) workflows in combination with artificial intelligence (AI) have been established in other areas of medicine and are being actively investigated in NASH to assist pathologists in the evaluation and scoring of NASH histology. DP/AI models can be used to automatically detect, localise, quantify, and score histological parameters and have the potential to reduce the impact of scoring variability in NASH clinical trials. This narrative review provides an overview of DP/AI tools in development for NASH, highlights key regulatory considerations, and discusses how these advances may impact the future of NASH clinical management and drug development. This should be a high priority in the NASH field, particularly to improve the development of safe and effective therapeutics.
Collapse
Affiliation(s)
- Vlad Ratziu
- Sorbonne Université, ICAN Institute for Cardiometabolism and Nutrition, Hospital Pitié-Salpêtrière, INSERM UMRS 1138 CRC, Paris, France.
| | | | | | | | | | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | | | | | - Kenneth Cusi
- Division of Endocrinology, Diabetes and Metabolism, University of Florida, Gainesville, FL, USA
| | - Scott L Friedman
- Division of Liver Diseases, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Eric Lawitz
- Texas Liver Institute, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Manuel Romero-Gómez
- Hospital Universitario Virgen del Rocío, CiberEHD, Insituto de Biomedicina de Sevilla (HUVR/CSIC/US), Universidad de Sevilla, Seville, Spain
| | - Detlef Schuppan
- Institute of Translational Immunology and Department of Medicine, University Medical Center, Mainz, Germany; Department of Hepatology and Gastroenterology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Rohit Loomba
- NAFLD Research Center, University of California at San Diego, San Diego, CA, USA
| | - Valérie Paradis
- Université Paris Cité, Service d'Anatomie Pathologique, Hôpital Beaujon, Paris, France
| | | | - Arun J Sanyal
- Division of Gastroenterology, Hepatology and Nutrition, Virginia Commonwealth University, Richmond, VA, USA
| |
Collapse
|
42
|
Guo Z, Ao S, Ao B. Few-shot learning based oral cancer diagnosis using a dual feature extractor prototypical network. J Biomed Inform 2024; 150:104584. [PMID: 38199300 DOI: 10.1016/j.jbi.2024.104584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 12/04/2023] [Accepted: 01/02/2024] [Indexed: 01/12/2024]
Abstract
A large global health issue is cancer, wherein early diagnosis and treatment have proven to be life-saving. This holds true for oral cancer, thus emphasizing the significance of timely intervention. Deep learning techniques have gained traction in early cancer detection, exhibiting promising outcomes in accurate diagnosis. However, collecting a substantial amount of training data poses a challenge for deep learning models in cancer diagnosis. To address this limitation, this study proposes an oral cancer diagnosis approach based on a few-shot learning framework that circumvents the need for extensive training data. Specifically, a prototypical network is employed to construct a diagnostic model, wherein two feature extractors are utilized to extract prototypical features and query features respectively, departing from the conventional use of a single feature extraction function in prototypical networks. Moreover, a customized loss function is designed for the proposed method. Rigorous experimentation using a histopathological image dataset demonstrates the superior performance of our proposed approach over comparison methods.
Collapse
Affiliation(s)
- Zijun Guo
- Department of Stomatology, Daping Hospital, Army Medical Center of PLA, Chongqing 400042, China
| | - Sha Ao
- The People's Hospital of Rongchang District in Chongqing, Chongqing 402460, China
| | - Bo Ao
- Traditional Chinese Medicine Hospital of Jiulongpo District in Chongqing, Chongqing 400080, China.
| |
Collapse
|
43
|
Wu W, Gao C, DiPalma J, Vosoughi S, Hassanpour S. Improving Representation Learning for Histopathologic Images with Cluster Constraints. PROCEEDINGS. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION 2024; 2023:21347-21357. [PMID: 38694561 PMCID: PMC11062482 DOI: 10.1109/iccv51070.2023.01957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2024]
Abstract
Recent advances in whole-slide image (WSI) scanners and computational capabilities have significantly propelled the application of artificial intelligence in histopathology slide analysis. While these strides are promising, current supervised learning approaches for WSI analysis come with the challenge of exhaustively labeling high-resolution slides-a process that is both labor-intensive and timeconsuming. In contrast, self-supervised learning (SSL) pretraining strategies are emerging as a viable alternative, given that they don't rely on explicit data annotations. These SSL strategies are quickly bridging the performance disparity with their supervised counterparts. In this context, we introduce an SSL framework. This framework aims for transferable representation learning and semantically meaningful clustering by synergizing invariance loss and clustering loss in WSI analysis. Notably, our approach outperforms common SSL methods in downstream classification and clustering tasks, as evidenced by tests on the Camelyon16 and a pancreatic cancer dataset. The code and additional details are accessible at https://github.com/wwyi1828/CluSiam.
Collapse
|
44
|
Hubbard-Perez M, Luchian A, Milford C, Ressel L. Use of deep learning for the classification of hyperplastic lymph node and common subtypes of canine lymphomas: a preliminary study. Front Vet Sci 2024; 10:1309877. [PMID: 38283371 PMCID: PMC10811236 DOI: 10.3389/fvets.2023.1309877] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Accepted: 12/04/2023] [Indexed: 01/30/2024] Open
Abstract
Artificial Intelligence has observed significant growth in its ability to classify different types of tumors in humans due to advancements in digital pathology technology. Among these tumors, lymphomas are quite common in dogs, despite studies on the application of AI in domestic species are scarce. This research aims to employ deep learning (DL) through convolutional neural networks (CNNs) to distinguish between normal lymph nodes and 3 WHO common subtypes of canine lymphomas. To train and validate the CNN, 1,530 high-resolution microscopic images derived from whole slide scans (WSIs) were used, including those of background areas, hyperplastic lymph nodes (n = 4), and three different lymphoma subtypes: diffuse large B cell lymphoma (DLBCL; n = 5), lymphoblastic (LBL; n = 5), and marginal zone lymphoma (MZL; n = 3). The CNN was able to correctly identify 456 images of the possible 457 test sets, achieving a maximum accuracy of 99.34%. The results of this study have demonstrated the feasibility of using deep learning to differentiate between hyperplastic lymph nodes and lymphomas, as well as to classify common WHO subtypes. Further research is required to explore the implications of these findings and validate the ability of the network to classify a broader range of lymphomas.
Collapse
Affiliation(s)
| | | | | | - Lorenzo Ressel
- DiMoLab, Institute of Infection Veterinary and Ecological Sciences, Department of Veterinary Anatomy Physiology and Pathology, University of Liverpool, Liverpool, United Kingdom
| |
Collapse
|
45
|
Kim S, Rakib Hasan K, Ando Y, Ko S, Lee D, Park NJY, Cho J. Improving Tumor-Infiltrating Lymphocytes Score Prediction in Breast Cancer with Self-Supervised Learning. Life (Basel) 2024; 14:90. [PMID: 38255705 PMCID: PMC11154396 DOI: 10.3390/life14010090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Revised: 01/02/2024] [Accepted: 01/03/2024] [Indexed: 01/24/2024] Open
Abstract
Tumor microenvironment (TME) plays a pivotal role in immuno-oncology, which investigates the intricate interactions between tumors and the human immune system. Specifically, tumor-infiltrating lymphocytes (TILs) are crucial biomarkers for evaluating the prognosis of breast cancer patients and have the potential to refine immunotherapy precision and accurately identify tumor cells in specific cancer types. In this study, we conducted tissue segmentation and lymphocyte detection tasks to predict TIL scores by employing self-supervised learning (SSL) model-based approaches capable of addressing limited labeling data issues. Our experiments showed a 1.9% improvement in tissue segmentation and a 2% improvement in lymphocyte detection over the ImageNet pre-training model. Using these SSL-based models, we achieved a TIL score of 0.718 with a 4.4% improvement. In particular, when trained with only 10% of the entire dataset, the SwAV pre-trained model exhibited a superior performance over other models. Our work highlights improved tissue segmentation and lymphocyte detection using the SSL model with less labeled data for TIL score prediction.
Collapse
Affiliation(s)
- Sijin Kim
- Department of Biomedical Science, Kyungpook National University, Daegu 41566, Republic of Korea; (S.K.); (K.R.H.); (Y.A.); (S.K.); (D.L.)
| | - Kazi Rakib Hasan
- Department of Biomedical Science, Kyungpook National University, Daegu 41566, Republic of Korea; (S.K.); (K.R.H.); (Y.A.); (S.K.); (D.L.)
| | - Yu Ando
- Department of Biomedical Science, Kyungpook National University, Daegu 41566, Republic of Korea; (S.K.); (K.R.H.); (Y.A.); (S.K.); (D.L.)
| | - Seokhwan Ko
- Department of Biomedical Science, Kyungpook National University, Daegu 41566, Republic of Korea; (S.K.); (K.R.H.); (Y.A.); (S.K.); (D.L.)
| | - Donghyeon Lee
- Department of Biomedical Science, Kyungpook National University, Daegu 41566, Republic of Korea; (S.K.); (K.R.H.); (Y.A.); (S.K.); (D.L.)
| | - Nora Jee-Young Park
- Department of Pathology, School of Medicine, Kyungpook National University, Daegu 41944, Republic of Korea;
- Department of Pathology, Kyungpook National University Chilgok Hospital, Daegu 41404, Republic of Korea
| | - Junghwan Cho
- Clinical Omics Institute, Kyungpook National University, Daegu 41405, Republic of Korea
| |
Collapse
|
46
|
Beam K, Sharma P, Levy P, Beam AL. Artificial intelligence in the neonatal intensive care unit: the time is now. J Perinatol 2024; 44:131-135. [PMID: 37443271 DOI: 10.1038/s41372-023-01719-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 06/24/2023] [Accepted: 07/03/2023] [Indexed: 07/15/2023]
Abstract
Artificial intelligence (AI) has the potential to revolutionize the neonatal intensive care unit (NICU) care by leveraging the large-scale, high-dimensional data that are generated by NICU patients. There is an emerging recognition that the confluence of technological progress, commercialization pathways, and rich data sets provides a unique opportunity for AI to make a lasting impact on the NICU. In this perspective article, we discuss four broad categories of AI applications in the NICU: imaging interpretation, prediction modeling of electronic health record data, integration of real-time monitoring data, and documentation and billing. By enhancing decision-making, streamlining processes, and improving patient outcomes, AI holds the potential to transform the quality of care for vulnerable newborns, making the excitement surrounding AI advancements well-founded and the potential for significant positive change stronger than ever before.
Collapse
Affiliation(s)
- Kristyn Beam
- Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Puneet Sharma
- Division of Newborn Medicine, Department of Pediatrics Boston Children's Hospital, Boston, MA, USA
| | - Phil Levy
- Division of Newborn Medicine, Department of Pediatrics Boston Children's Hospital, Boston, MA, USA
| | - Andrew L Beam
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
47
|
Farooq H, Saleem S, Aleem I, Iftikhar A, Sheikh UN, Naveed H. Toward interpretable and generalized mitosis detection in digital pathology using deep learning. Digit Health 2024; 10:20552076241255471. [PMID: 38778869 PMCID: PMC11110526 DOI: 10.1177/20552076241255471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 05/01/2024] [Indexed: 05/25/2024] Open
Abstract
Objective The mitotic activity index is an important prognostic factor in the diagnosis of cancer. The task of mitosis detection is difficult as the nuclei are microscopic in size and partially labeled, and there are many more non-mitotic nuclei compared to mitotic ones. In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method to tackle these challenges. Methods Our proposed methodology is inspired from recent research on deep learning and an extensive analysis on the dataset and training pipeline. We first used the MiDoG'22 dataset for training, validation, and testing. We then tested the methodology without fine-tuning on the TUPAC'16 dataset and on a real-time case from Shaukat Khanum Memorial Cancer Hospital and Research Centre. Results Our methodology has shown promising results both quantitatively and qualitatively. Quantitatively, our methodology achieved an F1-score of 0.87 on the MiDoG'22 dataset and an F1-score of 0.83 on the TUPAC dataset. Qualitatively, our methodology is generalizable and interpretable across various datasets and clinical settings. Conclusion In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method that can accurately predict mitotic nuclei. We illustrate the accuracy, generalizability, and interpretability of our approach across various datasets and clinical settings. Our methodology can speed up the adoption of computer-aided digital pathology in clinical settings.
Collapse
Affiliation(s)
- Hasan Farooq
- Computational Biology Research Lab, National University of Computer & Emerging Sciences, Islamabad, Pakistan
| | - Saira Saleem
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Iffat Aleem
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Ayesha Iftikhar
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Umer Nisar Sheikh
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore, Pakistan
| | - Hammad Naveed
- Computational Biology Research Lab, National University of Computer & Emerging Sciences, Islamabad, Pakistan
| |
Collapse
|
48
|
Kveton M, Hudec L, Vykopal I, Halinkovic M, Laco M, Felsoova A, Benesova W, Fabian O. Digital pathology in cardiac transplant diagnostics: from biopsies to algorithms. Cardiovasc Pathol 2024; 68:107587. [PMID: 37926351 DOI: 10.1016/j.carpath.2023.107587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 10/03/2023] [Accepted: 10/30/2023] [Indexed: 11/07/2023] Open
Abstract
In the field of heart transplantation, the ability to accurately and promptly diagnose cardiac allograft rejection is crucial. This comprehensive review explores the transformative role of digital pathology and computational pathology, especially through machine learning, in this critical domain. These methodologies harness large datasets to extract subtle patterns and valuable information that extend beyond human perceptual capabilities, potentially enhancing diagnostic outcomes. Current research indicates that these computer-based systems could offer accuracy and performance matching, or even exceeding, that of expert pathologists, thereby introducing more objectivity and reducing observer variability. Despite promising results, several challenges such as limited sample sizes, diverse data sources, and the absence of standardized protocols pose significant barriers to the widespread adoption of these techniques. The future of digital pathology in heart transplantation diagnostics depends on utilizing larger, more diverse patient cohorts, standardizing data collection, processing, and evaluation protocols, and fostering collaborative research efforts. The integration of various data types, including clinical, demographic, and imaging information, could further refine diagnostic precision. As researchers address these challenges and promote collaborative efforts, digital pathology has the potential to become an integral part of clinical practice, ultimately improving patient care in heart transplantation.
Collapse
Affiliation(s)
- Martin Kveton
- Third Faculty of Medicine, Charles University, Prague, Czech Republic; Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic.
| | - Lukas Hudec
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Ivan Vykopal
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Matej Halinkovic
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Miroslav Laco
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Andrea Felsoova
- Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic; Department of Histology and Embryology, Second Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Wanda Benesova
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Ondrej Fabian
- Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic; Department of Pathology and Molecular Medicine, Third Faculty of Medicine, Charles University and Thomayer Hospital, Prague, Czech Republic
| |
Collapse
|
49
|
Ortiz S, Rojas-Valenzuela I, Rojas F, Valenzuela O, Herrera LJ, Rojas I. Novel methodology for detecting and localizing cancer area in histopathological images based on overlapping patches. Comput Biol Med 2024; 168:107713. [PMID: 38000243 DOI: 10.1016/j.compbiomed.2023.107713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Revised: 11/07/2023] [Accepted: 11/15/2023] [Indexed: 11/26/2023]
Abstract
Cancer disease is one of the most important pathologies in the world, as it causes the death of millions of people, and the cure of this disease is limited in most cases. Rapid spread is one of the most important features of this disease, so many efforts are focused on its early-stage detection and localization. Medicine has made numerous advances in the recent decades with the help of artificial intelligence (AI), reducing costs and saving time. In this paper, deep learning models (DL) are used to present a novel method for detecting and localizing cancerous zones in WSI images, using tissue patch overlay to improve performance results. A novel overlapping methodology is proposed and discussed, together with different alternatives to evaluate the labels of the patches overlapping in the same zone to improve detection performance. The goal is to strengthen the labeling of different areas of an image with multiple overlapping patch testing. The results show that the proposed method improves the traditional framework and provides a different approach to cancer detection. The proposed method, based on applying 3x3 step 2 average pooling filters on overlapping patch labels, provides a better result with a 12.9% correction percentage for misclassified patches on the HUP dataset and 15.8% on the CINIJ dataset. In addition, a filter is implemented to correct isolated patches that were also misclassified. Finally, a CNN decision threshold study is performed to analyze the impact of the threshold value on the accuracy of the model. The alteration of the threshold decision along with the filter for isolated patches and the proposed method for overlapping patches, corrects about 20% of the patches that are mislabeled in the traditional method. As a whole, the proposed method achieves an accuracy rate of 94.6%. The code is available at https://github.com/sergioortiz26/Cancer_overlapping_filter_WSI_images.
Collapse
Affiliation(s)
- Sergio Ortiz
- Department of Computer Architecture and Technology, University of Granada, E.T.S. de Ingenierías Informática y de Telecomunicación, C/ Periodista Daniel Saucedo Aranda S/N CP:18071 Granada, Spain.
| | - Ignacio Rojas-Valenzuela
- Department of Computer Architecture and Technology, University of Granada, E.T.S. de Ingenierías Informática y de Telecomunicación, C/ Periodista Daniel Saucedo Aranda S/N CP:18071 Granada, Spain
| | - Fernando Rojas
- Department of Computer Architecture and Technology, University of Granada, E.T.S. de Ingenierías Informática y de Telecomunicación, C/ Periodista Daniel Saucedo Aranda S/N CP:18071 Granada, Spain
| | - Olga Valenzuela
- Department of Applied Mathematics, University of Granada, Facultad de Ciencias, Avenida de la Fuente Nueva S/N CP:18071 Granada, Spain
| | - Luis Javier Herrera
- Department of Computer Architecture and Technology, University of Granada, E.T.S. de Ingenierías Informática y de Telecomunicación, C/ Periodista Daniel Saucedo Aranda S/N CP:18071 Granada, Spain
| | - Ignacio Rojas
- Department of Computer Architecture and Technology, University of Granada, E.T.S. de Ingenierías Informática y de Telecomunicación, C/ Periodista Daniel Saucedo Aranda S/N CP:18071 Granada, Spain.
| |
Collapse
|
50
|
Esce AR, Baca AL, Redemann JP, Rebbe RW, Schultz F, Agarwal S, Hanson JA, Olson GT, Martin DR, Boyd NH. Predicting nodal metastases in squamous cell carcinoma of the oral tongue using artificial intelligence. Am J Otolaryngol 2024; 45:104102. [PMID: 37948827 DOI: 10.1016/j.amjoto.2023.104102] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 10/29/2023] [Indexed: 11/12/2023]
Abstract
OBJECTIVE The presence of occult nodal metastases in patients with squamous cell carcinoma (SCC) of the oral tongue has implications for treatment. Upwards of 30% of patients will have occult nodal metastases, yet a significant number of patients undergo unnecessary neck dissection to confirm nodal status. This study sought to predict the presence of nodal metastases in patients with SCC of the oral tongue using a convolutional neural network (CNN) that analyzed visual histopathology from the primary tumor alone. METHODS Cases of SCC of the oral tongue were identified from the records of a single institution. Only patients with complete pathology data were included in the study. The primary tumors were randomized into 2 groups for training and testing, which was performed at 2 different levels of supervision. Board-certified pathologists annotated each slide. HALO-AI convolutional neural network and image software was used to perform training and testing. Receiver operator characteristic (ROC) curves and the Youden J statistic were used for primary analysis. RESULTS Eighty-nine cases of SCC of the oral tongue were included in the study. The best performing algorithm had a high level of supervision and a sensitivity of 65% and specificity of 86% when identifying nodal metastases. The area under the curve (AUC) of the ROC curve for this algorithm was 0.729. CONCLUSION A CNN can produce an algorithm that is able to predict nodal metastases in patients with squamous cell carcinoma of the oral tongue by analyzing the visual histopathology of the primary tumor alone.
Collapse
Affiliation(s)
- Antoinette R Esce
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, 1 University of New Mexico, MSC10 5610, Albuquerque, NM, 87131, USA.
| | - Andrewe L Baca
- The University of New Mexico School of Medicine, 1 University of New Mexico, MSC08 4720, Albuquerque, NM 87131, USA
| | - Jordan P Redemann
- Department of Pathology, 1 University of New Mexico, MSC08 4640, Albuquerque, NM, 87131, USA.
| | - Ryan W Rebbe
- Department of Pathology, 1 University of New Mexico, MSC08 4640, Albuquerque, NM, 87131, USA.
| | - Fred Schultz
- Department of Pathology, 1 University of New Mexico, MSC08 4640, Albuquerque, NM, 87131, USA.
| | - Shweta Agarwal
- Department of Pathology, 1 University of New Mexico, MSC08 4640, Albuquerque, NM, 87131, USA.
| | - Joshua A Hanson
- Department of Pathology, 1 University of New Mexico, MSC08 4640, Albuquerque, NM, 87131, USA.
| | - Garth T Olson
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, 1 University of New Mexico, MSC10 5610, Albuquerque, NM, 87131, USA.
| | - David R Martin
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, 1 University of New Mexico, MSC10 5610, Albuquerque, NM, 87131, USA.
| | - Nathan H Boyd
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, 1 University of New Mexico, MSC10 5610, Albuquerque, NM, 87131, USA.
| |
Collapse
|