1
|
Wagner SJ, Matek C, Shetab Boushehri S, Boxberg M, Lamm L, Sadafi A, Winter DJE, Marr C, Peng T. Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology. Mod Pathol 2024; 37:100350. [PMID: 37827448 DOI: 10.1016/j.modpat.2023.100350] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 10/02/2023] [Accepted: 10/03/2023] [Indexed: 10/14/2023]
Abstract
Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.
Collapse
Affiliation(s)
- Sophia J Wagner
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; School of Computation, Information and Technology, Technical University of Munich, Garching, Germany
| | - Christian Matek
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Institute of Pathology, University Hospital Erlangen, Erlangen, Germany
| | - Sayedali Shetab Boushehri
- School of Computation, Information and Technology, Technical University of Munich, Garching, Germany; Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Data & Analytics (D&A), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Germany
| | - Melanie Boxberg
- Institute of Pathology, Technical University Munich, Munich, Germany; Institute of Pathology Munich-North, Munich, Germany
| | - Lorenz Lamm
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Helmholtz Pioneer Campus, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany
| | - Ario Sadafi
- School of Computation, Information and Technology, Technical University of Munich, Garching, Germany; Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany
| | - Dominik J E Winter
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; School of Life Sciences, Technical University of Munich, Weihenstephan, Germany
| | - Carsten Marr
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany.
| | - Tingying Peng
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany.
| |
Collapse
|
2
|
Jiang P, Liu J, Luo Q, Pang B, Xiao D, Cao D. Development of Automatic Portable Pathology Scanner and Its Evaluation for Clinical Practice. J Digit Imaging 2023; 36:1110-1122. [PMID: 36604365 PMCID: PMC10287606 DOI: 10.1007/s10278-022-00761-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 09/01/2022] [Accepted: 12/15/2022] [Indexed: 01/07/2023] Open
Abstract
Digital pathological scanners transform traditional glass slides into whole slide images (WSIs), which significantly improve the efficiency of pathological diagnosis and promote the development of digital pathology. However, the huge economic burden limits the spread and application of general WSI scanners in relatively remote and backward regions. In this paper, we develop an automatic portable cytopathology scanner based on mobile internet, Landing-Smart, to avert the above problems. Landing-Smart is a tiny device with a size of 208 mm × 107 mm × 104 mm and a weight of 1.8 kg, which integrates four main components including a smartphone, a glass slide carrier, an electric controller, and an optical imaging unit. By leveraging a simple optical imaging unit to substitute the sophisticated but complex conventional light microscope, the cost of Landing-Smart is less than $3000, much cheaper than general WSI scanners. On the one hand, Landing-Smart utilizes the built-in camera of the smartphone to acquire field of views (FoVs) in the section one by one. On the other hand, it uploads the images to the cloud server in real time via mobile internet, where the image processing and stitching method is implemented to generate the WSI of the cytological sample. The practical assessment of 209 cervical cytological specimens has demonstrated that Landing-Smart is comparable to general digital scanners in cytopathology diagnosis. Landing-Smart provides an effective tool for preliminary cytological screening in underdeveloped areas.
Collapse
Affiliation(s)
- Peng Jiang
- Institute of Artificial Intelligence, National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, 430072, China
| | - Juan Liu
- Institute of Artificial Intelligence, National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, 430072, China.
| | - Qiang Luo
- Landing Artificial Intelligence Center for Pathological Diagnosis, Wuhan, China
| | - Baochuan Pang
- Landing Artificial Intelligence Center for Pathological Diagnosis, Wuhan, China
| | - Di Xiao
- Landing Artificial Intelligence Center for Pathological Diagnosis, Wuhan, China
| | - Dehua Cao
- Landing Artificial Intelligence Center for Pathological Diagnosis, Wuhan, China
| |
Collapse
|
3
|
Plass M, Kargl M, Kiehl TR, Regitnig P, Geißler C, Evans T, Zerbe N, Carvalho R, Holzinger A, Müller H. Explainability and causability in digital pathology. J Pathol Clin Res 2023. [PMID: 37045794 DOI: 10.1002/cjp2.322] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 02/17/2023] [Accepted: 03/16/2023] [Indexed: 04/14/2023]
Abstract
The current move towards digital pathology enables pathologists to use artificial intelligence (AI)-based computer programmes for the advanced analysis of whole slide images. However, currently, the best-performing AI algorithms for image analysis are deemed black boxes since it remains - even to their developers - often unclear why the algorithm delivered a particular result. Especially in medicine, a better understanding of algorithmic decisions is essential to avoid mistakes and adverse effects on patients. This review article aims to provide medical experts with insights on the issue of explainability in digital pathology. A short introduction to the relevant underlying core concepts of machine learning shall nurture the reader's understanding of why explainability is a specific issue in this field. Addressing this issue of explainability, the rapidly evolving research field of explainable AI (XAI) has developed many techniques and methods to make black-box machine-learning systems more transparent. These XAI methods are a first step towards making black-box AI systems understandable by humans. However, we argue that an explanation interface must complement these explainable models to make their results useful to human stakeholders and achieve a high level of causability, i.e. a high level of causal understanding by the user. This is especially relevant in the medical field since explainability and causability play a crucial role also for compliance with regulatory requirements. We conclude by promoting the need for novel user interfaces for AI applications in pathology, which enable contextual understanding and allow the medical expert to ask interactive 'what-if'-questions. In pathology, such user interfaces will not only be important to achieve a high level of causability. They will also be crucial for keeping the human-in-the-loop and bringing medical experts' experience and conceptual knowledge to AI processes.
Collapse
Affiliation(s)
- Markus Plass
- Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria
| | - Michaela Kargl
- Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria
| | - Tim-Rasmus Kiehl
- Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany
| | - Peter Regitnig
- Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria
| | - Christian Geißler
- DAI-Labor, Agent Oriented Technologies (AOT), Technische Universität Berlin, Berlin, Germany
| | - Theodore Evans
- DAI-Labor, Agent Oriented Technologies (AOT), Technische Universität Berlin, Berlin, Germany
| | - Norman Zerbe
- Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany
| | - Rita Carvalho
- Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany
| | - Andreas Holzinger
- Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria
- Human-Centered AI Lab, University of Natural Resources and Life Sciences Vienna, Vienna, Austria
| | - Heimo Müller
- Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria
| |
Collapse
|
4
|
Using Deep Learning to Predict Final HER2 Status in Invasive Breast Cancers That are Equivocal (2+) by Immunohistochemistry. Appl Immunohistochem Mol Morphol 2022; 30:668-673. [PMID: 36251973 DOI: 10.1097/pai.0000000000001079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 09/17/2022] [Indexed: 11/25/2022]
Abstract
Invasive breast carcinomas are routinely tested for HER2 using immunohistochemistry (IHC), with reflex in situ hybridization (ISH) for those scored as equivocal (2+). ISH testing is expensive, time-consuming, and not universally available. In this study, we trained a deep learning algorithm to directly predict HER2 gene amplification status from HER2 2+ IHC slides. Data included 115 consecutive cases of invasive breast carcinoma scored as 2+ by IHC that had follow-up HER2 ISH testing. An external validation data set was created from 36 HER2 IHC slides prepared at an outside institution. All internal IHC slides were digitized and divided into training (80%), and test (20%) sets with 5-fold cross-validation. Small patches (256×256 pixels) were randomly extracted and used to train convolutional neural networks with EfficientNet B0 architecture using a transfer learning approach. Predictions for slides in the test set were made on individual patches, and these predictions were aggregated to generate an overall prediction for each slide. This resulted in a receiver operating characteristic area under the curve of 0.83 with an overall accuracy of 79% (sensitivity=0.70, specificity=0.82). Analysis of external validation slides resulted in a receiver operating characteristic area under the curve of 0.79 with an overall accuracy of 81% (sensitivity=0.50, specificity=0.82). Although the sensitivity and specificity are not high enough to negate the need for reflexive ISH testing entirely, this approach may be useful for triaging cases more likely to be HER2 positive and initiating treatment planning in centers where HER2 ISH testing is not readily available.
Collapse
|
5
|
Zhu J, Liu M, Li X. Progress on deep learning in digital pathology of breast cancer: a narrative review. Gland Surg 2022; 11:751-766. [PMID: 35531111 PMCID: PMC9068546 DOI: 10.21037/gs-22-11] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Accepted: 03/04/2022] [Indexed: 01/26/2024]
Abstract
BACKGROUND AND OBJECTIVE Pathology is the gold standard criteria for breast cancer diagnosis and has important guiding value in formulating the clinical treatment plan and predicting the prognosis. However, traditional microscopic examinations of tissue sections are time consuming and labor intensive, with unavoidable subjective variations. Deep learning (DL) can evaluate and extract the most important information from images with less need for human instruction, providing a promising approach to assist in the pathological diagnosis of breast cancer. To provide an informative and up-to-date summary on the topic of DL-based diagnostic systems for breast cancer pathology image analysis and discuss the advantages and challenges to the routine clinical application of digital pathology. METHODS A PubMed search with keywords ("breast neoplasm" or "breast cancer") and ("pathology" or "histopathology") and ("artificial intelligence" or "deep learning") was conducted. Relevant publications in English published from January 2000 to October 2021 were screened manually for their title, abstract, and even full text to determine their true relevance. References from the searched articles and other supplementary articles were also studied. KEY CONTENT AND FINDINGS DL-based computerized image analysis has obtained impressive achievements in breast cancer pathology diagnosis, classification, grading, staging, and prognostic prediction, providing powerful methods for faster, more reproducible, and more precise diagnoses. However, all artificial intelligence (AI)-assisted pathology diagnostic models are still in the experimental stage. Improving their economic efficiency and clinical adaptability are still required to be developed as the focus of further researches. CONCLUSIONS Having searched PubMed and other databases and summarized the application of DL-based AI models in breast cancer pathology, we conclude that DL is undoubtedly a promising tool for assisting pathologists in routines, but further studies are needed to realize the digitization and automation of clinical pathology.
Collapse
Affiliation(s)
- Jingjin Zhu
- School of Medicine, Nankai University, Tianjin, China
| | - Mei Liu
- Department of Pathology, Chinese People’s Liberation Army General Hospital, Beijing, China
| | - Xiru Li
- Department of General Surgery, Chinese People’s Liberation Army General Hospital, Beijing, China
| |
Collapse
|
6
|
Topalovic N, Mazic S, Nesic D, Vukovic O, Cumic J, Laketic D, Stasevic Karlicic I, Pantic I. Association between Chromatin Structural Organization of Peripheral Blood Neutrophils and Self-Perceived Mental Stress: Gray-Level Co-occurrence Matrix Analysis. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2021; 27:1-7. [PMID: 34334154 DOI: 10.1017/s143192762101240x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Methods based on the evaluation of textural patterns in microscopy, such as the “gray-level co-occurrence matrix” (GLCM) analysis are modern and innovative computer and mathematical algorithms that can be used to quantify subtle structural changes in cells and their organelles. Potential application of GLCM method in the fields of psychophysiology and psychiatry to this date has not been systematically investigated. The main objective of our study was to test the existence and strength of the association between chromatin structural organization of peripheral blood neutrophils and levels of self-perceived mental stress. The research was done on a sample of 100 healthy student athletes, and the Depression, Anxiety, and Stress Scales (DASS-21) were used for the estimation of psychological distress. Chromatin textural homogeneity and uniformity were negatively correlated (p < 0.01) with mental distress and had relatively good discriminatory power in differentiating participants with normal and elevated stress levels. As an addition, we propose the creation of a machine learning model based on binomial logistic regression that uses these and other GLCM features to predict stress elevation. To the best of our knowledge, these results are one of the first to establish the link between neutrophil chromatin structural organization quantified by the GLCM method and indicators of normal psychological functioning.
Collapse
Affiliation(s)
- Nikola Topalovic
- University of Belgrade, Faculty of Medicine, Institute of Medical Physiology, Visegradska 26/II, RS-11129, Belgrade, Serbia
| | - Sanja Mazic
- University of Belgrade, Faculty of Medicine, Institute of Medical Physiology, Visegradska 26/II, RS-11129, Belgrade, Serbia
| | - Dejan Nesic
- University of Belgrade, Faculty of Medicine, Institute of Medical Physiology, Visegradska 26/II, RS-11129, Belgrade, Serbia
| | - Olivera Vukovic
- University of Belgrade, Faculty of Medicine, Institute of Mental Health, Palmoticeva 37, RS-11000, Belgrade, Serbia
| | - Jelena Cumic
- University of Belgrade, Faculty of Medicine, University Clinical Centre of Serbia, Dr. Koste Todorovica 8, RS-11129, Belgrade, Serbia
| | - Darko Laketic
- University of Belgrade, Faculty of Medicine, Institute of Anatomy, Dr Subotica 4/2, RS-11129, Belgrade, Serbia
| | | | - Igor Pantic
- University of Belgrade, Faculty of Medicine, Institute of Medical Physiology, Visegradska 26/II, RS-11129, Belgrade, Serbia
- University of Haifa, 199 Abba Hushi Blvd. Mount Carmel, HaifaIL-3498838, Israel
| |
Collapse
|