1
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
2
|
McCaffrey C, Jahangir C, Murphy C, Burke C, Gallagher WM, Rahman A. Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer. Expert Rev Mol Diagn 2024; 24:363-377. [PMID: 38655907 DOI: 10.1080/14737159.2024.2346545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Accepted: 04/19/2024] [Indexed: 04/26/2024]
Abstract
INTRODUCTION Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes. AREAS COVERED In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings. EXPERT OPINION The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.
Collapse
Affiliation(s)
- Christine McCaffrey
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Chowdhury Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Clodagh Murphy
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Caoimbhe Burke
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Arman Rahman
- UCD School of Medicine, UCD Conway Institute, University College Dublin, Dublin, Ireland
| |
Collapse
|
3
|
Brancato V, Esposito G, Coppola L, Cavaliere C, Mirabelli P, Scapicchio C, Borgheresi R, Neri E, Salvatore M, Aiello M. Standardizing digital biobanks: integrating imaging, genomic, and clinical data for precision medicine. J Transl Med 2024; 22:136. [PMID: 38317237 PMCID: PMC10845786 DOI: 10.1186/s12967-024-04891-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 01/14/2024] [Indexed: 02/07/2024] Open
Abstract
Advancements in data acquisition and computational methods are generating a large amount of heterogeneous biomedical data from diagnostic domains such as clinical imaging, pathology, and next-generation sequencing (NGS), which help characterize individual differences in patients. However, this information needs to be available and suitable to promote and support scientific research and technological development, supporting the effective adoption of the precision medicine approach in clinical practice. Digital biobanks can catalyze this process, facilitating the sharing of curated and standardized imaging data, clinical, pathological and molecular data, crucial to enable the development of a comprehensive and personalized data-driven diagnostic approach in disease management and fostering the development of computational predictive models. This work aims to frame this perspective, first by evaluating the state of standardization of individual diagnostic domains and then by identifying challenges and proposing a possible solution towards an integrative approach that can guarantee the suitability of information that can be shared through a digital biobank. Our analysis of the state of the art shows the presence and use of reference standards in biobanks and, generally, digital repositories for each specific domain. Despite this, standardization to guarantee the integration and reproducibility of the numerical descriptors generated by each domain, e.g. radiomic, pathomic and -omic features, is still an open challenge. Based on specific use cases and scenarios, an integration model, based on the JSON format, is proposed that can help address this problem. Ultimately, this work shows how, with specific standardization and promotion efforts, the digital biobank model can become an enabling technology for the comprehensive study of diseases and the effective development of data-driven technologies at the service of precision medicine.
Collapse
Affiliation(s)
| | - Giuseppina Esposito
- Bio Check Up S.R.L, 80121, Naples, Italy
- Department of Advanced Biomedical Sciences, University of Naples Federico II, 80131, Naples, Italy
| | | | | | - Peppino Mirabelli
- UOS Laboratori di Ricerca e Biobanca, AORN Santobono-Pausilipon, Via Teresa Ravaschieri, 8, 80122, Naples, Italy
| | - Camilla Scapicchio
- Academic Radiology, Department of Translational Research, University of Pisa, via Roma, 67, 56126, Pisa, Italy
| | - Rita Borgheresi
- Academic Radiology, Department of Translational Research, University of Pisa, via Roma, 67, 56126, Pisa, Italy
| | - Emanuele Neri
- Academic Radiology, Department of Translational Research, University of Pisa, via Roma, 67, 56126, Pisa, Italy
| | | | | |
Collapse
|
4
|
Saednia K, Tran WT, Sadeghi-Naini A. A hierarchical self-attention-guided deep learning framework to predict breast cancer response to chemotherapy using pre-treatment tumor biopsies. Med Phys 2023; 50:7852-7864. [PMID: 37403567 DOI: 10.1002/mp.16574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 06/06/2023] [Accepted: 06/10/2023] [Indexed: 07/06/2023] Open
Abstract
BACKGROUND Pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) has demonstrated a strong correlation to improved survival in breast cancer (BC) patients. However, pCR rates to NAC are less than 30%, depending on the BC subtype. Early prediction of NAC response would facilitate therapeutic modifications for individual patients, potentially improving overall treatment outcomes and patient survival. PURPOSE This study, for the first time, proposes a hierarchical self-attention-guided deep learning framework to predict NAC response in breast cancer patients using digital histopathological images of pre-treatment biopsy specimens. METHODS Digitized hematoxylin and eosin-stained slides of BC core needle biopsies were obtained from 207 patients treated with NAC, followed by surgery. The response to NAC for each patient was determined using the standard clinical and pathological criteria after surgery. The digital pathology images were processed through the proposed hierarchical framework consisting of patch-level and tumor-level processing modules followed by a patient-level response prediction component. A combination of convolutional layers and transformer self-attention blocks were utilized in the patch-level processing architecture to generate optimized feature maps. The feature maps were analyzed through two vision transformer architectures adapted for the tumor-level processing and the patient-level response prediction components. The feature map sequences for these transformer architectures were defined based on the patch positions within the tumor beds and the bed positions within the biopsy slide, respectively. A five-fold cross-validation at the patient level was applied on the training set (144 patients with 9430 annotated tumor beds and 1,559,784 patches) to train the models and optimize the hyperparameters. An unseen independent test set (63 patients with 3574 annotated tumor beds and 173,637 patches) was used to evaluate the framework. RESULTS The obtained results on the test set showed an AUC of 0.89 and an F1-score of 90% for predicting pCR to NAC a priori by the proposed hierarchical framework. Similar frameworks with the patch-level, patch-level + tumor-level, and patch-level + patient-level processing components resulted in AUCs of 0.79, 0.81, and 0.84 and F1-scores of 86%, 87%, and 89%, respectively. CONCLUSIONS The results demonstrate a high potential of the proposed hierarchical deep-learning methodology for analyzing digital pathology images of pre-treatment tumor biopsies to predict the pathological response of breast cancer to NAC.
Collapse
Affiliation(s)
- Khadijeh Saednia
- Department of Electrical Engineering and Computer Science, Lassonde School of Engineering, York University, Toronto, Ontario, Canada
- Department of Radiation Oncology, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
| | - William T Tran
- Department of Radiation Oncology, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, Ontario, Canada
- Temerity Centre for AI Research and Education in Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Ali Sadeghi-Naini
- Department of Electrical Engineering and Computer Science, Lassonde School of Engineering, York University, Toronto, Ontario, Canada
- Department of Radiation Oncology, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
- Temerity Centre for AI Research and Education in Medicine, University of Toronto, Toronto, Ontario, Canada
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, Ontario, Canada
| |
Collapse
|
5
|
Gorman C, Punzo D, Octaviano I, Pieper S, Longabaugh WJR, Clunie DA, Kikinis R, Fedorov AY, Herrmann MD. Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology. Nat Commun 2023; 14:1572. [PMID: 36949078 PMCID: PMC10033920 DOI: 10.1038/s41467-023-37224-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 03/08/2023] [Indexed: 03/24/2023] Open
Abstract
The exchange of large and complex slide microscopy imaging data in biomedical research and pathology practice is impeded by a lack of data standardization and interoperability, which is detrimental to the reproducibility of scientific findings and clinical integration of technological innovations. We introduce Slim, an open-source, web-based slide microscopy viewer that implements the internationally accepted Digital Imaging and Communications in Medicine (DICOM) standard to achieve interoperability with a multitude of existing medical imaging systems. We showcase the capabilities of Slim as the slide microscopy viewer of the NCI Imaging Data Commons and demonstrate how the viewer enables interactive visualization of traditional brightfield microscopy and highly-multiplexed immunofluorescence microscopy images from The Cancer Genome Atlas and Human Tissue Atlas Network, respectively, using standard DICOMweb services. We further show how Slim enables the collection of standardized image annotations for the development or validation of machine learning models and the visual interpretation of model inference results in the form of segmentation masks, spatial heat maps, or image-derived measurements.
Collapse
Affiliation(s)
- Chris Gorman
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | | | | | | | | | | | - Ron Kikinis
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA
| | - Andrey Y Fedorov
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA.
| | - Markus D Herrmann
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
6
|
Bridge CP, Gorman C, Pieper S, Doyle SW, Lennerz JK, Kalpathy-Cramer J, Clunie DA, Fedorov AY, Herrmann MD. Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and Radiology. J Digit Imaging 2022; 35:1719-1737. [PMID: 35995898 PMCID: PMC9712874 DOI: 10.1007/s10278-022-00683-y] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Revised: 05/20/2022] [Accepted: 05/26/2022] [Indexed: 10/15/2022] Open
Abstract
Machine learning (ML) is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but the lack of interoperability between ML systems and enterprise medical imaging systems has been a major barrier for clinical integration and evaluation. The DICOM® standard specifies information object definitions (IODs) and services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software libraries and tools that simplify working with datasets in DICOM format. Here we present the highdicom library, which provides a high-level application programming interface (API) for the Python programming language that abstracts low-level details of the standard and enables encoding and decoding of image-derived information in DICOM format in a few lines of Python code. The highdicom library leverages NumPy arrays for efficient data representation and ties into the extensive Python ecosystem for image processing and machine learning. Simultaneously, by simplifying creation and parsing of DICOM-compliant files, highdicom achieves interoperability with the medical imaging systems that hold the data used to train and run ML models, and ultimately communicate and store model outputs for clinical use. We demonstrate through experiments with slide microscopy and computed tomography imaging, that, by bridging these two ecosystems, highdicom enables developers and researchers to train and evaluate state-of-the-art ML models in pathology and radiology while remaining compliant with the DICOM standard and interoperable with clinical systems at all stages. To promote standardization of ML research and streamline the ML model development and deployment process, we made the library available free and open-source at https://github.com/herrmannlab/highdicom .
Collapse
Affiliation(s)
- Christopher P Bridge
- Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA
- MGH & BWH Center for Clinical Data Science, Mass General Brigham, Boston, MA, USA
| | - Chris Gorman
- Computational Pathology, Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | | | - Sean W Doyle
- MGH & BWH Center for Clinical Data Science, Mass General Brigham, Boston, MA, USA
| | - Jochen K Lennerz
- Center for Integrated Diagnostics, Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
- Department of Pathology, Harvard Medical School, Boston, MA, USA
| | - Jayashree Kalpathy-Cramer
- Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA
- MGH & BWH Center for Clinical Data Science, Mass General Brigham, Boston, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | | | - Andriy Y Fedorov
- Department of Radiology, Harvard Medical School, Boston, MA, USA
- Surgical Planning Laboratory, Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA
| | - Markus D Herrmann
- Computational Pathology, Department of Pathology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Pathology, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
7
|
Kuczkiewicz-Siemion O, Sokół K, Puton B, Borkowska A, Szumera-Ciećkiewicz A. The Role of Pathology-Based Methods in Qualitative and Quantitative Approaches to Cancer Immunotherapy. Cancers (Basel) 2022; 14:cancers14153833. [PMID: 35954496 PMCID: PMC9367614 DOI: 10.3390/cancers14153833] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Revised: 07/25/2022] [Accepted: 08/02/2022] [Indexed: 11/16/2022] Open
Abstract
Simple Summary Immunotherapy has become the filar of modern oncological treatment, and programmed death-ligand 1 expression is one of the primary immune markers assessed by pathologists. However, there are still some issues concerning the evaluation of the marker and limited information about the interaction between the tumour and associated immune cells. Recent studies have focused on cancer immunology to try to understand the complex tumour microenvironment, and multiplex imaging methods are more widely used for this purpose. The presented article aims to provide an overall review of a different multiplex in situ method using spectral imaging, supported by automated image-acquisition and software-assisted marker visualisation and interpretation. Multiplex imaging methods could improve the current understanding of complex tumour-microenvironment immunology and could probably help to better match patients to appropriate treatment regimens. Abstract Immune checkpoint inhibitors, including those concerning programmed cell death 1 (PD-1) and its ligand (PD-L1), have revolutionised the cancer therapy approach in the past decade. However, not all patients benefit from immunotherapy equally. The prediction of patient response to this type of therapy is mainly based on conventional immunohistochemistry, which is limited by intraobserver variability, semiquantitative assessment, or single-marker-per-slide evaluation. Multiplex imaging techniques and digital image analysis are powerful tools that could overcome some issues concerning tumour-microenvironment studies. This novel approach to biomarker assessment offers a better understanding of the complicated interactions between tumour cells and their environment. Multiplex labelling enables the detection of multiple markers simultaneously and the exploration of their spatial organisation. Evaluating a variety of immune cell phenotypes and differentiating their subpopulations is possible while preserving tissue histology in most cases. Multiplexing supported by digital pathology could allow pathologists to visualise and understand every cell in a single tissue slide and provide meaning in a complex tumour-microenvironment contexture. This review aims to provide an overview of the different multiplex imaging methods and their application in PD-L1 biomarker assessment. Moreover, we discuss digital imaging techniques, with a focus on slide scanners and software.
Collapse
Affiliation(s)
- Olga Kuczkiewicz-Siemion
- Department of Pathology, Maria Sklodowska-Curie National Research Institute of Oncology, 02-781 Warsaw, Poland
- Diagnostic Hematology Department, Institute of Hematology and Transfusion Medicine, 02-776 Warsaw, Poland
- Correspondence: (O.K.-S.); (A.S.-C.)
| | - Kamil Sokół
- Diagnostic Hematology Department, Institute of Hematology and Transfusion Medicine, 02-776 Warsaw, Poland
| | - Beata Puton
- Department of Pathology, Maria Sklodowska-Curie National Research Institute of Oncology, 02-781 Warsaw, Poland
| | - Aneta Borkowska
- Department of Soft Tissue/Bone Sarcoma and Melanoma, Maria Sklodowska-Curie National Research Institute of Oncology, 02-781 Warsaw, Poland
| | - Anna Szumera-Ciećkiewicz
- Department of Pathology, Maria Sklodowska-Curie National Research Institute of Oncology, 02-781 Warsaw, Poland
- Correspondence: (O.K.-S.); (A.S.-C.)
| |
Collapse
|
8
|
Hanna MG, Ardon O, Reuter VE, Sirintrapun SJ, England C, Klimstra DS, Hameed MR. Integrating digital pathology into clinical practice. Mod Pathol 2022; 35:152-164. [PMID: 34599281 DOI: 10.1038/s41379-021-00929-0] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 09/03/2021] [Accepted: 09/12/2021] [Indexed: 11/09/2022]
Abstract
The field of anatomic pathology has been evolving in the last few decades and the advancements have been largely fostered by innovative technology. Immunohistochemistry enabled a paradigm shift in discovery and diagnostic evaluation, followed by booming genomic advancements which allowed for submicroscopic pathologic characterization, and now the field of digital pathology coupled with machine learning and big data acquisition is paving the way to revolutionize the pathology medical domain. Whole slide imaging (WSI) is a disruptive technology where glass slides are digitized to produce on-screen whole slide images. Specifically, in the past decade, there have been significant advances in digital pathology systems that have allowed this technology to promote integration into clinical practice. Whole slide images (WSI), or digital slides, can be viewed and navigated comparable to glass slides on a microscope, as digital files. Whole slide imaging has increased in adoption among pathologists, pathology departments, and scientists for clinical, educational, and research initiatives. Integration of digital pathology systems requires a coordinated effort with numerous stakeholders, not only within the pathology department, but across the entire enterprise. Each pathology department has distinct needs, use cases and blueprints, however the framework components and variables for successful clinical integration can be generalized across any organization seeking to undergo a digital transformation at any scale. This article will review those components and considerations for integrating digital pathology systems into clinical practice.
Collapse
Affiliation(s)
- Matthew G Hanna
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
| | - Orly Ardon
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Victor E Reuter
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Christine England
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - David S Klimstra
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Meera R Hameed
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
| |
Collapse
|
9
|
Zuraw A, Aeffner F. Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review. Vet Pathol 2021; 59:6-25. [PMID: 34521285 DOI: 10.1177/03009858211040484] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Since whole-slide imaging has been commercially available for over 2 decades, digital pathology has become a constantly expanding aspect of the pathology profession that will continue to significantly impact how pathologists conduct their craft. While some aspects, such as whole-slide imaging for archiving, consulting, and teaching, have gained broader acceptance, other facets such as quantitative tissue image analysis and artificial intelligence-based assessments are still met with some reservations. While most vendors in this space have focused on diagnostic applications, that is, viewing one or few slides at a time, some are developing solutions tailored more specifically to the various aspects of veterinary pathology including updated diagnostic, discovery, and research applications. This has especially advanced the use of digital pathology in toxicologic pathology and drug development, for primary reads as well as peer reviews. It is crucial that pathologists gain a deeper understanding of digital pathology and tissue image analysis technology and their applications in order to fully use these tools in a way that enhances and improves the pathologist's assessment as well as work environment. This review focuses on an updated introduction to the basics of digital pathology and image analysis and introduces emerging topics around artificial intelligence and machine learning.
Collapse
Affiliation(s)
| | - Famke Aeffner
- Amgen Inc, Amgen Research, South San Francisco, CA, USA
| |
Collapse
|
10
|
Roth CJ, Clunie DA, Vining DJ, Berkowitz SJ, Berlin A, Bissonnette JP, Clark SD, Cornish TC, Eid M, Gaskin CM, Goel AK, Jacobs GC, Kwan D, Luviano DM, McBee MP, Miller K, Hafiz AM, Obcemea C, Parwani AV, Rotemberg V, Silver EL, Storm ES, Tcheng JE, Thullner KS, Folio LR. Multispecialty Enterprise Imaging Workgroup Consensus on Interactive Multimedia Reporting Current State and Road to the Future: HIMSS-SIIM Collaborative White Paper. J Digit Imaging 2021; 34:495-522. [PMID: 34131793 PMCID: PMC8329131 DOI: 10.1007/s10278-021-00450-5] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Revised: 03/05/2021] [Accepted: 03/19/2021] [Indexed: 12/20/2022] Open
Abstract
Diagnostic and evidential static image, video clip, and sound multimedia are captured during routine clinical care in cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, endoscopic procedural specialties, and other medical disciplines. Providers typically describe the multimedia findings in contemporaneous electronic health record clinical notes or associate a textual interpretative report. Visual communication aids commonly used to connect, synthesize, and supplement multimedia and descriptive text outside medicine remain technically challenging to integrate into patient care. Such beneficial interactive elements may include hyperlinks between text, multimedia elements, alphanumeric and geometric annotations, tables, graphs, timelines, diagrams, anatomic maps, and hyperlinks to external educational references that patients or provider consumers may find valuable. This HIMSS-SIIM Enterprise Imaging Community workgroup white paper outlines the current and desired clinical future state of interactive multimedia reporting (IMR). The workgroup adopted a consensus definition of IMR as “interactive medical documentation that combines clinical images, videos, sound, imaging metadata, and/or image annotations with text, typographic emphases, tables, graphs, event timelines, anatomic maps, hyperlinks, and/or educational resources to optimize communication between medical professionals, and between medical professionals and their patients.” This white paper also serves as a precursor for future efforts toward solving technical issues impeding routine interactive multimedia report creation and ingestion into electronic health records.
Collapse
Affiliation(s)
| | | | - David J Vining
- Department of Abdominal Imaging, MD Anderson Cancer Center, Houston, TX, USA
| | - Seth J Berkowitz
- Department of Radiology, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Alejandro Berlin
- Radiation Medicine Program, Princess Margaret Cancer Centre - University Health Network, Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| | - Jean-Pierre Bissonnette
- Departments of Radiation Oncology and Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Shawn D Clark
- University of Miami Hospitals and Clinics, Miami, FL, USA
| | - Toby C Cornish
- Department of Pathology, University of Colorado School of Medicine, Aurora, CO, USA
| | - Monief Eid
- eHealth & Digital Transformation Agency, Ministry of Health, Riyadh, Saudi Arabia
| | - Cree M Gaskin
- Department of Radiology and Medical Imaging, University of Virginia, Charlottesville, VA, USA
| | | | | | - David Kwan
- Health Technology and Information Management, Ontario Health (Cancer Care Ontario), Toronto, ON, Canada
| | - Damien M Luviano
- Department of Surgery, Virginia Tech Carilion School of Medicine, Roanoke, VA, USA
| | - Morgan P McBee
- Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | | | - Abdul Moiz Hafiz
- Division of Cardiology, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Ceferino Obcemea
- Radiation Research Program, National Cancer Institute, Bethesda, MD, USA
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Veronica Rotemberg
- Dermatology Service, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Erik S Storm
- Department of Radiology and Medical Education, Salem VA Medical Center, Salem, VA, USA
| | - James E Tcheng
- Department of Medicine, Division of Cardiology, Duke University, Durham, NC, USA
| | | | - Les R Folio
- Lead CT Radiologist, NIH Clinical Center, Bethesda, MD, USA
| |
Collapse
|
11
|
Gu Q, Prodduturi N, Jiang J, Flotte TJ, Hart SN. Dicom_wsi: A Python Implementation for Converting Whole-Slide Images to Digital Imaging and Communications in Medicine Compliant Files. J Pathol Inform 2021; 12:21. [PMID: 34267986 PMCID: PMC8274303 DOI: 10.4103/jpi.jpi_88_20] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2020] [Revised: 01/19/2021] [Accepted: 02/27/2021] [Indexed: 12/23/2022] Open
Abstract
Background: Adoption of the Digital Imaging and Communications in Medicine (DICOM) standard for whole slide images (WSIs) has been slow, despite significant time and effort by standards curators. One reason for the lack of adoption is that there are few tools which exist that can meet the requirements of WSIs, given an evolving ecosystem of best practices for implementation. Eventually, vendors will conform to the specification to ensure enterprise interoperability, but what about archived slides? Millions of slides have been scanned in various proprietary formats, many with examples of rare histologies. Our hypothesis is that if users and developers had access to easy to use tools for migrating proprietary formats to the open DICOM standard, then more tools would be developed as DICOM first implementations. Methods: The technology we present here is dicom_wsi, a Python based toolkit for converting any slide capable of being read by the OpenSlide library into DICOM conformant and validated implementations. Moreover, additional postprocessing such as background removal, digital transformations (e.g., ink removal), and annotation storage are also described. dicom_wsi is a free and open source implementation that anyone can use or modify to meet their specific purposes. Results: We compare the output of dicom_wsi to two other existing implementations of WSI to DICOM converters and also validate the images using DICOM capable image viewers. Conclusion: dicom_wsi represents the first step in a long process of DICOM adoption for WSI. It is the first open source implementation released in the developer friendly Python programming language and can be freely downloaded at .
Collapse
Affiliation(s)
- Qiangqiang Gu
- Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, Mayo College of Medicine, Rochester, Minnesota, USA
| | - Naresh Prodduturi
- Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, Mayo College of Medicine, Rochester, Minnesota, USA
| | - Jun Jiang
- Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, Mayo College of Medicine, Rochester, Minnesota, USA
| | - Thomas J Flotte
- Department of Laboratory Medicine and Pathology, Mayo College of Medicine, Rochester, Minnesota, USA
| | - Steven N Hart
- Department of Health Sciences Research, Division of Biomedical Statistics and Informatics, Mayo College of Medicine, Rochester, Minnesota, USA
| |
Collapse
|
12
|
Xing F, Zhang X, Cornish TC. Artificial intelligence for pathology. Artif Intell Med 2021. [DOI: 10.1016/b978-0-12-821259-2.00011-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
13
|
Clunie DA. DICOM Format and Protocol Standardization-A Core Requirement for Digital Pathology Success. Toxicol Pathol 2020; 49:738-749. [PMID: 33063645 DOI: 10.1177/0192623320965893] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
As the use of digital techniques in toxicologic pathology expands, challenges of scalability and interoperability come to the fore. Proprietary formats and closed single-vendor platforms prevail but depend on the availability and maintenance of multiformat conversion libraries. Expedient for small deployments, this is not sustainable at an industrial scale. Primarily known as a standard for radiology, the Digital Imaging and Communications in Medicine (DICOM) standard has been evolving to support other specialties since its inception, to become the single ubiquitous standard throughout medical imaging. The adoption of DICOM for whole slide imaging (WSI) has been sluggish. Prospects for widespread commercially viable clinical use of digital pathology change the incentives. Connectathons using DICOM have demonstrated its feasibility for WSI and virtual microscopy. Adoption of DICOM for digital and computational pathology will allow the reuse of enterprise-wide infrastructure for storage, security, and business continuity. The DICOM embedded metadata allows detached files to remain useful. Bright-field and multichannel fluorescence, Z-stacks, cytology, and sparse and fully tiled encoding are supported. External terminologies and standard compression schemes are supported. Color consistency is defined using International Color Consortium profiles. The DICOM files can be dual personality Tagged Image File Format (TIFF) for legacy support. Annotations for computational pathology results can be encoded.
Collapse
|
14
|
Mazer BL, Paulson N, Sinard JH. Protecting the Pathology Commons in the Digital Era. Arch Pathol Lab Med 2020; 144:1037-1040. [PMID: 32579394 DOI: 10.5858/arpa.2020-0022-ed] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/12/2020] [Indexed: 11/06/2022]
Affiliation(s)
- Benjamin L Mazer
- From the Department of Pathology, Yale University School of Medicine, New Haven, Connecticut
| | - Nathan Paulson
- From the Department of Pathology, Yale University School of Medicine, New Haven, Connecticut
| | - John H Sinard
- From the Department of Pathology, Yale University School of Medicine, New Haven, Connecticut
| |
Collapse
|
15
|
Marble HD, Huang R, Dudgeon SN, Lowe A, Herrmann MD, Blakely S, Leavitt MO, Isaacs M, Hanna MG, Sharma A, Veetil J, Goldberg P, Schmid JH, Lasiter L, Gallas BD, Abels E, Lennerz JK. A Regulatory Science Initiative to Harmonize and Standardize Digital Pathology and Machine Learning Processes to Speed up Clinical Innovation to Patients. J Pathol Inform 2020; 11:22. [PMID: 33042601 PMCID: PMC7518200 DOI: 10.4103/jpi.jpi_27_20] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2020] [Revised: 04/20/2020] [Accepted: 06/16/2020] [Indexed: 12/13/2022] Open
Abstract
Unlocking the full potential of pathology data by gaining computational access to histological pixel data and metadata (digital pathology) is one of the key promises of computational pathology. Despite scientific progress and several regulatory approvals for primary diagnosis using whole-slide imaging, true clinical adoption at scale is slower than anticipated. In the U.S., advances in digital pathology are often siloed pursuits by individual stakeholders, and to our knowledge, there has not been a systematic approach to advance the field through a regulatory science initiative. The Alliance for Digital Pathology (the Alliance) is a recently established, volunteer, collaborative, regulatory science initiative to standardize digital pathology processes to speed up innovation to patients. The purpose is: (1) to account for the patient perspective by including patient advocacy; (2) to investigate and develop methods and tools for the evaluation of effectiveness, safety, and quality to specify risks and benefits in the precompetitive phase; (3) to help strategize the sequence of clinically meaningful deliverables; (4) to encourage and streamline the development of ground-truth data sets for machine learning model development and validation; and (5) to clarify regulatory pathways by investigating relevant regulatory science questions. The Alliance accepts participation from all stakeholders, and we solicit clinically relevant proposals that will benefit the field at large. The initiative will dissolve once a clinical, interoperable, modularized, integrated solution (from tissue acquisition to diagnostic algorithm) has been implemented. In times of rapidly evolving discoveries, scientific input from subject-matter experts is one essential element to inform regulatory guidance and decision-making. The Alliance aims to establish and promote synergistic regulatory science efforts that will leverage diverse inputs to move digital pathology forward and ultimately improve patient care.
Collapse
Affiliation(s)
- Hetal Desai Marble
- Department of Pathology, Center for Integrated Diagnostics, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
| | - Richard Huang
- Department of Pathology, Center for Integrated Diagnostics, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
| | - Sarah Nixon Dudgeon
- Division of Imaging, Diagnostics, and Software Reliability, Center for Devices and Radiological Health, Food and Drug Administration, Office of Science and Engineering Laboratories, Silver Spring, MD, USA
| | | | - Markus D Herrmann
- Department of Pathology, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
| | | | | | - Mike Isaacs
- Department of Pathology and Immunology, Washington University School of Medicine, St. Louis, MO, USA
| | - Matthew G Hanna
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Ashish Sharma
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Jithesh Veetil
- Medical Device Innovation Consortium, Arlington, VA, USA
| | | | | | | | - Brandon D Gallas
- Division of Imaging, Diagnostics, and Software Reliability, Center for Devices and Radiological Health, Food and Drug Administration, Office of Science and Engineering Laboratories, Silver Spring, MD, USA
| | | | - Jochen K Lennerz
- Department of Pathology, Center for Integrated Diagnostics, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|
16
|
Abstract
Quantitative biomarkers are key prognostic and predictive factors in the diagnosis and treatment of cancer. In the clinical laboratory, the majority of biomarker quantitation is still performed manually, but digital image analysis (DIA) methods have been steadily growing and account for around 25% of all quantitative immunohistochemistry (IHC) testing performed today. Quantitative DIA is primarily employed in the analysis of breast cancer IHC biomarkers, including estrogen receptor, progesterone receptor, and human epidermal growth factor receptor 2/neu; more recently clinical applications have expanded to include human epidermal growth factor receptor 2/neu in gastroesophageal adenocarcinomas and Ki-67 in both breast cancer and gastrointestinal and pancreatic neuroendocrine tumors. Evidence in the literature suggests that DIA has significant benefits over manual quantitation of IHC biomarkers, such as increased objectivity, accuracy, and reproducibility. Despite this fact, a number of barriers to the adoption of DIA in the clinical laboratory persist. These include difficulties in integrating DIA into clinical workflows, lack of standards for integrating DIA software with laboratory information systems and digital pathology systems, costs of implementing DIA, inadequate reimbursement relative to those costs, and other factors. These barriers to adoption may be overcome with international standards such as Digital Imaging and Communications in Medicine (DICOM), increased adoption of routine digital pathology workflows, the application of artificial intelligence to DIA, and the emergence of new clinical applications for DIA.
Collapse
|
17
|
Convergence of Digital Pathology and Artificial Intelligence Tools in Anatomic Pathology Practice: Current Landscape and Future Directions. Adv Anat Pathol 2020; 27:221-226. [PMID: 32541593 DOI: 10.1097/pap.0000000000000271] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
18
|
Abstract
Pathology has benefited from advanced innovation with novel technology to implement a digital solution. Whole slide imaging is a disruptive technology where glass slides are scanned to produce digital images. There have been significant advances in whole slide scanning hardware and software that have allowed for ready access of whole slide images. The digital images, or whole slide images, can be viewed comparable to glass slides in a microscope, as digital files. Whole slide imaging has increased in adoption among pathologists, pathology departments, and scientists for clinical, educational, and research initiatives. Worldwide usage of whole slide imaging has grown significantly. Pathology regulatory organizations (ie, College of American Pathologists) have put forth guidelines for clinical validation, and the US Food and Drug Administration have also approved whole slide imaging for primary diagnosis. This article will review the digital pathology ecosystem and discuss clinical and nonclinical applications of its use.
Collapse
|
19
|
Vicente‐Munuera P, Burgos‐Panadero R, Noguera I, Navarro S, Noguera R, Escudero LM. The topology of vitronectin: A complementary feature for neuroblastoma risk classification based on computer-aided detection. Int J Cancer 2020; 146:553-565. [PMID: 31173338 PMCID: PMC6899647 DOI: 10.1002/ijc.32495] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2019] [Revised: 04/30/2019] [Accepted: 05/27/2019] [Indexed: 12/13/2022]
Abstract
Tumors are complex networks of constantly interacting elements: tumor cells, stromal cells, immune and stem cells, blood/lympathic vessels, nerve fibers and extracellular matrix components. These elements can influence their microenvironment through mechanical and physical signals to promote tumor cell growth. To get a better understanding of tumor biology, cooperation between multidisciplinary fields is needed. Diverse mathematic computations and algorithms have been designed to find prognostic targets and enhance diagnostic assessment. In this work, we use computational digital tools to study the topology of vitronectin, a glycoprotein of the extracellular matrix. Vitronectin is linked to angiogenesis and migration, two processes closely related to tumor cell spread. Here, we investigate whether the distribution of this molecule in the tumor stroma may confer mechanical properties affecting neuroblastoma aggressiveness. Combining image analysis and graph theory, we analyze different topological features that capture the organizational cues of vitronectin in histopathological images taken from human samples. We find that the Euler number and the branching of territorial vitronectin, two topological features, could allow for a more precise pretreatment risk stratification to guide treatment strategies in neuroblastoma patients. A large amount of recently synthesized VN would create migration tracks, pinpointed by both topological features, for malignant neuroblasts, so that dramatic change in the extracellular matrix would increase tumor aggressiveness and worsen patient outcomes.
Collapse
Affiliation(s)
- Pablo Vicente‐Munuera
- Departamento de Biología CelularInstituto de Biomedicina de Sevilla (IBiS), Hospital Universitario Virgen del Roció/CSIC/Universidad de SevillaSevilleSpain
- Biomedical Network Research Centre on Neurodegenerative Diseases (CIBERNED)MadridSpain
| | - Rebeca Burgos‐Panadero
- Department of Pathology, Medical SchoolUniversity of Valencia/INCLIVAValenciaSpain
- Biomedical Network Research Centre on Oncology (CIBERONC)MadridSpain
| | - Inmaculada Noguera
- Central Support Service for Experimental Research (SCSIE), University of ValenciaValenciaSpain
| | - Samuel Navarro
- Department of Pathology, Medical SchoolUniversity of Valencia/INCLIVAValenciaSpain
- Biomedical Network Research Centre on Oncology (CIBERONC)MadridSpain
| | - Rosa Noguera
- Department of Pathology, Medical SchoolUniversity of Valencia/INCLIVAValenciaSpain
- Biomedical Network Research Centre on Oncology (CIBERONC)MadridSpain
| | - Luis M. Escudero
- Departamento de Biología CelularInstituto de Biomedicina de Sevilla (IBiS), Hospital Universitario Virgen del Roció/CSIC/Universidad de SevillaSevilleSpain
- Biomedical Network Research Centre on Neurodegenerative Diseases (CIBERNED)MadridSpain
| |
Collapse
|
20
|
Clunie DA. Dual-Personality DICOM-TIFF for Whole Slide Images: A Migration Technique for Legacy Software. J Pathol Inform 2019; 10:12. [PMID: 31057981 PMCID: PMC6489422 DOI: 10.4103/jpi.jpi_93_18] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2018] [Accepted: 03/06/2019] [Indexed: 02/06/2023] Open
Abstract
Despite recently organized Digital Imaging and Communications in Medicine (DICOM) testing and demonstration events involving numerous participating vendors, it is still the case that scanner manufacturers, software developers, and users continue to depend on proprietary file formats rather than adopting the standard DICOM whole slide microscopic image object. Many proprietary formats are Tagged Image File Format (TIFF) based, and existing applications and libraries can read tiled TIFF files. The sluggish adoption of DICOM for whole slide image encoding can be temporarily mitigated by the use of dual-personality DICOM-TIFF files. These are compatible with the installed base of TIFF-based software, as well as newer DICOM-based software. The DICOM file format was deliberately designed to support this dual-personality capability for such transitional situations, although it is rarely used. Furthermore, existing TIFF files can be converted into dual-personality DICOM-TIFF without changing the pixel data. This paper demonstrates the feasibility of extending the dual-personality concept to multiframe-tiled pyramidal whole slide images and explores the issues encountered. Open source code and sample converted images are provided for testing.
Collapse
|
21
|
Radiology and pathology: From old friends to strategic partners. RADIOLOGIA 2019. [DOI: 10.1016/j.rxeng.2019.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
22
|
Affiliation(s)
- Aurelio Ariza
- Hospital Universitari Germans Trias i Pujol y Universidad Autónoma de Barcelona, Barcelona, España; Presidente de la Comisión Nacional de Anatomía Patológica, Madrid, España; Secretario de la European Society of Pathology.
| |
Collapse
|
23
|
Dietz RL, Hartman DJ, Zheng L, Wiley C, Pantanowitz L. Review of the use of telepathology for intraoperative consultation. Expert Rev Med Devices 2018; 15:883-890. [PMID: 30451027 DOI: 10.1080/17434440.2018.1549987] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
INTRODUCTION The use of telepathology in intraoperative consultations has been increasing due to limited time and availability of pathologists, and the demand for increased access to pathology subspecialists in difficult cases. The five main categories of telepathology are (1) static, (2) dynamic, (3) robotic, (4) whole slide imaging (WSI), and (5) hybrid methods. The majority of these methods have been found to offer diagnostic accuracy rates similar to conventional microscopy, at the cost of slightly prolonged time to evaluate slides. AREAS COVERED Herein we discuss the salient features of each telepathology method and provide examples of their performance reported in the literature. EXPERT COMMENTARY Telepathology systems from any of the aforementioned categories can be employed to achieve timely and accurate diagnoses as long as they meet clinical needs and are validated for the intended use case. The decision to purchase a particular system depends on the clinical application, specific needs and budget of the laboratory, as well as the personal preference of the telepathologists involved. The adoption of telepathology practice is likely to expand in order to meet the increasing demand for subspecialist consultation and as technology advances to improve diagnostic accuracy and workflow.
Collapse
Affiliation(s)
- Robin L Dietz
- a Department of Pathology , UPMC, UPMC Cancer Pavillion Suite 201 , Pittsburgh , PA , USA
| | - Douglas J Hartman
- b Department of Pathology, Division of Pathology Informatics , UPMC Presbyterian Hospital , Pittsburgh , PA , USA
| | - Liu Zheng
- c Department of Pathology and Immunology , Washington University School of Medicine , St. Louis , MO , USA
| | - Clayton Wiley
- d Division of Neuropathology , UPMC Presbyterian Hospital , Pittsburgh , PA , USA
| | - Liron Pantanowitz
- a Department of Pathology , UPMC, UPMC Cancer Pavillion Suite 201 , Pittsburgh , PA , USA
| |
Collapse
|
24
|
Wong M, Frye J, Kim S, Marchevsky AM. The Use of Screencasts with Embedded Whole-Slide Scans and Hyperlinks to Teach Anatomic Pathology in a Supervised Digital Environment. J Pathol Inform 2018; 9:39. [PMID: 30607306 PMCID: PMC6289000 DOI: 10.4103/jpi.jpi_44_18] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2018] [Accepted: 09/27/2018] [Indexed: 01/10/2023] Open
Abstract
BACKGROUND There is an increasing interest in using digitized whole-slide imaging (WSI) for routine surgical pathology diagnoses. Screencasts are digital recordings of computer screen output with advanced interactive features that allow for the preparation of videos. Screencasts that include hyperlinks to WSIs could help teach pathology residents how to become familiar with technologies that they are likely to use in their future career. MATERIALS AND METHODS Twenty screencasts were prepared with Camtasia 2.0 software (TechSmith, Okemos, MI, USA). They included clinical history, videos of chest X-rays and/or chest computed tomography images, links to WSI digitized with an Aperio Turbo AT scanner (Leica Biosystems, Buffalo Grove, IL, USA), pre- and posttests, and faculty-narrated videos of the WSI in a manner closely resembling a slide seminar and other educational materials. Screencasts were saved in a hospital network, Screencast.com, YouTube.com, and Vimeo.com. The screencasts were viewed by 12 pathology residents and fellows who made diagnoses, answered the quizzes, and took a survey with questions designed to evaluate their perception of the quality of this technology. Quiz results were automatically e-mailed to faculty. Pre- and posttest results were compared using a paired t-test. RESULTS Screencasts can be viewed with Windows PC and Mac operating systems and mobile devices; only videos saved in our network and screencast.com could be used to generate quizzes. Participants' feedback was very favorable with average scores ranging from 4.5 to 4.8 (on a scale of 5). Mean posttest scores (87.0% [±21.6%]) were significantly improved over those in the pretest quizzes (48.5% [±31.2%]) (P < 0.0001). CONCLUSION Screencasts with WSI that allow residents and fellows to diagnose cases using digital microscopy may prove to be a useful technology to enhance the pathology education. Future studies with larger numbers of screencasts and participants are needed to optimize various teaching strategies.
Collapse
Affiliation(s)
- Mary Wong
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Joseph Frye
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Stacey Kim
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Alberto M. Marchevsky
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| |
Collapse
|
25
|
Herrmann MD, Clunie DA, Fedorov A, Doyle SW, Pieper S, Klepeis V, Le LP, Mutter GL, Milstone DS, Schultz TJ, Kikinis R, Kotecha GK, Hwang DH, Andriole KP, Iafrate AJ, Brink JA, Boland GW, Dreyer KJ, Michalski M, Golden JA, Louis DN, Lennerz JK. Implementing the DICOM Standard for Digital Pathology. J Pathol Inform 2018; 9:37. [PMID: 30533276 PMCID: PMC6236926 DOI: 10.4103/jpi.jpi_42_18] [Citation(s) in RCA: 63] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2018] [Accepted: 08/06/2018] [Indexed: 11/29/2022] Open
Abstract
Background: Digital Imaging and Communications in Medicine (DICOM®) is the standard for the representation, storage, and communication of medical images and related information. A DICOM file format and communication protocol for pathology have been defined; however, adoption by vendors and in the field is pending. Here, we implemented the essential aspects of the standard and assessed its capabilities and limitations in a multisite, multivendor healthcare network. Methods: We selected relevant DICOM attributes, developed a program that extracts pixel data and pixel-related metadata, integrated patient and specimen-related metadata, populated and encoded DICOM attributes, and stored DICOM files. We generated the files using image data from four vendor-specific image file formats and clinical metadata from two departments with different laboratory information systems. We validated the generated DICOM files using recognized DICOM validation tools and measured encoding, storage, and access efficiency for three image compression methods. Finally, we evaluated storing, querying, and retrieving data over the web using existing DICOM archive software. Results: Whole slide image data can be encoded together with relevant patient and specimen-related metadata as DICOM objects. These objects can be accessed efficiently from files or through RESTful web services using existing software implementations. Performance measurements show that the choice of image compression method has a major impact on data access efficiency. For lossy compression, JPEG achieves the fastest compression/decompression rates. For lossless compression, JPEG-LS significantly outperforms JPEG 2000 with respect to data encoding and decoding speed. Conclusion: Implementation of DICOM allows efficient access to image data as well as associated metadata. By leveraging a wealth of existing infrastructure solutions, the use of DICOM facilitates enterprise integration and data exchange for digital pathology.
Collapse
Affiliation(s)
| | | | - Andriy Fedorov
- Department of Radiology, Surgical Planning Laboratory, Brigham and Women's Hospital, Boston, MA, USA.,Harvard Medical School, Boston, MA, USA
| | - Sean W Doyle
- MGH and BWH Center for Clinical Data Science, Boston, MA, USA
| | | | - Veronica Klepeis
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - Long P Le
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - George L Mutter
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - David S Milstone
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Thomas J Schultz
- Enterprise Medical Imaging, Massachusetts General Hospital, Boston, MA, USA
| | - Ron Kikinis
- Department of Radiology, Surgical Planning Laboratory, Brigham and Women's Hospital, Boston, MA, USA.,Harvard Medical School, Boston, MA, USA
| | - Gopal K Kotecha
- MGH and BWH Center for Clinical Data Science, Boston, MA, USA
| | - David H Hwang
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Katherine P Andriole
- MGH and BWH Center for Clinical Data Science, Boston, MA, USA.,Harvard Medical School, Boston, MA, USA.,Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA
| | - A John Iafrate
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - James A Brink
- Harvard Medical School, Boston, MA, USA.,Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Giles W Boland
- Harvard Medical School, Boston, MA, USA.,Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA
| | - Keith J Dreyer
- MGH and BWH Center for Clinical Data Science, Boston, MA, USA.,Harvard Medical School, Boston, MA, USA.,Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Mark Michalski
- MGH and BWH Center for Clinical Data Science, Boston, MA, USA.,Harvard Medical School, Boston, MA, USA.,Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Jeffrey A Golden
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - David N Louis
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - Jochen K Lennerz
- Harvard Medical School, Boston, MA, USA.,Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|
26
|
Zarella MD, Bowman; D, Aeffner F, Farahani N, Xthona; A, Absar SF, Parwani A, Bui M, Hartman DJ. A Practical Guide to Whole Slide Imaging: A White Paper From the Digital Pathology Association. Arch Pathol Lab Med 2018; 143:222-234. [DOI: 10.5858/arpa.2018-0343-ra] [Citation(s) in RCA: 150] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Context.—
Whole slide imaging (WSI) represents a paradigm shift in pathology, serving as a necessary first step for a wide array of digital tools to enter the field. Its basic function is to digitize glass slides, but its impact on pathology workflows, reproducibility, dissemination of educational material, expansion of service to underprivileged areas, and intrainstitutional and interinstitutional collaboration exemplifies a significant innovative movement with far-reaching effects. Although the benefits of WSI to pathology practices, academic centers, and research institutions are many, the complexities of implementation remain an obstacle to widespread adoption. In the wake of the first regulatory clearance of WSI for primary diagnosis in the United States, some barriers to adoption have fallen. Nevertheless, implementation of WSI remains a difficult prospect for many institutions, especially those with stakeholders unfamiliar with the technologies necessary to implement a system or who cannot effectively communicate to executive leadership and sponsors the benefits of a technology that may lack clear and immediate reimbursement opportunity.
Objectives.—
To present an overview of WSI technology—present and future—and to demonstrate several immediate applications of WSI that support pathology practice, medical education, research, and collaboration.
Data Sources.—
Peer-reviewed literature was reviewed by pathologists, scientists, and technologists who have practical knowledge of and experience with WSI.
Conclusions.—
Implementation of WSI is a multifaceted and inherently multidisciplinary endeavor requiring contributions from pathologists, technologists, and executive leadership. Improved understanding of the current challenges to implementation, as well as the benefits and successes of the technology, can help prospective users identify the best path for success.
Collapse
Affiliation(s)
- Mark D. Zarella
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Douglas Bowman;
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Famke Aeffner
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Navid Farahani
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Albert Xthona;
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Syeda Fatima Absar
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Anil Parwani
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Marilyn Bui
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| | - Douglas J. Hartman
- From the Department of Pathology & Laboratory Medicine, Drexel University College of Medicine, Philadelphia, Pennsylvania (Drs Zarella and Absar); Pharma Services, Indica Labs, Inc, Corrales, New Mexico (Mr Bowman); Comparative Biology and Safety Sciences, Amgen, Inc, South San Francisco, California (Dr Aeffner); 3Scan, San Francisco, California (Dr Farahani); Barco, Inc, Beaverton, Oregon (Mr Xt
| |
Collapse
|
27
|
Hart SN. Will Digital Pathology be as Disruptive as Genomics? J Pathol Inform 2018; 9:27. [PMID: 30167342 PMCID: PMC6106127 DOI: 10.4103/jpi.jpi_25_18] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2018] [Accepted: 06/20/2018] [Indexed: 11/23/2022] Open
Abstract
Digital pathology is the science of performing traditional pathological assessment in a digital environment. A digital transition is long overdue since histochemical analysis such as hematoxylin and eosin staining has remained unchanged in over 100 years. Importantly, the digitization of whole slide images further lends itself to advances in computational pathology and artificial intelligence to transform qualitative assessment into quantitative assessment. The impact of this transition from a computational infrastructure perspective is reminiscent of a similar transition in the field of genomics. In this article, I describe some of the similarities between genomics and digital pathology as well as highlight some key lessons learned to prevent the same mistakes and delays that slowed the genomics revolution.
Collapse
Affiliation(s)
- Steven N Hart
- Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA
| |
Collapse
|