1
|
Masatti L, Marchetti M, Pirrotta S, Spagnol G, Corrà A, Ferrari J, Noventa M, Saccardi C, Calura E, Tozzi R. The unveiled mosaic of intra-tumor heterogeneity in ovarian cancer through spatial transcriptomic technologies: A systematic review. Transl Res 2024; 273:104-114. [PMID: 39111726 DOI: 10.1016/j.trsl.2024.08.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2024] [Revised: 07/16/2024] [Accepted: 08/01/2024] [Indexed: 08/12/2024]
Abstract
Epithelial ovarian cancer is a significant global health issue among women. Diagnosis and treatment pose challenges due to difficulties in predicting patient responses to therapy, primarily stemming from gaps in understanding tumor chemoresistance mechanisms. Recent advancements in transcriptomic technologies like single-cell RNA sequencing and spatial transcriptomics have greatly improved our understanding of ovarian cancer intratumor heterogeneity and tumor microenvironment composition. Spatial transcriptomics, in particular, comprises a plethora of technologies that enable the detection of hundreds of transcriptomes and their spatial distribution within a histological section, facilitating the study of cell types, states, and interactions within the tumor and its microenvironment. Studies investigating the spatial distribution of gene expression in ovarian cancer masses have identified specific features that impact prognosis and therapy outcomes. Emerging evidence suggests that specific spatial patterns of tumor cells and their immune and non-immune microenvironment significantly influence therapy response, as well as the behavior and progression of primary tumors and metastatic sites. The importance of spatially contextualizing ovarian cancer transcriptomes is underscored by these findings, which will advance our understanding and therapeutic approaches for this complex disease.
Collapse
Affiliation(s)
- Laura Masatti
- Department of Biology, University of Padova, Padova, Italy
| | - Matteo Marchetti
- Department of Gynecology and Obstetrics, Division of Women and Children, Padova University Hospital, Padova, Italy
| | | | - Giulia Spagnol
- Department of Gynecology and Obstetrics, Division of Women and Children, Padova University Hospital, Padova, Italy
| | - Anna Corrà
- Department of Biology, University of Padova, Padova, Italy; Fondazione Istituto di Ricerca Pediatrica Città della Speranza, Padova, Italy
| | - Jacopo Ferrari
- Department of Gynecology and Obstetrics, Division of Women and Children, Padova University Hospital, Padova, Italy
| | - Marco Noventa
- Department of Gynecology and Obstetrics, Division of Women and Children, Padova University Hospital, Padova, Italy
| | - Carlo Saccardi
- Department of Gynecology and Obstetrics, Division of Women and Children, Padova University Hospital, Padova, Italy
| | - Enrica Calura
- Department of Biology, University of Padova, Padova, Italy.
| | - Roberto Tozzi
- Department of Gynecology and Obstetrics, Division of Women and Children, Padova University Hospital, Padova, Italy
| |
Collapse
|
2
|
Pathak MP, Pathak K, Saikia R, Gogoi U, Ahmad MZ, Patowary P, Das A. Immunomodulatory effect of mushrooms and their bioactive compounds in cancer: A comprehensive review. Biomed Pharmacother 2022; 149:112901. [DOI: 10.1016/j.biopha.2022.112901] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Revised: 03/25/2022] [Accepted: 03/26/2022] [Indexed: 11/28/2022] Open
|
3
|
Villa Nova M, Lin TP, Shanehsazzadeh S, Jain K, Ng SCY, Wacker R, Chichakly K, Wacker MG. Nanomedicine Ex Machina: Between Model-Informed Development and Artificial Intelligence. Front Digit Health 2022; 4:799341. [PMID: 35252958 PMCID: PMC8894322 DOI: 10.3389/fdgth.2022.799341] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Accepted: 01/26/2022] [Indexed: 12/12/2022] Open
Abstract
Today, a growing number of computational aids and simulations are shaping model-informed drug development. Artificial intelligence, a family of self-learning algorithms, is only the latest emerging trend applied by academic researchers and the pharmaceutical industry. Nanomedicine successfully conquered several niche markets and offers a wide variety of innovative drug delivery strategies. Still, only a small number of patients benefit from these advanced treatments, and the number of data sources is very limited. As a consequence, “big data” approaches are not always feasible and smart combinations of human and artificial intelligence define the research landscape. These methodologies will potentially transform the future of nanomedicine and define new challenges and limitations of machine learning in their development. In our review, we present an overview of modeling and artificial intelligence applications in the development and manufacture of nanomedicines. Also, we elucidate the role of each method as a facilitator of breakthroughs and highlight important limitations.
Collapse
Affiliation(s)
- Mônica Villa Nova
- Department of Pharmacy, State University of Maringá, Maringá, Brazil
| | - Tzu Ping Lin
- Wacker Research Lab, Department of Pharmacy, Faculty of Science, National University of Singapore, Singapore, Singapore
| | - Saeed Shanehsazzadeh
- Biological Resources Imaging Laboratory, Mark Wainwright Analytical Centre, UNSW Sydney, Sydney, NSW, Australia
| | - Kinjal Jain
- Wacker Research Lab, Department of Pharmacy, Faculty of Science, National University of Singapore, Singapore, Singapore
| | - Samuel Cheng Yong Ng
- Wacker Research Lab, Department of Pharmacy, Faculty of Science, National University of Singapore, Singapore, Singapore
| | | | | | - Matthias G. Wacker
- Wacker Research Lab, Department of Pharmacy, Faculty of Science, National University of Singapore, Singapore, Singapore
- *Correspondence: Matthias G. Wacker
| |
Collapse
|
4
|
Foran DJ, Durbin EB, Chen W, Sadimin E, Sharma A, Banerjee I, Kurc T, Li N, Stroup AM, Harris G, Gu A, Schymura M, Gupta R, Bremer E, Balsamo J, DiPrima T, Wang F, Abousamra S, Samaras D, Hands I, Ward K, Saltz JH. An Expandable Informatics Framework for Enhancing Central Cancer Registries with Digital Pathology Specimens, Computational Imaging Tools, and Advanced Mining Capabilities. J Pathol Inform 2022; 13:5. [PMID: 35136672 PMCID: PMC8794027 DOI: 10.4103/jpi.jpi_31_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 04/30/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Population-based state cancer registries are an authoritative source for cancer statistics in the United States. They routinely collect a variety of data, including patient demographics, primary tumor site, stage at diagnosis, first course of treatment, and survival, on every cancer case that is reported across all U.S. states and territories. The goal of our project is to enrich NCI's Surveillance, Epidemiology, and End Results (SEER) registry data with high-quality population-based biospecimen data in the form of digital pathology, machine-learning-based classifications, and quantitative histopathology imaging feature sets (referred to here as Pathomics features). MATERIALS AND METHODS As part of the project, the underlying informatics infrastructure was designed, tested, and implemented through close collaboration with several participating SEER registries to ensure consistency with registry processes, computational scalability, and ability to support creation of population cohorts that span multiple sites. Utilizing computational imaging algorithms and methods to both generate indices and search for matches makes it possible to reduce inter- and intra-observer inconsistencies and to improve the objectivity with which large image repositories are interrogated. RESULTS Our team has created and continues to expand a well-curated repository of high-quality digitized pathology images corresponding to subjects whose data are routinely collected by the collaborating registries. Our team has systematically deployed and tested key, visual analytic methods to facilitate automated creation of population cohorts for epidemiological studies and tools to support visualization of feature clusters and evaluation of whole-slide images. As part of these efforts, we are developing and optimizing advanced search and matching algorithms to facilitate automated, content-based retrieval of digitized specimens based on their underlying image features and staining characteristics. CONCLUSION To meet the challenges of this project, we established the analytic pipelines, methods, and workflows to support the expansion and management of a growing repository of high-quality digitized pathology and information-rich, population cohorts containing objective imaging and clinical attributes to facilitate studies that seek to discriminate among different subtypes of disease, stratify patient populations, and perform comparisons of tumor characteristics within and across patient cohorts. We have also successfully developed a suite of tools based on a deep-learning method to perform quantitative characterizations of tumor regions, assess infiltrating lymphocyte distributions, and generate objective nuclear feature measurements. As part of these efforts, our team has implemented reliable methods that enable investigators to systematically search through large repositories to automatically retrieve digitized pathology specimens and correlated clinical data based on their computational signatures.
Collapse
Affiliation(s)
- David J. Foran
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
- Department of Pathology and Laboratory Medicine, Rutgers-Robert Wood Johnson Medical School, Piscataway, NJ, USA
| | - Eric B. Durbin
- Kentucky Cancer Registry, Markey Cancer Center, University of Kentucky, Lexington, KY, USA
- Division of Biomedical Informatics, Department of Internal Medicine, College of Medicine, Lexington, KY, USA
| | - Wenjin Chen
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Evita Sadimin
- Center for Biomedical Informatics, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
- Department of Pathology and Laboratory Medicine, Rutgers-Robert Wood Johnson Medical School, Piscataway, NJ, USA
| | - Ashish Sharma
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Imon Banerjee
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Nan Li
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Antoinette M. Stroup
- New Jersey State Cancer Registry, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Gerald Harris
- New Jersey State Cancer Registry, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Annie Gu
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
| | - Maria Schymura
- New York State Cancer Registry, New York State Department of Health, Albany, NY, USA
| | - Rajarsi Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Erich Bremer
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Joseph Balsamo
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Tammy DiPrima
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Feiqiao Wang
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Shahira Abousamra
- Department of Computer Science, Stony Brook University, Stony Brook, NY, USA
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY, USA
| | - Isaac Hands
- Division of Biomedical Informatics, Department of Internal Medicine, College of Medicine, Lexington, KY, USA
| | - Kevin Ward
- Georgia State Cancer Registry, Georgia Department of Public Health, Atlanta, GA, USA
| | - Joel H. Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| |
Collapse
|
5
|
Sharma A, Tarbox L, Kurc T, Bona J, Smith K, Kathiravelu P, Bremer E, Saltz JH, Prior F. PRISM: A Platform for Imaging in Precision Medicine. JCO Clin Cancer Inform 2021; 4:491-499. [PMID: 32479186 PMCID: PMC7328100 DOI: 10.1200/cci.20.00001] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
PURPOSE Precision medicine requires an understanding of individual variability, which can only be acquired from large data collections such as those supported by the Cancer Imaging Archive (TCIA). We have undertaken a program to extend the types of data TCIA can support. This, in turn, will enable TCIA to play a key role in precision medicine research by collecting and disseminating high-quality, state-of-the-art, quantitative imaging data that meet the evolving needs of the cancer research community. METHODS A modular technology platform is presented that would allow existing data resources, such as TCIA, to evolve into a comprehensive data resource that meets the needs of users engaged in translational research for imaging-based precision medicine. This Platform for Imaging in Precision Medicine (PRISM) helps streamline the deployment and improve TCIA's efficiency and sustainability. More importantly, its inherent modular architecture facilitates a piecemeal adoption by other data repositories. RESULTS PRISM includes services for managing radiology and pathology images and features and associated clinical data. A semantic layer is being built to help users explore diverse collections and pool data sets to create specialized cohorts. PRISM includes tools for image curation and de-identification. It includes image visualization and feature exploration tools. The entire platform is distributed as a series of containerized microservices with representational state transfer interfaces. CONCLUSION PRISM is helping modernize, scale, and sustain the technology stack that powers TCIA. Repositories can take advantage of individual PRISM services such as de-identification and quality control. PRISM is helping scale image informatics for cancer research at a time when the size, complexity, and demands to integrate image data with other precision medicine data-intensive commons are mounting.
Collapse
Affiliation(s)
| | - Lawrence Tarbox
- University of Arkansas for Medical Sciences, Little Rock, AR
| | | | - Jonathan Bona
- University of Arkansas for Medical Sciences, Little Rock, AR
| | - Kirk Smith
- University of Arkansas for Medical Sciences, Little Rock, AR
| | | | | | | | - Fred Prior
- University of Arkansas for Medical Sciences, Little Rock, AR
| |
Collapse
|
6
|
Zhong A, Li X, Wu D, Ren H, Kim K, Kim Y, Buch V, Neumark N, Bizzo B, Tak WY, Park SY, Lee YR, Kang MK, Park JG, Kim BS, Chung WJ, Guo N, Dayan I, Kalra MK, Li Q. Deep metric learning-based image retrieval system for chest radiograph and its clinical applications in COVID-19. Med Image Anal 2021; 70:101993. [PMID: 33711739 PMCID: PMC8032481 DOI: 10.1016/j.media.2021.101993] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2020] [Revised: 01/19/2021] [Accepted: 02/01/2021] [Indexed: 12/13/2022]
Abstract
In recent years, deep learning-based image analysis methods have been widely applied in computer-aided detection, diagnosis and prognosis, and has shown its value during the public health crisis of the novel coronavirus disease 2019 (COVID-19) pandemic. Chest radiograph (CXR) has been playing a crucial role in COVID-19 patient triaging, diagnosing and monitoring, particularly in the United States. Considering the mixed and unspecific signals in CXR, an image retrieval model of CXR that provides both similar images and associated clinical information can be more clinically meaningful than a direct image diagnostic model. In this work we develop a novel CXR image retrieval model based on deep metric learning. Unlike traditional diagnostic models which aim at learning the direct mapping from images to labels, the proposed model aims at learning the optimized embedding space of images, where images with the same labels and similar contents are pulled together. The proposed model utilizes multi-similarity loss with hard-mining sampling strategy and attention mechanism to learn the optimized embedding space, and provides similar images, the visualizations of disease-related attention maps and useful clinical information to assist clinical decisions. The model is trained and validated on an international multi-site COVID-19 dataset collected from 3 different sources. Experimental results of COVID-19 image retrieval and diagnosis tasks show that the proposed model can serve as a robust solution for CXR analysis and patient management for COVID-19. The model is also tested on its transferability on a different clinical decision support task for COVID-19, where the pre-trained model is applied to extract image features from a new dataset without any further training. The extracted features are then combined with COVID-19 patient's vitals, lab tests and medical histories to predict the possibility of airway intubation in 72 hours, which is strongly associated with patient prognosis, and is crucial for patient care and hospital resource planning. These results demonstrate our deep metric learning based image retrieval model is highly efficient in the CXR retrieval, diagnosis and prognosis, and thus has great clinical value for the treatment and management of COVID-19 patients.
Collapse
Affiliation(s)
- Aoxiao Zhong
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States; School of Engineering and Applied Sciences, Harvard University, Boston, MA, United States
| | - Xiang Li
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Dufan Wu
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Hui Ren
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Kyungsang Kim
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Younggon Kim
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Varun Buch
- MGH & BWH Center for Clinical Data Science, Boston, MA, United States
| | - Nir Neumark
- MGH & BWH Center for Clinical Data Science, Boston, MA, United States
| | - Bernardo Bizzo
- MGH & BWH Center for Clinical Data Science, Boston, MA, United States
| | - Won Young Tak
- Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea
| | - Soo Young Park
- Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea
| | - Yu Rim Lee
- Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea
| | - Min Kyu Kang
- Department of Internal Medicine, Yeungnam University College of Medicine, Daegu, South Korea
| | - Jung Gil Park
- Department of Internal Medicine, Yeungnam University College of Medicine, Daegu, South Korea
| | - Byung Seok Kim
- Department of Internal Medicine, Catholic University of Daegu School of Medicine, Daegu, South Korea
| | - Woo Jin Chung
- Department of Internal Medicine, Keimyung University School of Medicine, Daegu, South Korea
| | - Ning Guo
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Ittai Dayan
- School of Engineering and Applied Sciences, Harvard University, Boston, MA, United States
| | - Mannudeep K Kalra
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States
| | - Quanzheng Li
- Department of Radiology, Massachusetts General Hospital, Boston, MA, United States; MGH & BWH Center for Clinical Data Science, Boston, MA, United States.
| |
Collapse
|
7
|
Xing F, Zhang X, Cornish TC. Artificial intelligence for pathology. Artif Intell Med 2021. [DOI: 10.1016/b978-0-12-821259-2.00011-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
8
|
Characterizing Immune Responses in Whole Slide Images of Cancer With Digital Pathology and Pathomics. CURRENT PATHOBIOLOGY REPORTS 2020. [DOI: 10.1007/s40139-020-00217-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Abstract
Purpose of Review
Our goal is to show how readily available Pathomics tissue analytics can be used to study tumor immune interactions in cancer. We provide a brief overview of how Pathomics complements traditional histopathologic examination of cancer tissue samples. We highlight a novel Pathomics application, Tumor-TILs, that quantitatively measures and generates maps of tumor infiltrating lymphocytes in breast, pancreatic, and lung cancer by leveraging deep learning computer vision applications to perform automated analyses of whole slide images.
Recent Findings
Tumor-TIL maps have been generated to analyze WSIs from thousands of cases of breast, pancreatic, and lung cancer. We report the availability of these tools in an effort to promote collaborative research and motivate future development of ensemble Pathomics applications to discover novel biomarkers and perform a wide range of correlative clinicopathologic research in cancer immunopathology and beyond.
Summary
Tumor immune interactions in cancer are a fascinating aspect of cancer pathobiology with particular significance due to the emergence of immunotherapy. We present simple yet powerful specialized Pathomics methods that serve as powerful clinical research tools and potential standalone clinical screening tests to predict clinical outcomes and treatment responses for precision medicine applications in immunotherapy.
Collapse
|
9
|
Panayides AS, Amini A, Filipovic ND, Sharma A, Tsaftaris SA, Young A, Foran D, Do N, Golemati S, Kurc T, Huang K, Nikita KS, Veasey BP, Zervakis M, Saltz JH, Pattichis CS. AI in Medical Imaging Informatics: Current Challenges and Future Directions. IEEE J Biomed Health Inform 2020; 24:1837-1857. [PMID: 32609615 PMCID: PMC8580417 DOI: 10.1109/jbhi.2020.2991043] [Citation(s) in RCA: 105] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.
Collapse
|
10
|
Kurc T, Bakas S, Ren X, Bagari A, Momeni A, Huang Y, Zhang L, Kumar A, Thibault M, Qi Q, Wang Q, Kori A, Gevaert O, Zhang Y, Shen D, Khened M, Ding X, Krishnamurthi G, Kalpathy-Cramer J, Davis J, Zhao T, Gupta R, Saltz J, Farahani K. Segmentation and Classification in Digital Pathology for Glioma Research: Challenges and Deep Learning Approaches. Front Neurosci 2020; 14:27. [PMID: 32153349 PMCID: PMC7046596 DOI: 10.3389/fnins.2020.00027] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2019] [Accepted: 01/10/2020] [Indexed: 12/12/2022] Open
Abstract
Biomedical imaging Is an important source of information in cancer research. Characterizations of cancer morphology at onset, progression, and in response to treatment provide complementary information to that gleaned from genomics and clinical data. Accurate extraction and classification of both visual and latent image features Is an increasingly complex challenge due to the increased complexity and resolution of biomedical image data. In this paper, we present four deep learning-based image analysis methods from the Computational Precision Medicine (CPM) satellite event of the 21st International Medical Image Computing and Computer Assisted Intervention (MICCAI 2018) conference. One method Is a segmentation method designed to segment nuclei in whole slide tissue images (WSIs) of adult diffuse glioma cases. It achieved a Dice similarity coefficient of 0.868 with the CPM challenge datasets. Three methods are classification methods developed to categorize adult diffuse glioma cases into oligodendroglioma and astrocytoma classes using radiographic and histologic image data. These methods achieved accuracy values of 0.75, 0.80, and 0.90, measured as the ratio of the number of correct classifications to the number of total cases, with the challenge datasets. The evaluations of the four methods indicate that (1) carefully constructed deep learning algorithms are able to produce high accuracy in the analysis of biomedical image data and (2) the combination of radiographic with histologic image information improves classification performance.
Collapse
Affiliation(s)
- Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
| | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics, University of Pennsylvania, Philadelphia, PA, United States
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Xuhua Ren
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Aditya Bagari
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Alexandre Momeni
- Department of Medicine and Biomedical Data Science, Stanford University, Stanford, CA, United States
| | - Yue Huang
- School of Informatics, Xiamen University, Xiamen, China
| | - Lichi Zhang
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Ashish Kumar
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Marc Thibault
- Department of Medicine and Biomedical Data Science, Stanford University, Stanford, CA, United States
| | - Qi Qi
- School of Informatics, Xiamen University, Xiamen, China
| | - Qian Wang
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Avinash Kori
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Olivier Gevaert
- Department of Medicine and Biomedical Data Science, Stanford University, Stanford, CA, United States
| | - Yunlong Zhang
- School of Informatics, Xiamen University, Xiamen, China
| | - Dinggang Shen
- Department of Radiology and BRIC, The University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Mahendra Khened
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Xinghao Ding
- School of Informatics, Xiamen University, Xiamen, China
| | | | - Jayashree Kalpathy-Cramer
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - James Davis
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Tianhao Zhao
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Rajarsi Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
| | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, United States
| |
Collapse
|
11
|
Pucci C, Martinelli C, Ciofani G. Innovative approaches for cancer treatment: current perspectives and new challenges. Ecancermedicalscience 2019; 13:961. [PMID: 31537986 PMCID: PMC6753017 DOI: 10.3332/ecancer.2019.961] [Citation(s) in RCA: 342] [Impact Index Per Article: 68.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022] Open
Abstract
Every year, cancer is responsible for millions of deaths worldwide and, even though much progress has been achieved in medicine, there are still many issues that must be addressed in order to improve cancer therapy. For this reason, oncological research is putting a lot of effort towards finding new and efficient therapies which can alleviate critical side effects caused by conventional treatments. Different technologies are currently under evaluation in clinical trials or have been already introduced into clinical practice. While nanomedicine is contributing to the development of biocompatible materials both for diagnostic and therapeutic purposes, bioengineering of extracellular vesicles and cells derived from patients has allowed designing ad hoc systems and univocal targeting strategies. In this review, we will provide an in-depth analysis of the most innovative advances in basic and applied cancer research.
Collapse
Affiliation(s)
- Carlotta Pucci
- Smart Bio-Interfaces, Istituto Italiano di Tecnologia, 56025 Pisa, Italy
| | - Chiara Martinelli
- Smart Bio-Interfaces, Istituto Italiano di Tecnologia, 56025 Pisa, Italy
| | - Gianni Ciofani
- Smart Bio-Interfaces, Istituto Italiano di Tecnologia, 56025 Pisa, Italy.,Department of Mechanical and Aerospace Engineering, Politecnico di Torino, 10129 Torino, Italy
| |
Collapse
|
12
|
Gupta R, Kurc T, Sharma A, Almeida JS, Saltz J. The Emergence of Pathomics. CURRENT PATHOBIOLOGY REPORTS 2019. [DOI: 10.1007/s40139-019-00200-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
13
|
Vo H, Kong J, Teng D, Liang Y, Aji A, Teodoro G, Wang F. MaReIA: A Cloud MapReduce Based High Performance Whole Slide Image Analysis Framework. DISTRIBUTED AND PARALLEL DATABASES 2019; 37:251-272. [PMID: 31217669 PMCID: PMC6583906 DOI: 10.1007/s10619-018-7237-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Recent advancements in systematic analysis of high resolution whole slide images have increase efficiency of diagnosis, prognosis and prediction of cancer and important diseases. Due to the enormous sizes and dimensions of whole slide images, the analysis requires extensive computing resources which are not commonly available. Images have to be tiled for processing due to computer memory limitations, which lead to inaccurate results due to the ignorance of boundary crossing objects. Thus, we propose a generic and highly scalable cloud-based image analysis framework for whole slide images. The framework enables parallelized integration of image analysis steps, such as segmentation and aggregation of micro-structures in a single pipeline, and generation of final objects manageable by databases. The core concept relies on the abstraction of objects in whole slide images as different classes of spatial geometries, which in turn can be handled as text based records in MapReduce. The framework applies an overlapping partitioning scheme on images, and provides parallelization of tiling and image segmentation based on MapReduce architecture. It further provides robust object normalization, graceful handling of boundary objects with an efficient spatial indexing based matching method to generate accurate results. Our experiments on Amazon EMR show that MaReIA is highly scalable, generic and extremely cost effective by benchmark tests.
Collapse
Affiliation(s)
- Hoang Vo
- Department of Computer Science, Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY
| | - Jun Kong
- Department of Biomedical Informatics, Emory University, Atlanta, GA
| | - Dejun Teng
- Department of Computer Science and Engineering, Ohio State University, Columbus, OH
| | - Yanhui Liang
- Department of Computer Science, Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY
| | | | - George Teodoro
- Department of Computer Science, University of Brasília, Brasília, DF, Brazil
| | - Fusheng Wang
- Department of Computer Science, Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY
| |
Collapse
|
14
|
Sapkota M, Shi X, Xing F, Yang L. Deep Convolutional Hashing for Low-Dimensional Binary Embedding of Histopathological Images. IEEE J Biomed Health Inform 2019; 23:805-816. [PMID: 29993648 PMCID: PMC6429565 DOI: 10.1109/jbhi.2018.2827703] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Compact binary representations of histopa-thology images using hashing methods provide efficient approximate nearest neighbor search for direct visual query in large-scale databases. They can be utilized to measure the probability of the abnormality of the query image based on the retrieved similar cases, thereby providing support for medical diagnosis. They also allow for efficient managing of large-scale image databases because of a low storage requirement. However, the effectiveness of binary representations heavily relies on the visual descriptors that represent the semantic information in the histopathological images. Traditional approaches with hand-crafted visual descriptors might fail due to significant variations in image appearance. Recently, deep learning architectures provide promising solutions to address this problem using effective semantic representations. In this paper, we propose a deep convolutional hashing method that can be trained "point-wise" to simultaneously learn both semantic and binary representations of histopathological images. Specifically, we propose a convolutional neural network that introduces a latent binary encoding (LBE) layer for low-dimensional feature embedding to learn binary codes. We design a joint optimization objective function that encourages the network to learn discriminative representations from the label information, and reduce the gap between the real-valued low-dimensional embedded features and desired binary values. The binary encoding for new images can be obtained by forward propagating through the network and quantizing the output of the LBE layer. Experimental results on a large-scale histopathological image dataset demonstrate the effectiveness of the proposed method.
Collapse
|
15
|
Saltz J, Sharma A, Iyer G, Bremer E, Wang F, Jasniewski A, DiPrima T, Almeida JS, Gao Y, Zhao T, Saltz M, Kurc T. A Containerized Software System for Generation, Management, and Exploration of Features from Whole Slide Tissue Images. Cancer Res 2017; 77:e79-e82. [PMID: 29092946 DOI: 10.1158/0008-5472.can-17-0316] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2017] [Revised: 06/17/2017] [Accepted: 09/01/2017] [Indexed: 11/16/2022]
Abstract
Well-curated sets of pathology image features will be critical to clinical studies that aim to evaluate and predict treatment responses. Researchers require information synthesized across multiple biological scales, from the patient to the molecular scale, to more effectively study cancer. This article describes a suite of services and web applications that allow users to select regions of interest in whole slide tissue images, run a segmentation pipeline on the selected regions to extract nuclei and compute shape, size, intensity, and texture features, store and index images and analysis results, and visualize and explore images and computed features. All the services are deployed as containers and the user-facing interfaces as web-based applications. The set of containers and web applications presented in this article is used in cancer research studies of morphologic characteristics of tumor tissues. The software is free and open source. Cancer Res; 77(21); e79-82. ©2017 AACR.
Collapse
Affiliation(s)
- Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York.
| | - Ashish Sharma
- Department of Biomedical Informatics, Emory University, Atlanta, Georgia
| | - Ganesh Iyer
- Department of Biomedical Informatics, Emory University, Atlanta, Georgia
| | - Erich Bremer
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | - Feiqiao Wang
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | - Alina Jasniewski
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | - Tammy DiPrima
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | - Jonas S Almeida
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | - Yi Gao
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | - Tianhao Zhao
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York.,Department of Pathology, Stony Brook University, Stony Brook, New York
| | - Mary Saltz
- Department of Radiology, Stony Brook University, Stony Brook, New York
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York.,Scientific Data Group, Oak Ridge National Laboratory, Oak Ridge, Tennessee
| |
Collapse
|
16
|
Li Z, Zhang X, Müller H, Zhang S. Large-scale retrieval for medical image analytics: A comprehensive review. Med Image Anal 2017; 43:66-84. [PMID: 29031831 DOI: 10.1016/j.media.2017.09.007] [Citation(s) in RCA: 75] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2017] [Revised: 08/01/2017] [Accepted: 09/29/2017] [Indexed: 12/27/2022]
Abstract
Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.
Collapse
Affiliation(s)
- Zhongyu Li
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA
| | - Xiaofan Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA
| | - Henning Müller
- Information Systems Institute, HES-SO Valais, Sierre, Switzerland
| | - Shaoting Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA.
| |
Collapse
|
17
|
Chennubhotla C, Clarke LP, Fedorov A, Foran D, Harris G, Helton E, Nordstrom R, Prior F, Rubin D, Saltz JH, Shalley E, Sharma A. An Assessment of Imaging Informatics for Precision Medicine in Cancer. Yearb Med Inform 2017; 26:110-119. [PMID: 29063549 DOI: 10.15265/iy-2017-041] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022] Open
Abstract
Objectives: Precision medicine requires the measurement, quantification, and cataloging of medical characteristics to identify the most effective medical intervention. However, the amount of available data exceeds our current capacity to extract meaningful information. We examine the informatics needs to achieve precision medicine from the perspective of quantitative imaging and oncology. Methods: The National Cancer Institute (NCI) organized several workshops on the topic of medical imaging and precision medicine. The observations and recommendations are summarized herein. Results: Recommendations include: use of standards in data collection and clinical correlates to promote interoperability; data sharing and validation of imaging tools; clinician's feedback in all phases of research and development; use of open-source architecture to encourage reproducibility and reusability; use of challenges which simulate real-world situations to incentivize innovation; partnership with industry to facilitate commercialization; and education in academic communities regarding the challenges involved with translation of technology from the research domain to clinical utility and the benefits of doing so. Conclusions: This article provides a survey of the role and priorities for imaging informatics to help advance quantitative imaging in the era of precision medicine. While these recommendations were drawn from oncology, they are relevant and applicable to other clinical domains where imaging aids precision medicine.
Collapse
|
18
|
Wen S, Kurc TM, Gao Y, Zhao T, Saltz JH, Zhu W. A Methodology for Texture Feature-based Quality Assessment in Nucleus Segmentation of Histopathology Image. J Pathol Inform 2017; 8:38. [PMID: 28966837 PMCID: PMC5609357 DOI: 10.4103/jpi.jpi_43_17] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2017] [Accepted: 07/11/2017] [Indexed: 12/31/2022] Open
Abstract
CONTEXT Image segmentation pipelines often are sensitive to algorithm input parameters. Algorithm parameters optimized for a set of images do not necessarily produce good-quality-segmentation results for other images. Even within an image, some regions may not be well segmented due to a number of factors, including multiple pieces of tissue with distinct characteristics, differences in staining of the tissue, normal versus tumor regions, and tumor heterogeneity. Evaluation of quality of segmentation results is an important step in image analysis. It is very labor intensive to do quality assessment manually with large image datasets because a whole-slide tissue image may have hundreds of thousands of nuclei. Semi-automatic mechanisms are needed to assist researchers and application developers to detect image regions with bad segmentations efficiently. AIMS Our goal is to develop and evaluate a machine-learning-based semi-automated workflow to assess quality of nucleus segmentation results in a large set of whole-slide tissue images. METHODS We propose a quality control methodology, in which machine-learning algorithms are trained with image intensity and texture features to produce a classification model. This model is applied to image patches in a whole-slide tissue image to predict the quality of nucleus segmentation in each patch. The training step of our methodology involves the selection and labeling of regions by a pathologist in a set of images to create the training dataset. The image regions are partitioned into patches. A set of intensity and texture features is computed for each patch. A classifier is trained with the features and the labels assigned by the pathologist. At the end of this process, a classification model is generated. The classification step applies the classification model to unlabeled test images. Each test image is partitioned into patches. The classification model is applied to each patch to predict the patch's label. RESULTS The proposed methodology has been evaluated by assessing the segmentation quality of a segmentation method applied to images from two cancer types in The Cancer Genome Atlas; WHO Grade II lower grade glioma (LGG) and lung adenocarcinoma (LUAD). The results show that our method performs well in predicting patches with good-quality segmentations and achieves F1 scores 84.7% for LGG and 75.43% for LUAD. CONCLUSIONS As image scanning technologies advance, large volumes of whole-slide tissue images will be available for research and clinical use. Efficient approaches for the assessment of quality and robustness of output from computerized image analysis workflows will become increasingly critical to extracting useful quantitative information from tissue images. Our work demonstrates the feasibility of machine-learning-based semi-automated techniques to assist researchers and algorithm developers in this process.
Collapse
Affiliation(s)
- Si Wen
- Department of Applied Mathematics and Statistics, State University of New York at Stony Brook, Stony Brook, NY, USA
| | - Tahsin M. Kurc
- Department of Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, NY, USA
| | - Yi Gao
- Department of Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, NY, USA
| | - Tianhao Zhao
- Department of Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, NY, USA
| | - Joel H. Saltz
- Department of Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, NY, USA
| | - Wei Zhu
- Department of Applied Mathematics and Statistics, State University of New York at Stony Brook, Stony Brook, NY, USA
| |
Collapse
|
19
|
Xu Y, Li Y, Shen Z, Wu Z, Gao T, Fan Y, Lai M, Chang EIC. Parallel multiple instance learning for extremely large histopathology image analysis. BMC Bioinformatics 2017; 18:360. [PMID: 28774262 PMCID: PMC5543478 DOI: 10.1186/s12859-017-1768-8] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2017] [Accepted: 07/19/2017] [Indexed: 11/21/2022] Open
Abstract
Background Histopathology images are critical for medical diagnosis, e.g., cancer and its treatment. A standard histopathology slice can be easily scanned at a high resolution of, say, 200,000×200,000 pixels. These high resolution images can make most existing imaging processing tools infeasible or less effective when operated on a single machine with limited memory, disk space and computing power. Results In this paper, we propose an algorithm tackling this new emerging “big data” problem utilizing parallel computing on High-Performance-Computing (HPC) clusters. Experimental results on a large-scale data set (1318 images at a scale of 10 billion pixels each) demonstrate the efficiency and effectiveness of the proposed algorithm for low-latency real-time applications. Conclusions The framework proposed an effective and efficient system for extremely large histopathology image analysis. It is based on the multiple instance learning formulation for weakly-supervised learning for image classification, segmentation and clustering. When a max-margin concept is adopted for different clusters, we obtain further improvement in clustering performance.
Collapse
Affiliation(s)
- Yan Xu
- State Key Laboratory of Software Development Environment and Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education and Research Institute of Beihang University in Shenzhen, Beijing, China.,Microsoft Research Asia, Beijing, China
| | - Yeshu Li
- School of Computer Science and Engineering, Beihang University, Beijing, China
| | - Zhengyang Shen
- State Key Laboratory of Software Development Environment and Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education and Research Institute of Beihang University in Shenzhen, Beijing, China
| | - Ziwei Wu
- College of Information and Electrical Engineering, China Agricultural University, Beijing, China
| | - Teng Gao
- Microsoft Research Asia, Beijing, China
| | - Yubo Fan
- State Key Laboratory of Software Development Environment and Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education and Research Institute of Beihang University in Shenzhen, Beijing, China
| | - Maode Lai
- Department of Pathology, School of Medicine, Zhejiang University, Zhejiang, China
| | | |
Collapse
|
20
|
Saltz J, Almeida J, Gao Y, Sharma A, Bremer E, DiPrima T, Saltz M, Kalpathy-Cramer J, Kurc T. Towards Generation, Management, and Exploration of Combined Radiomics and Pathomics Datasets for Cancer Research. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2017; 2017:85-94. [PMID: 28815113 PMCID: PMC5543366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Cancer is a complex multifactorial disease state and the ability to anticipate and steer treatment results will require information synthesis across multiple scales from the host to the molecular level. Radiomics and Pathomics, where image features are extracted from routine diagnostic Radiology and Pathology studies, are also evolving as valuable diagnostic and prognostic indicators in cancer. This information explosion provides new opportunities for integrated, multi-scale investigation of cancer, but also mandates a need to build systematic and integrated approaches to manage, query and mine combined Radiomics and Pathomics data. In this paper, we describe a suite of tools and web-based applications towards building a comprehensive framework to support the generation, management and interrogation of large volumes of Radiomics and Pathomics feature sets and the investigation of correlations between image features, molecular data, and clinical outcome.
Collapse
Affiliation(s)
- Joel Saltz
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY
| | - Jonas Almeida
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY
| | - Yi Gao
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY
| | - Ashish Sharma
- Biomedical Informatics Department, Emory University, Atlanta, GA
| | - Erich Bremer
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY
| | - Tammy DiPrima
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY
| | - Mary Saltz
- Department of Radiology, Stony Brook University, Stony Brook, NY
| | | | - Tahsin Kurc
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY
- Scientific Data Group, Oak Ridge National Laboratory, Oak Ridge, TN
| |
Collapse
|
21
|
Roszkowiak L, Lopez C. PATMA: parser of archival tissue microarray. PeerJ 2016; 4:e2741. [PMID: 27920955 PMCID: PMC5136132 DOI: 10.7717/peerj.2741] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2016] [Accepted: 10/31/2016] [Indexed: 02/04/2023] Open
Abstract
Tissue microarrays are commonly used in modern pathology for cancer tissue evaluation, as it is a very potent technique. Tissue microarray slides are often scanned to perform computer-aided histopathological analysis of the tissue cores. For processing the image, splitting the whole virtual slide into images of individual cores is required. The only way to distinguish cores corresponding to specimens in the tissue microarray is through their arrangement. Unfortunately, distinguishing the correct order of cores is not a trivial task as they are not labelled directly on the slide. The main aim of this study was to create a procedure capable of automatically finding and extracting cores from archival images of the tissue microarrays. This software supports the work of scientists who want to perform further image processing on single cores. The proposed method is an efficient and fast procedure, working in fully automatic or semi-automatic mode. A total of 89% of punches were correctly extracted with automatic selection. With an addition of manual correction, it is possible to fully prepare the whole slide image for extraction in 2 min per tissue microarray. The proposed technique requires minimum skill and time to parse big array of cores from tissue microarray whole slide image into individual core images.
Collapse
Affiliation(s)
- Lukasz Roszkowiak
- Laboratory of Processing Systems of Microscopic Image Information, Nalecz Institute of Biocybernetics and Biomedical Engineering, Polish Academy of Sciences, Warsaw, Poland
| | - Carlos Lopez
- Molecular Biology & Research Laboratory (IISPV, URV), Hospital de Tortosa Verge de la Cinta, Tortosa, Spain
| |
Collapse
|
22
|
Zhang X, Dou H, Ju T, Xu J, Zhang S. Fusing Heterogeneous Features From Stacked Sparse Autoencoder for Histopathological Image Analysis. IEEE J Biomed Health Inform 2016; 20:1377-83. [DOI: 10.1109/jbhi.2015.2461671] [Citation(s) in RCA: 53] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
23
|
Jiang M, Zhang S, Huang J, Yang L, Metaxas DN. Scalable histopathological image analysis via supervised hashing with multiple features. Med Image Anal 2016; 34:3-12. [PMID: 27521299 DOI: 10.1016/j.media.2016.07.011] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2016] [Revised: 04/08/2016] [Accepted: 07/28/2016] [Indexed: 11/18/2022]
Abstract
Histopathology is crucial to diagnosis of cancer, yet its interpretation is tedious and challenging. To facilitate this procedure, content-based image retrieval methods have been developed as case-based reasoning tools. Especially, with the rapid growth of digital histopathology, hashing-based retrieval approaches are gaining popularity due to their exceptional efficiency and scalability. Nevertheless, few hashing-based histopathological image analysis methods perform feature fusion, despite the fact that it is a common practice to improve image retrieval performance. In response, we exploit joint kernel-based supervised hashing (JKSH) to integrate complementary features in a hashing framework. Specifically, hashing functions are designed based on linearly combined kernel functions associated with individual features. Supervised information is incorporated to bridge the semantic gap between low-level features and high-level diagnosis. An alternating optimization method is utilized to learn the kernel combination and hashing functions. The obtained hashing functions compress multiple high-dimensional features into tens of binary bits, enabling fast retrieval from a large database. Our approach is extensively validated on 3121 breast-tissue histopathological images by distinguishing between actionable and benign cases. It achieves 88.1% retrieval precision and 91.3% classification accuracy within 16.5 ms query time, comparing favorably with traditional methods.
Collapse
Affiliation(s)
- Menglin Jiang
- Department of Computer Science, Rutgers University, Piscataway, NJ 08854, USA
| | - Shaoting Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA.
| | - Junzhou Huang
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
| | - Lin Yang
- Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Dimitris N Metaxas
- Department of Computer Science, Rutgers University, Piscataway, NJ 08854, USA
| |
Collapse
|
24
|
Xing F, Yang L. Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review. IEEE Rev Biomed Eng 2016; 9:234-63. [PMID: 26742143 PMCID: PMC5233461 DOI: 10.1109/rbme.2016.2515127] [Citation(s) in RCA: 212] [Impact Index Per Article: 26.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.
Collapse
|
25
|
MIAP – Web-based platform for the computer analysis of microscopic images to support the pathological diagnosis. Biocybern Biomed Eng 2016. [DOI: 10.1016/j.bbe.2016.06.006] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
26
|
Kurc T, Qi X, Wang D, Wang F, Teodoro G, Cooper L, Nalisnik M, Yang L, Saltz J, Foran DJ. Scalable analysis of Big pathology image data cohorts using efficient methods and high-performance computing strategies. BMC Bioinformatics 2015; 16:399. [PMID: 26627175 PMCID: PMC4667532 DOI: 10.1186/s12859-015-0831-6] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2015] [Accepted: 11/16/2015] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND We describe a suite of tools and methods that form a core set of capabilities for researchers and clinical investigators to evaluate multiple analytical pipelines and quantify sensitivity and variability of the results while conducting large-scale studies in investigative pathology and oncology. The overarching objective of the current investigation is to address the challenges of large data sizes and high computational demands. RESULTS The proposed tools and methods take advantage of state-of-the-art parallel machines and efficient content-based image searching strategies. The content based image retrieval (CBIR) algorithms can quickly detect and retrieve image patches similar to a query patch using a hierarchical analysis approach. The analysis component based on high performance computing can carry out consensus clustering on 500,000 data points using a large shared memory system. CONCLUSIONS Our work demonstrates efficient CBIR algorithms and high performance computing can be leveraged for efficient analysis of large microscopy images to meet the challenges of clinically salient applications in pathology. These technologies enable researchers and clinical investigators to make more effective use of the rich informational content contained within digitized microscopy specimens.
Collapse
Affiliation(s)
- Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, USA.
| | - Xin Qi
- Department of Pathology & Laboratory Medicine, Rutgers -- Robert Wood Johnson Medical School, New Brunswick, USA.
- Rutgers Cancer Institute of New Jersey, New Brunswick, USA.
| | - Daihou Wang
- Department of Electrical and Computer Engineering, Rutgers University, New Brunswick, USA.
| | - Fusheng Wang
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, USA.
- Department of Computer Science, Stony Brook University, Stony Brook, USA.
| | - George Teodoro
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, USA.
- Department of Computer Science, University of Brasilia, Brasília, Brazil.
| | - Lee Cooper
- Department of Biomedical Informatics, Emory University, Atlanta, USA.
| | - Michael Nalisnik
- Department of Biomedical Informatics, Emory University, Atlanta, USA.
| | - Lin Yang
- Department of Biomedical Engineering, University of Florida, Gainesville, USA.
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, USA.
| | - David J Foran
- Department of Pathology & Laboratory Medicine, Rutgers -- Robert Wood Johnson Medical School, New Brunswick, USA.
- Rutgers Cancer Institute of New Jersey, New Brunswick, USA.
| |
Collapse
|
27
|
Zhang X, Xing F, Su H, Yang L, Zhang S. High-throughput histopathological image analysis via robust cell segmentation and hashing. Med Image Anal 2015; 26:306-15. [PMID: 26599156 PMCID: PMC4679540 DOI: 10.1016/j.media.2015.10.005] [Citation(s) in RCA: 66] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2014] [Revised: 05/13/2015] [Accepted: 10/16/2015] [Indexed: 11/27/2022]
Abstract
Computer-aided diagnosis of histopathological images usually requires to examine all cells for accurate diagnosis. Traditional computational methods may have efficiency issues when performing cell-level analysis. In this paper, we propose a robust and scalable solution to enable such analysis in a real-time fashion. Specifically, a robust segmentation method is developed to delineate cells accurately using Gaussian-based hierarchical voting and repulsive balloon model. A large-scale image retrieval approach is also designed to examine and classify each cell of a testing image by comparing it with a massive database, e.g., half-million cells extracted from the training dataset. We evaluate this proposed framework on a challenging and important clinical use case, i.e., differentiation of two types of lung cancers (the adenocarcinoma and squamous carcinoma), using thousands of lung microscopic tissue images extracted from hundreds of patients. Our method has achieved promising accuracy and running time by searching among half-million cells .
Collapse
Affiliation(s)
- Xiaofan Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA
| | - Fuyong Xing
- Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Hai Su
- Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Lin Yang
- Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA; Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Shaoting Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA.
| |
Collapse
|
28
|
Jagga Z, Gupta D. Machine learning for biomarker identification in cancer research - developments toward its clinical application. Per Med 2015; 12:371-387. [PMID: 29771660 DOI: 10.2217/pme.15.5] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
The patterns identified from the systematically collected molecular profiles of patient tumor samples, along with clinical metadata, can assist personalized treatments for effective management of cancer patients with similar molecular subtypes. There is an unmet need to develop computational algorithms for cancer diagnosis, prognosis and therapeutics that can identify complex patterns and help in classifications based on plethora of emerging cancer research outcomes in public domain. Machine learning, a branch of artificial intelligence, holds a great potential for pattern recognition in cryptic cancer datasets, as evident from recent literature survey. In this review, we focus on the current status of machine learning applications in cancer research, highlighting trends and analyzing major achievements, roadblocks and challenges toward its implementation in clinics.
Collapse
Affiliation(s)
- Zeenia Jagga
- Bioinformatics Laboratory, Structural & Computational Biology Group, International Centre for Genetic Engineering & Biotechnology (ICGEB), Aruna Asaf Ali Marg, New Delhi 110 067, India
| | - Dinesh Gupta
- Bioinformatics Laboratory, Structural & Computational Biology Group, International Centre for Genetic Engineering & Biotechnology (ICGEB), Aruna Asaf Ali Marg, New Delhi 110 067, India
| |
Collapse
|
29
|
Zhang X, Liu W, Dundar M, Badve S, Zhang S. Towards large-scale histopathological image analysis: hashing-based image retrieval. IEEE TRANSACTIONS ON MEDICAL IMAGING 2015; 34:496-506. [PMID: 25314696 DOI: 10.1109/tmi.2014.2361481] [Citation(s) in RCA: 77] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Automatic analysis of histopathological images has been widely utilized leveraging computational image-processing methods and modern machine learning techniques. Both computer-aided diagnosis (CAD) and content-based image-retrieval (CBIR) systems have been successfully developed for diagnosis, disease detection, and decision support in this area. Recently, with the ever-increasing amount of annotated medical data, large-scale and data-driven methods have emerged to offer a promise of bridging the semantic gap between images and diagnostic information. In this paper, we focus on developing scalable image-retrieval techniques to cope intelligently with massive histopathological images. Specifically, we present a supervised kernel hashing technique which leverages a small amount of supervised information in learning to compress a 10 000-dimensional image feature vector into only tens of binary bits with the informative signatures preserved. These binary codes are then indexed into a hash table that enables real-time retrieval of images in a large database. Critically, the supervised information is employed to bridge the semantic gap between low-level image features and high-level diagnostic information. We build a scalable image-retrieval framework based on the supervised hashing technique and validate its performance on several thousand histopathological images acquired from breast microscopic tissues. Extensive evaluations are carried out in terms of image classification (i.e., benign versus actionable categorization) and retrieval tests. Our framework achieves about 88.1% classification accuracy as well as promising time efficiency. For example, the framework can execute around 800 queries in only 0.01 s, comparing favorably with other commonly used dimensionality reduction and feature selection methods.
Collapse
|
30
|
Veta M, Pluim JPW, van Diest PJ, Viergever MA. Breast cancer histopathology image analysis: a review. IEEE Trans Biomed Eng 2015; 61:1400-11. [PMID: 24759275 DOI: 10.1109/tbme.2014.2303852] [Citation(s) in RCA: 261] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
This paper presents an overview of methods that have been proposed for the analysis of breast cancer histopathology images. This research area has become particularly relevant with the advent of whole slide imaging (WSI) scanners, which can perform cost-effective and high-throughput histopathology slide digitization, and which aim at replacing the optical microscope as the primary tool used by pathologist. Breast cancer is the most prevalent form of cancers among women, and image analysis methods that target this disease have a huge potential to reduce the workload in a typical pathology lab and to improve the quality of the interpretation. This paper is meant as an introduction for nonexperts. It starts with an overview of the tissue preparation, staining and slide digitization processes followed by a discussion of the different image processing techniques and applications, ranging from analysis of tissue staining to computer-aided diagnosis, and prognosis of breast cancer patients.
Collapse
|
31
|
Yang L, Qi X, Xing F, Kurc T, Saltz J, Foran DJ. Parallel content-based sub-image retrieval using hierarchical searching. Bioinformatics 2013; 30:996-1002. [PMID: 24215030 DOI: 10.1093/bioinformatics/btt623] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
MOTIVATION The capacity to systematically search through large image collections and ensembles and detect regions exhibiting similar morphological characteristics is central to pathology diagnosis. Unfortunately, the primary methods used to search digitized, whole-slide histopathology specimens are slow and prone to inter- and intra-observer variability. The central objective of this research was to design, develop, and evaluate a content-based image retrieval system to assist doctors for quick and reliable content-based comparative search of similar prostate image patches. METHOD Given a representative image patch (sub-image), the algorithm will return a ranked ensemble of image patches throughout the entire whole-slide histology section which exhibits the most similar morphologic characteristics. This is accomplished by first performing hierarchical searching based on a newly developed hierarchical annular histogram (HAH). The set of candidates is then further refined in the second stage of processing by computing a color histogram from eight equally divided segments within each square annular bin defined in the original HAH. A demand-driven master-worker parallelization approach is employed to speed up the searching procedure. Using this strategy, the query patch is broadcasted to all worker processes. Each worker process is dynamically assigned an image by the master process to search for and return a ranked list of similar patches in the image. RESULTS The algorithm was tested using digitized hematoxylin and eosin (H&E) stained prostate cancer specimens. We have achieved an excellent image retrieval performance. The recall rate within the first 40 rank retrieved image patches is ∼90%. AVAILABILITY AND IMPLEMENTATION Both the testing data and source code can be downloaded from http://pleiad.umdnj.edu/CBII/Bioinformatics/.
Collapse
Affiliation(s)
- Lin Yang
- Division of Biomedical Informatics, Department of Biostatistics and Department of Computer Science, University of Kentucky, Lexington, KY, Center for Biomedical Imaging and Informatics, The Cancer Institute of New Jersey, Rutgers University, New Brunswick, NJ, Center for Comprehensive Informatics, Emory University, Atlanta, GA and Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | | | | | | | | | | |
Collapse
|
32
|
Vergara-Niedermayr C, Wang F, Pan T, Kurc T, Saltz J. Semantically Interoperable XML Data. INTERNATIONAL JOURNAL OF SEMANTIC COMPUTING 2013; 7:237-255. [PMID: 25298789 PMCID: PMC4185431 DOI: 10.1142/s1793351x13500037] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
XML is ubiquitously used as an information exchange platform for web-based applications in healthcare, life sciences, and many other domains. Proliferating XML data are now managed through latest native XML database technologies. XML data sources conforming to common XML schemas could be shared and integrated with syntactic interoperability. Semantic interoperability can be achieved through semantic annotations of data models using common data elements linked to concepts from ontologies. In this paper, we present a framework and software system to support the development of semantic interoperable XML based data sources that can be shared through a Grid infrastructure. We also present our work on supporting semantic validated XML data through semantic annotations for XML Schema, semantic validation and semantic authoring of XML data. We demonstrate the use of the system for a biomedical database of medical image annotations and markups.
Collapse
Affiliation(s)
| | - Fusheng Wang
- Center for Comprehensive Informatics, Department of Biomedical Informatics, Emory University, Atlanta, Georgia, USA
| | - Tony Pan
- Center for Comprehensive Informatics, Department of Biomedical Informatics, Emory University, Atlanta, Georgia, USA
| | - Tahsin Kurc
- Center for Comprehensive Informatics, Department of Biomedical Informatics, Emory University, Atlanta, Georgia, USA
| | - Joel Saltz
- Center for Comprehensive Informatics, Department of Biomedical Informatics, Emory University, Atlanta, Georgia, USA
| |
Collapse
|
33
|
Kothari S, Phan JH, Stokes TH, Wang MD. Pathology imaging informatics for quantitative analysis of whole-slide images. J Am Med Inform Assoc 2013; 20:1099-108. [PMID: 23959844 PMCID: PMC3822114 DOI: 10.1136/amiajnl-2012-001540] [Citation(s) in RCA: 182] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023] Open
Abstract
Objectives With the objective of bringing clinical decision support systems to reality, this article reviews histopathological whole-slide imaging informatics methods, associated challenges, and future research opportunities. Target audience This review targets pathologists and informaticians who have a limited understanding of the key aspects of whole-slide image (WSI) analysis and/or a limited knowledge of state-of-the-art technologies and analysis methods. Scope First, we discuss the importance of imaging informatics in pathology and highlight the challenges posed by histopathological WSI. Next, we provide a thorough review of current methods for: quality control of histopathological images; feature extraction that captures image properties at the pixel, object, and semantic levels; predictive modeling that utilizes image features for diagnostic or prognostic applications; and data and information visualization that explores WSI for de novo discovery. In addition, we highlight future research directions and discuss the impact of large public repositories of histopathological data, such as the Cancer Genome Atlas, on the field of pathology informatics. Following the review, we present a case study to illustrate a clinical decision support system that begins with quality control and ends with predictive modeling for several cancer endpoints. Currently, state-of-the-art software tools only provide limited image processing capabilities instead of complete data analysis for clinical decision-making. We aim to inspire researchers to conduct more research in pathology imaging informatics so that clinical decision support can become a reality.
Collapse
Affiliation(s)
- Sonal Kothari
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA
| | | | | | | |
Collapse
|
34
|
Abstract
Clinical research informatics is the rapidly evolving sub-discipline within biomedical informatics that focuses on developing new informatics theories, tools, and solutions to accelerate the full translational continuum: basic research to clinical trials (T1), clinical trials to academic health center practice (T2), diffusion and implementation to community practice (T3), and ‘real world’ outcomes (T4). We present a conceptual model based on an informatics-enabled clinical research workflow, integration across heterogeneous data sources, and core informatics tools and platforms. We use this conceptual model to highlight 18 new articles in the JAMIA special issue on clinical research informatics.
Collapse
Affiliation(s)
- Michael G Kahn
- Department of Pediatrics, University of Colorado, Aurora, Colorado 80045, USA.
| | | |
Collapse
|
35
|
Cooper LAD, Carter AB, Farris AB, Wang F, Kong J, Gutman DA, Widener P, Pan TC, Cholleti SR, Sharma A, Kurc TM, Brat DJ, Saltz JH. Digital Pathology: Data-Intensive Frontier in Medical Imaging: Health-information sharing, specifically of digital pathology, is the subject of this paper which discusses how sharing the rich images in pathology can stretch the capabilities of all otherwise well-practiced disciplines. PROCEEDINGS OF THE IEEE. INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS 2012; 100:991-1003. [PMID: 25328166 PMCID: PMC4197933 DOI: 10.1109/jproc.2011.2182074] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Pathology is a medical subspecialty that practices the diagnosis of disease. Microscopic examination of tissue reveals information enabling the pathologist to render accurate diagnoses and to guide therapy. The basic process by which anatomic pathologists render diagnoses has remained relatively unchanged over the last century, yet advances in information technology now offer significant opportunities in image-based diagnostic and research applications. Pathology has lagged behind other healthcare practices such as radiology where digital adoption is widespread. As devices that generate whole slide images become more practical and affordable, practices will increasingly adopt this technology and eventually produce an explosion of data that will quickly eclipse the already vast quantities of radiology imaging data. These advances are accompanied by significant challenges for data management and storage, but they also introduce new opportunities to improve patient care by streamlining and standardizing diagnostic approaches and uncovering disease mechanisms. Computer-based image analysis is already available in commercial diagnostic systems, but further advances in image analysis algorithms are warranted in order to fully realize the benefits of digital pathology in medical discovery and patient care. In coming decades, pathology image analysis will extend beyond the streamlining of diagnostic workflows and minimizing interobserver variability and will begin to provide diagnostic assistance, identify therapeutic targets, and predict patient outcomes and therapeutic responses.
Collapse
Affiliation(s)
- Lee A. D. Cooper
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Alexis B. Carter
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA 30306 USA
| | - Alton B. Farris
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA 30306 USA
| | - Fusheng Wang
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Jun Kong
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - David A. Gutman
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Patrick Widener
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Tony C. Pan
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Sharath R. Cholleti
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Ashish Sharma
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Tahsin M. Kurc
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| | - Daniel J. Brat
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA 30306 USA
| | - Joel H. Saltz
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30306 USA
| |
Collapse
|
36
|
Wang F, Oh TW, Vergara-Niedermayr C, Kurc T, Saltz J. Managing and Querying Whole Slide Images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2012; 8319:83190J. [PMID: 22844574 PMCID: PMC3405921 DOI: 10.1117/12.912388] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
High-resolution pathology images provide rich information about the morphological and functional characteristics of biological systems, and are transforming the field of pathology into a new era. To facilitate the use of digital pathology imaging for biomedical research and clinical diagnosis, it is essential to manage and query both whole slide images (WSI) and analytical results generated from images, such as annotations made by humans and computed features and classifications made by computer algorithms. There are unique requirements on modeling, managing and querying whole slide images, including compatibility with standards, scalability, support of image queries at multiple granularities, and support of integrated queries between images and derived results from the images. In this paper, we present our work on developing the Pathology Image Database System (PIDB), which is a standard oriented image database to support retrieval of images, tiles, regions and analytical results, image visualization and experiment management through a unified interface and architecture. The system is deployed for managing and querying whole slide images for In Silico brain tumor studies at Emory University. PIDB is generic and open source, and can be easily used to support other biomedical research projects. It has the potential to be integrated into a Picture Archiving and Communications System (PACS) with powerful query capabilities to support pathology imaging.
Collapse
Affiliation(s)
- Fusheng Wang
- Center for Comprehensive Informatics, Emory University, Atlanta, Georgia, USA
- Department of Biomedical Informatics, Emory University, Atlanta, USA
| | - Tae W. Oh
- Center for Comprehensive Informatics, Emory University, Atlanta, Georgia, USA
| | | | - Tahsin Kurc
- Center for Comprehensive Informatics, Emory University, Atlanta, Georgia, USA
- Department of Biomedical Informatics, Emory University, Atlanta, USA
| | - Joel Saltz
- Center for Comprehensive Informatics, Emory University, Atlanta, Georgia, USA
- Department of Biomedical Informatics, Emory University, Atlanta, USA
| |
Collapse
|
37
|
Kong J, Cooper LA, Wang F, Gutman DA, Gao J, Chisolm C, Sharma A, Pan T, Van Meir EG, Kurc TM, Moreno CS, Saltz JH, Brat DJ. Integrative, multimodal analysis of glioblastoma using TCGA molecular data, pathology images, and clinical outcomes. IEEE Trans Biomed Eng 2011; 58:3469-74. [PMID: 21947516 PMCID: PMC3292263 DOI: 10.1109/tbme.2011.2169256] [Citation(s) in RCA: 46] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Multimodal, multiscale data synthesis is becoming increasingly critical for successful translational biomedical research. In this letter, we present a large-scale investigative initiative on glioblastoma, a high-grade brain tumor, with complementary data types using in silico approaches. We integrate and analyze data from The Cancer Genome Atlas Project on glioblastoma that includes novel nuclear phenotypic data derived from microscopic slides, genotypic signatures described by transcriptional class and genetic alterations, and clinical outcomes defined by response to therapy and patient survival. Our preliminary results demonstrate numerous clinically and biologically significant correlations across multiple data types, revealing the power of in silico multimodal data integration for cancer research.
Collapse
Affiliation(s)
- Jun Kong
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Lee A.D. Cooper
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Fusheng Wang
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - David A. Gutman
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Jingjing Gao
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Candace Chisolm
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Ashish Sharma
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Tony Pan
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Erwin G. Van Meir
- Department of Neurosurgery and Hematology and Medical Oncology, School of Medicine and Winship Cancer Institute, Emory University, Atlanta, GA 30322, USA
| | - Tahsin M. Kurc
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Carlos S. Moreno
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Joel H. Saltz
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| | - Daniel J. Brat
- Center for Comprehensive Informatics, Emory University, Atlanta, GA 30322, USA
| |
Collapse
|
38
|
Butte AJ, Shah NH. Computationally translating molecular discoveries into tools for medicine: translational bioinformatics articles now featured in JAMIA. J Am Med Inform Assoc 2011; 18:352-3. [PMID: 21672904 DOI: 10.1136/amiajnl-2011-000343] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
|