1
|
Yu T, Zhong X, Li D, Zhu J, Tuchin VV, Zhu D. Delivery and kinetics of immersion optical clearing agents in tissues: Optical imaging from ex vivo to in vivo. Adv Drug Deliv Rev 2024:115470. [PMID: 39481483 DOI: 10.1016/j.addr.2024.115470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2024] [Revised: 09/30/2024] [Accepted: 10/27/2024] [Indexed: 11/02/2024]
Abstract
Advanced optical imaging provides a powerful tool for the structural and functional analysis of tissues with high resolution and contrast, but the imaging performance decreases as light propagates deeper into the tissue. Tissue optical clearing technique demonstrates an innovative way to realize deep-tissue imaging and have emerged substantially in the last two decades. Here, we briefly reviewed the basic principles of tissue optical clearing techniques in the view of delivery strategies via either free diffusion or external forces-driven advection, and the commonly-used optical techniques for monitoring kinetics of clearing agents in tissue, as well as their ex vivo to in vivo applications in multiple biomedical research fields. With future efforts on the even distribution of both clearing agents and probes, excavation of more effective clearing agents, and automation of tissue clearing processes, tissue optical clearing should provide more insights into the fundamental questions in biological events clinical diagnostics.
Collapse
Affiliation(s)
- Tingting Yu
- Britton Chance Center for Biomedical Photonics-MoE Key Laboratory for Biomedical Photonics, Advanced Biomedical Imaging Facility-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
| | - Xiang Zhong
- Britton Chance Center for Biomedical Photonics-MoE Key Laboratory for Biomedical Photonics, Advanced Biomedical Imaging Facility-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
| | - Dongyu Li
- Britton Chance Center for Biomedical Photonics-MoE Key Laboratory for Biomedical Photonics, Advanced Biomedical Imaging Facility-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China; School of Optical and Electronic Information-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
| | - Jingtan Zhu
- Britton Chance Center for Biomedical Photonics-MoE Key Laboratory for Biomedical Photonics, Advanced Biomedical Imaging Facility-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
| | - Valery V Tuchin
- Institute of Physics and Science Medical Center, Saratov State University, Saratov 410012, Russia; Laboratory of Laser Molecular Imaging and Machine Learning, Tomsk State University, Tomsk 634050, Russia; Institute of Precision Mechanics and Control, FRS "Saratov Scientific Centre of the RAS", Saratov 410028, Russia
| | - Dan Zhu
- Britton Chance Center for Biomedical Photonics-MoE Key Laboratory for Biomedical Photonics, Advanced Biomedical Imaging Facility-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China.
| |
Collapse
|
2
|
Pati P, Karkampouna S, Bonollo F, Compérat E, Radić M, Spahn M, Martinelli A, Wartenberg M, Kruithof-de Julio M, Rapsomaniki M. Accelerating histopathology workflows with generative AI-based virtually multiplexed tumour profiling. NAT MACH INTELL 2024; 6:1077-1093. [PMID: 39309216 PMCID: PMC11415301 DOI: 10.1038/s42256-024-00889-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 07/29/2024] [Indexed: 09/25/2024]
Abstract
Understanding the spatial heterogeneity of tumours and its links to disease initiation and progression is a cornerstone of cancer biology. Presently, histopathology workflows heavily rely on hematoxylin and eosin and serial immunohistochemistry staining, a cumbersome, tissue-exhaustive process that results in non-aligned tissue images. We propose the VirtualMultiplexer, a generative artificial intelligence toolkit that effectively synthesizes multiplexed immunohistochemistry images for several antibody markers (namely AR, NKX3.1, CD44, CD146, p53 and ERG) from only an input hematoxylin and eosin image. The VirtualMultiplexer captures biologically relevant staining patterns across tissue scales without requiring consecutive tissue sections, image registration or extensive expert annotations. Thorough qualitative and quantitative assessment indicates that the VirtualMultiplexer achieves rapid, robust and precise generation of virtually multiplexed imaging datasets of high staining quality that are indistinguishable from the real ones. The VirtualMultiplexer is successfully transferred across tissue scales and patient cohorts with no need for model fine-tuning. Crucially, the virtually multiplexed images enabled training a graph transformer that simultaneously learns from the joint spatial distribution of several proteins to predict clinically relevant endpoints. We observe that this multiplexed learning scheme was able to greatly improve clinical prediction, as corroborated across several downstream tasks, independent patient cohorts and cancer types. Our results showcase the clinical relevance of artificial intelligence-assisted multiplexed tumour imaging, accelerating histopathology workflows and cancer biology.
Collapse
Affiliation(s)
| | - Sofia Karkampouna
- Urology Research Laboratory, Department for BioMedical Research, University of Bern, Bern, Switzerland
- Department of Urology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Francesco Bonollo
- Urology Research Laboratory, Department for BioMedical Research, University of Bern, Bern, Switzerland
| | - Eva Compérat
- Department of Pathology, Medical University of Vienna, Vienna, Austria
| | - Martina Radić
- Urology Research Laboratory, Department for BioMedical Research, University of Bern, Bern, Switzerland
| | - Martin Spahn
- Department of Urology, Lindenhofspital Bern, Bern, Switzerland
- Department of Urology, University Duisburg-Essen, Essen, Germany
| | - Adriano Martinelli
- IBM Research Europe, Rüschlikon, Switzerland
- ETH Zürich, Zürich, Switzerland
- Biomedical Data Science Center, Lausanne University Hospital, Lausanne, Switzerland
| | - Martin Wartenberg
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Marianna Kruithof-de Julio
- Urology Research Laboratory, Department for BioMedical Research, University of Bern, Bern, Switzerland
- Department of Urology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
- Translational Organoid Resource, Department for BioMedical Research, University of Bern, Bern, Switzerland
| | - Marianna Rapsomaniki
- IBM Research Europe, Rüschlikon, Switzerland
- Biomedical Data Science Center, Lausanne University Hospital, Lausanne, Switzerland
- Faculty of Biology and Medicine, University of Lausanne, Lausanne, Switzerland
| |
Collapse
|
3
|
Hou X, Guan Z, Zhang X, Hu X, Zou S, Liang C, Shi L, Zhang K, You H. Evaluation of tumor budding with virtual panCK stains generated by novel multi-model CNN framework. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 257:108352. [PMID: 39241330 DOI: 10.1016/j.cmpb.2024.108352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 06/03/2024] [Accepted: 07/22/2024] [Indexed: 09/09/2024]
Abstract
As the global incidence of cancer continues to rise rapidly, the need for swift and precise diagnoses has become increasingly pressing. Pathologists commonly rely on H&E-panCK stain pairs for various aspects of cancer diagnosis, including the detection of occult tumor cells and the evaluation of tumor budding. Nevertheless, conventional chemical staining methods suffer from notable drawbacks, such as time-intensive processes and irreversible staining outcomes. The virtual stain technique, leveraging generative adversarial network (GAN), has emerged as a promising alternative to chemical stains. This approach aims to transform biopsy scans (often H&E) into other stain types. Despite achieving notable progress in recent years, current state-of-the-art virtual staining models confront challenges that hinder their efficacy, particularly in achieving accurate staining outcomes under specific conditions. These limitations have impeded the practical integration of virtual staining into diagnostic practices. To address the goal of producing virtual panCK stains capable of replacing chemical panCK, we propose an innovative multi-model framework. Our approach involves employing a combination of Mask-RCNN (for cell segmentation) and GAN models to extract cytokeratin distribution from chemical H&E images. Additionally, we introduce a tailored dynamic GAN model to convert H&E images into virtual panCK stains, integrating the derived cytokeratin distribution. Our framework is motivated by the fact that the unique pattern of the panCK is derived from cytokeratin distribution. As a proof of concept, we employ our virtual panCK stains to evaluate tumor budding in 45 H&E whole-slide images taken from breast cancer-invaded lymph nodes . Through thorough validation by both pathologists and the QuPath software, our virtual panCK stains demonstrate a remarkable level of accuracy. In stark contrast, the accuracy of state-of-the-art single cycleGAN virtual panCK stains is negligible. To our best knowledge, this is the first instance of a multi-model virtual panCK framework and the utilization of virtual panCK for tumor budding assessment. Our framework excels in generating dependable virtual panCK stains with significantly improved efficiency, thereby considerably reducing turnaround times in diagnosis. Furthermore, its outcomes are easily comprehensible even to pathologists who may not be well-versed in computer technology. We firmly believe that our framework has the potential to advance the field of virtual stain, thereby making significant strides towards improved cancer diagnosis.
Collapse
Affiliation(s)
- Xingzhong Hou
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China; School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, 100190, China
| | - Zhen Guan
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China
| | - Xianwei Zhang
- Department of Pathology, Henan Provincial People's Hospital; People's Hospital of Zhengzhou University, Zhengzhou, Henan 450003, China
| | - Xiao Hu
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Department of Pathology, Peking University Cancer Hospital & Institute, Beijing, China
| | - Shuangmei Zou
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Chunzi Liang
- School of Laboratory Medicine, Hubei University of Chinese Medicine, 16 Huangjia Lake West Road, Wuhan, Hubei 430065, China.
| | - Lulin Shi
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China; School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, 100190, China
| | - Kaitai Zhang
- State Key Laboratory of Molecular Oncology, Department of Etiology and Carcinogenesis, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China.
| | - Haihang You
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China; Zhongguancun Laboratory, Beijing 102206, China
| |
Collapse
|
4
|
Jacob AM, Lindemann AF, Wagenpfeil J, Geiger S, Layer YC, Salam B, Panahabadi S, Kurt D, Wintergerst MWM, Schildberg FA, Kuetting D, Attenberger UI, Abdullah Z, Böhner AMC. Autofluorescence-based tissue characterization enhances clinical prospects of light-sheet-microscopy. Sci Rep 2024; 14:18033. [PMID: 39098935 PMCID: PMC11298517 DOI: 10.1038/s41598-024-67366-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 07/10/2024] [Indexed: 08/06/2024] Open
Abstract
Light sheet fluorescence microscopy (LSFM) is a transformative imaging method that enables the visualization of non-dissected specimen in real-time 3D. Optical clearing of tissues is essential for LSFM, typically employing toxic solvents. Here, we test the applicability of a non-hazardous alternative, ethyl cinnamate (ECi). We comprehensively characterized autofluorescence (AF) spectra in diverse murine tissues-ocular globe, knee, and liver-employing LSFM under various excitation wavelengths (405-785 nm) to test the feasibility of unstained samples for diagnostic purposes, in particular regarding percutaneous biopsies, as they constitute to most harvested type of tissue sample in clinical routine. Ocular globe structures were best discerned with 640 nm excitation. Knee tissue showed complex variation in AF spectra variation influenced by tissue depth and structure. Liver exhibited a unique AF pattern, likely linked to vasculature. Hepatic tissue samples were used to demonstrate the compatibility of our protocol for antibody staining. Furthermore, we employed machine learning to augment raw images and segment liver structures based on AF spectra. Radiologists rated representative samples transferred to the clinical assessment software. Learning-generated images scored highest in quality. Additionally, we investigated an actual murine biopsy. Our study pioneers the application of AF spectra for tissue characterization and diagnostic potential of optically cleared unstained percutaneous biopsies, contributing to the clinical translation of LSFM.
Collapse
Affiliation(s)
- Alice M Jacob
- Institute of Molecular Medicine and Experimental Immunology, Medical Faculty, University Hospital Bonn, University of Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Anna F Lindemann
- Institute of Molecular Medicine and Experimental Immunology, Medical Faculty, University Hospital Bonn, University of Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Julia Wagenpfeil
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Sergej Geiger
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Yannik C Layer
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Babak Salam
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Sarah Panahabadi
- Clinic for Diagnostic and Interventional Neuroradiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Darius Kurt
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | | | - Frank A Schildberg
- Clinic for Orthopedics and Trauma Surgery, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Daniel Kuetting
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Ulrike I Attenberger
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna and General Hospital, Waehringer Guertel 18-20, 1090, Vienna, Austria
| | - Zeinab Abdullah
- Institute of Molecular Medicine and Experimental Immunology, Medical Faculty, University Hospital Bonn, University of Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
| | - Alexander M C Böhner
- Clinics for Diagnostic and Interventional Radiology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany.
| |
Collapse
|
5
|
Dai C, Zeng X, Zhang X, Liu Z, Cheng S. Machine learning-based integration develops a mitophagy-related lncRNA signature for predicting the progression of prostate cancer: a bioinformatic analysis. Discov Oncol 2024; 15:316. [PMID: 39073679 PMCID: PMC11286916 DOI: 10.1007/s12672-024-01189-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/26/2024] [Accepted: 07/23/2024] [Indexed: 07/30/2024] Open
Abstract
Prostate cancer remains a complex and challenging disease, necessitating innovative approaches for prognosis and therapeutic guidance. This study integrates machine learning techniques to develop a novel mitophagy-related long non-coding RNA (lncRNA) signature for predicting the progression of prostate cancer. Leveraging the TCGA-PRAD dataset, we identify a set of four key lncRNAs and formulate a riskscore, revealing its potential as a prognostic indicator. Subsequent analyses unravel the intricate connections between riskscore, immune cell infiltration, mutational landscapes, and treatment outcomes. Notably, the pan-cancer exploration of YEATS2-AS1 highlights its pervasive impact, demonstrating elevated expression across various malignancies. Furthermore, drug sensitivity predictions based on riskscore guide personalized chemotherapy strategies, with drugs like Carmustine and Entinostat showing distinct suitability for high and low-risk group patients. Regression analysis exposes significant correlations between the mitophagy-related lncRNAs, riskscore, and key mitophagy-related genes. Molecular docking analyses reveal promising interactions between Cyclophosphamide and proteins encoded by these genes, suggesting potential therapeutic avenues. This comprehensive study not only introduces a robust prognostic tool but also provides valuable insights into the molecular intricacies and potential therapeutic interventions in prostate cancer, paving the way for more personalized and effective clinical approaches.
Collapse
Affiliation(s)
- Caixia Dai
- Department of Urology, The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China
| | - Xiangju Zeng
- Department of Outpatient, The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China
| | - Xiuhong Zhang
- Department of Urology, The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China
| | - Ziqi Liu
- Department of Acupuncture and Moxibustion, The First Hospital of Hunan University of Chinese Medicine, Changsha, Hunan, China
| | - Shunhua Cheng
- Department of Urology, The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China.
| |
Collapse
|
6
|
Asadi-Aghbolaghi M, Darbandsari A, Zhang A, Contreras-Sanz A, Boschman J, Ahmadvand P, Köbel M, Farnell D, Huntsman DG, Churg A, Black PC, Wang G, Gilks CB, Farahani H, Bashashati A. Learning generalizable AI models for multi-center histopathology image classification. NPJ Precis Oncol 2024; 8:151. [PMID: 39030380 PMCID: PMC11271637 DOI: 10.1038/s41698-024-00652-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 07/11/2024] [Indexed: 07/21/2024] Open
Abstract
Investigation of histopathology slides by pathologists is an indispensable component of the routine diagnosis of cancer. Artificial intelligence (AI) has the potential to enhance diagnostic accuracy, improve efficiency, and patient outcomes in clinical pathology. However, variations in tissue preparation, staining protocols, and histopathology slide digitization could result in over-fitting of deep learning models when trained on the data from only one center, thereby underscoring the necessity to generalize deep learning networks for multi-center use. Several techniques, including the use of grayscale images, color normalization techniques, and Adversarial Domain Adaptation (ADA) have been suggested to generalize deep learning algorithms, but there are limitations to their effectiveness and discriminability. Convolutional Neural Networks (CNNs) exhibit higher sensitivity to variations in the amplitude spectrum, whereas humans predominantly rely on phase-related components for object recognition. As such, we propose Adversarial fourIer-based Domain Adaptation (AIDA) which applies the advantages of a Fourier transform in adversarial domain adaptation. We conducted a comprehensive examination of subtype classification tasks in four cancers, incorporating cases from multiple medical centers. Specifically, the datasets included multi-center data for 1113 ovarian cancer cases, 247 pleural cancer cases, 422 bladder cancer cases, and 482 breast cancer cases. Our proposed approach significantly improved performance, achieving superior classification results in the target domain, surpassing the baseline, color augmentation and normalization techniques, and ADA. Furthermore, extensive pathologist reviews suggested that our proposed approach, AIDA, successfully identifies known histotype-specific features. This superior performance highlights AIDA's potential in addressing generalization challenges in deep learning models for multi-center histopathology datasets.
Collapse
Affiliation(s)
| | - Amirali Darbandsari
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Allen Zhang
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | | | - Jeffrey Boschman
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Pouya Ahmadvand
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Martin Köbel
- Department of Pathology and Laboratory Medicine, University of Calgary, Calgary, AB, Canada
| | - David Farnell
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | - David G Huntsman
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- BC Cancer Research Institute, Vancouver, BC, Canada
| | - Andrew Churg
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | - Peter C Black
- Department of Urologic Sciences, University of British Columbia, Vancouver, BC, Canada
| | - Gang Wang
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| | - C Blake Gilks
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
- Vancouver General Hospital, Vancouver, BC, Canada
| | - Hossein Farahani
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Ali Bashashati
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.
| |
Collapse
|
7
|
Ertürk A. Deep 3D histology powered by tissue clearing, omics and AI. Nat Methods 2024; 21:1153-1165. [PMID: 38997593 DOI: 10.1038/s41592-024-02327-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Accepted: 05/28/2024] [Indexed: 07/14/2024]
Abstract
To comprehensively understand tissue and organism physiology and pathophysiology, it is essential to create complete three-dimensional (3D) cellular maps. These maps require structural data, such as the 3D configuration and positioning of tissues and cells, and molecular data on the constitution of each cell, spanning from the DNA sequence to protein expression. While single-cell transcriptomics is illuminating the cellular and molecular diversity across species and tissues, the 3D spatial context of these molecular data is often overlooked. Here, I discuss emerging 3D tissue histology techniques that add the missing third spatial dimension to biomedical research. Through innovations in tissue-clearing chemistry, labeling and volumetric imaging that enhance 3D reconstructions and their synergy with molecular techniques, these technologies will provide detailed blueprints of entire organs or organisms at the cellular level. Machine learning, especially deep learning, will be essential for extracting meaningful insights from the vast data. Further development of integrated structural, molecular and computational methods will unlock the full potential of next-generation 3D histology.
Collapse
Affiliation(s)
- Ali Ertürk
- Institute for Tissue Engineering and Regenerative Medicine, Helmholtz Zentrum München, Neuherberg, Germany.
- Institute for Stroke and Dementia Research, Klinikum der Universität München, Ludwig-Maximilians University, Munich, Germany.
- School of Medicine, Koç University, İstanbul, Turkey.
- Deep Piction GmbH, Munich, Germany.
| |
Collapse
|
8
|
Song AH, Williams M, Williamson DFK, Chow SSL, Jaume G, Gao G, Zhang A, Chen B, Baras AS, Serafin R, Colling R, Downes MR, Farré X, Humphrey P, Verrill C, True LD, Parwani AV, Liu JTC, Mahmood F. Analysis of 3D pathology samples using weakly supervised AI. Cell 2024; 187:2502-2520.e17. [PMID: 38729110 PMCID: PMC11168832 DOI: 10.1016/j.cell.2024.03.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 01/15/2024] [Accepted: 03/25/2024] [Indexed: 05/12/2024]
Abstract
Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.
Collapse
Affiliation(s)
- Andrew H Song
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Mane Williams
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Sarah S L Chow
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Gan Gao
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Andrew Zhang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Alexander S Baras
- Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA; Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Robert Serafin
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Richard Colling
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK
| | - Michelle R Downes
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Xavier Farré
- Public Health Agency of Catalonia, Lleida, Spain
| | - Peter Humphrey
- Department of Pathology, Yale School of Medicine, New Haven, CT, USA
| | - Clare Verrill
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Lawrence D True
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Jonathan T C Liu
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA.
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
| |
Collapse
|
9
|
Bishop KW, Erion Barner LA, Han Q, Baraznenok E, Lan L, Poudel C, Gao G, Serafin RB, Chow SSL, Glaser AK, Janowczyk A, Brenes D, Huang H, Miyasato D, True LD, Kang S, Vaughan JC, Liu JTC. An end-to-end workflow for nondestructive 3D pathology. Nat Protoc 2024; 19:1122-1148. [PMID: 38263522 DOI: 10.1038/s41596-023-00934-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 10/23/2023] [Indexed: 01/25/2024]
Abstract
Recent advances in 3D pathology offer the ability to image orders of magnitude more tissue than conventional pathology methods while also providing a volumetric context that is not achievable with 2D tissue sections, and all without requiring destructive tissue sectioning. Generating high-quality 3D pathology datasets on a consistent basis, however, is not trivial and requires careful attention to a series of details during tissue preparation, imaging and initial data processing, as well as iterative optimization of the entire process. Here, we provide an end-to-end procedure covering all aspects of a 3D pathology workflow (using light-sheet microscopy as an illustrative imaging platform) with sufficient detail to perform well-controlled preclinical and clinical studies. Although 3D pathology is compatible with diverse staining protocols and computationally generated color palettes for visual analysis, this protocol focuses on the use of a fluorescent analog of hematoxylin and eosin, which remains the most common stain used for gold-standard pathological reports. We present our guidelines for a broad range of end users (e.g., biologists, clinical researchers and engineers) in a simple format. The end-to-end workflow requires 3-6 d to complete, bearing in mind that data analysis may take longer.
Collapse
Affiliation(s)
- Kevin W Bishop
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
- Department of Bioengineering, University of Washington, Seattle, WA, USA
| | | | - Qinghua Han
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
- Department of Bioengineering, University of Washington, Seattle, WA, USA
| | - Elena Baraznenok
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
- Department of Bioengineering, University of Washington, Seattle, WA, USA
| | - Lydia Lan
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
- Department of Biology, University of Washington, Seattle, WA, USA
| | - Chetan Poudel
- Department of Chemistry, University of Washington, Seattle, WA, USA
| | - Gan Gao
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Robert B Serafin
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Sarah S L Chow
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Adam K Glaser
- Allen Institute for Neural Dynamics, Seattle, WA, USA
| | - Andrew Janowczyk
- Department of Biomedical Engineering, Emory University, Atlanta, GA, USA
- Department of Oncology, Division of Precision Oncology, University Hospital of Geneva, Geneva, Switzerland
- Department of Diagnostics, Division of Clinical Pathology, University Hospital of Geneva, Geneva, Switzerland
| | - David Brenes
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Hongyi Huang
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Dominie Miyasato
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Lawrence D True
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA
| | - Soyoung Kang
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Joshua C Vaughan
- Department of Chemistry, University of Washington, Seattle, WA, USA
- Department of Physiology and Biophysics, University of Washington, Seattle, WA, USA
| | - Jonathan T C Liu
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA.
- Department of Bioengineering, University of Washington, Seattle, WA, USA.
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA.
| |
Collapse
|
10
|
Abraham TM, Levenson R. Current Landscape of Advanced Imaging Tools for Pathology Diagnostics. Mod Pathol 2024; 37:100443. [PMID: 38311312 DOI: 10.1016/j.modpat.2024.100443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 12/13/2023] [Accepted: 01/26/2024] [Indexed: 02/10/2024]
Abstract
Histopathology relies on century-old workflows of formalin fixation, paraffin embedding, sectioning, and staining tissue specimens on glass slides. Despite being robust, this conventional process is slow, labor-intensive, and limited to providing two-dimensional views. Emerging technologies promise to enhance and accelerate histopathology. Slide-free microscopy allows rapid imaging of fresh, unsectioned specimens, overcoming slide preparation delays. Methods such as fluorescence confocal microscopy, multiphoton microscopy, along with more recent innovations including microscopy with UV surface excitation and fluorescence-imitating brightfield imaging can generate images resembling conventional histology directly from the surface of tissue specimens. Slide-free microscopy enable applications such as rapid intraoperative margin assessment and, with appropriate technology, three-dimensional histopathology. Multiomics profiling techniques, including imaging mass spectrometry and Raman spectroscopy, provide highly multiplexed molecular maps of tissues, although clinical translation remains challenging. Artificial intelligence is aiding the adoption of new imaging modalities via virtual staining, which converts methods such as slide-free microscopy into synthetic brightfield-like or even molecularly informed images. Although not yet commonplace, these emerging technologies collectively demonstrate the potential to modernize histopathology. Artificial intelligence-assisted workflows will ease the transition to new imaging modalities. With further validation, these advances may transform the century-old conventional histopathology pipeline to better serve 21st-century medicine. This review provides an overview of these enabling technology platforms and discusses their potential impact.
Collapse
Affiliation(s)
- Tanishq Mathew Abraham
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Richard Levenson
- Department of Pathology and Laboratory Medicine, UC Davis Health, Sacramento, California.
| |
Collapse
|
11
|
Wang R, Chow SSL, Serafin RB, Xie W, Han Q, Baraznenok E, Lan L, Bishop KW, Liu JTC. Direct three-dimensional segmentation of prostate glands with nnU-Net. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:036001. [PMID: 38434772 PMCID: PMC10905031 DOI: 10.1117/1.jbo.29.3.036001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Revised: 02/06/2024] [Accepted: 02/09/2024] [Indexed: 03/05/2024]
Abstract
Significance In recent years, we and others have developed non-destructive methods to obtain three-dimensional (3D) pathology datasets of clinical biopsies and surgical specimens. For prostate cancer risk stratification (prognostication), standard-of-care Gleason grading is based on examining the morphology of prostate glands in thin 2D sections. This motivates us to perform 3D segmentation of prostate glands in our 3D pathology datasets for the purposes of computational analysis of 3D glandular features that could offer improved prognostic performance. Aim To facilitate prostate cancer risk assessment, we developed a computationally efficient and accurate deep learning model for 3D gland segmentation based on open-top light-sheet microscopy datasets of human prostate biopsies stained with a fluorescent analog of hematoxylin and eosin (H&E). Approach For 3D gland segmentation based on our H&E-analog 3D pathology datasets, we previously developed a hybrid deep learning and computer vision-based pipeline, called image translation-assisted segmentation in 3D (ITAS3D), which required a complex two-stage procedure and tedious manual optimization of parameters. To simplify this procedure, we use the 3D gland-segmentation masks previously generated by ITAS3D as training datasets for a direct end-to-end deep learning-based segmentation model, nnU-Net. The inputs to this model are 3D pathology datasets of prostate biopsies rapidly stained with an inexpensive fluorescent analog of H&E and the outputs are 3D semantic segmentation masks of the gland epithelium, gland lumen, and surrounding stromal compartments within the tissue. Results nnU-Net demonstrates remarkable accuracy in 3D gland segmentations even with limited training data. Moreover, compared with the previous ITAS3D pipeline, nnU-Net operation is simpler and faster, and it can maintain good accuracy even with lower-resolution inputs. Conclusions Our trained DL-based 3D segmentation model will facilitate future studies to demonstrate the value of computational 3D pathology for guiding critical treatment decisions for patients with prostate cancer.
Collapse
Affiliation(s)
- Rui Wang
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
| | - Sarah S. L. Chow
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
| | - Robert B. Serafin
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
| | - Weisi Xie
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
| | - Qinghua Han
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
| | - Elena Baraznenok
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
| | - Lydia Lan
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
- University of Washington, Department of Biology, Seattle, Washington, United States
| | - Kevin W. Bishop
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
| | - Jonathan T. C. Liu
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
- University of Washington, Department of Laboratory Medicine and Pathology, Seattle, Washington, United States
| |
Collapse
|
12
|
Weber A, Enderle-Ammour K, Kurowski K, Metzger MC, Poxleitner P, Werner M, Rothweiler R, Beck J, Straehle J, Schmelzeisen R, Steybe D, Bronsert P. AI-Based Detection of Oral Squamous Cell Carcinoma with Raman Histology. Cancers (Basel) 2024; 16:689. [PMID: 38398080 PMCID: PMC10886627 DOI: 10.3390/cancers16040689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 02/02/2024] [Accepted: 02/02/2024] [Indexed: 02/25/2024] Open
Abstract
Stimulated Raman Histology (SRH) employs the stimulated Raman scattering (SRS) of photons at biomolecules in tissue samples to generate histological images. Subsequent pathological analysis allows for an intraoperative evaluation without the need for sectioning and staining. The objective of this study was to investigate a deep learning-based classification of oral squamous cell carcinoma (OSCC) and the sub-classification of non-malignant tissue types, as well as to compare the performances of the classifier between SRS and SRH images. Raman shifts were measured at wavenumbers k1 = 2845 cm-1 and k2 = 2930 cm-1. SRS images were transformed into SRH images resembling traditional H&E-stained frozen sections. The annotation of 6 tissue types was performed on images obtained from 80 tissue samples from eight OSCC patients. A VGG19-based convolutional neural network was then trained on 64 SRS images (and corresponding SRH images) and tested on 16. A balanced accuracy of 0.90 (0.87 for SRH images) and F1-scores of 0.91 (0.91 for SRH) for stroma, 0.98 (0.96 for SRH) for adipose tissue, 0.90 (0.87 for SRH) for squamous epithelium, 0.92 (0.76 for SRH) for muscle, 0.87 (0.90 for SRH) for glandular tissue, and 0.88 (0.87 for SRH) for tumor were achieved. The results of this study demonstrate the suitability of deep learning for the intraoperative identification of tissue types directly on SRS and SRH images.
Collapse
Affiliation(s)
- Andreas Weber
- Institute for Surgical Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Faculty of Biology, University of Freiburg, 79104 Freiburg, Germany
| | - Kathrin Enderle-Ammour
- Institute for Surgical Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - Konrad Kurowski
- Institute for Surgical Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Tumorbank Comprehensive Cancer Center Freiburg, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Core Facility for Histopathology and Digital Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - Marc C. Metzger
- Department of Oral and Maxillofacial Surgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - Philipp Poxleitner
- Department of Oral and Maxillofacial Surgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Center for Advanced Surgical Tissue Analysis (CAST), University of Freiburg, 79106 Freiburg, Germany
- Department of Oral and Maxillofacial Surgery and Facial Plastic Surgery, University Hospital, LMU Munich, 80337 Munich, Germany
| | - Martin Werner
- Institute for Surgical Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Tumorbank Comprehensive Cancer Center Freiburg, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - René Rothweiler
- Department of Oral and Maxillofacial Surgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - Jürgen Beck
- Center for Advanced Surgical Tissue Analysis (CAST), University of Freiburg, 79106 Freiburg, Germany
- Department of Neurosurgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - Jakob Straehle
- Center for Advanced Surgical Tissue Analysis (CAST), University of Freiburg, 79106 Freiburg, Germany
- Department of Neurosurgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| | - Rainer Schmelzeisen
- Department of Oral and Maxillofacial Surgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Center for Advanced Surgical Tissue Analysis (CAST), University of Freiburg, 79106 Freiburg, Germany
| | - David Steybe
- Department of Oral and Maxillofacial Surgery, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Center for Advanced Surgical Tissue Analysis (CAST), University of Freiburg, 79106 Freiburg, Germany
- Department of Oral and Maxillofacial Surgery and Facial Plastic Surgery, University Hospital, LMU Munich, 80337 Munich, Germany
| | - Peter Bronsert
- Institute for Surgical Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Tumorbank Comprehensive Cancer Center Freiburg, Medical Center, University of Freiburg, 79106 Freiburg, Germany
- Core Facility for Histopathology and Digital Pathology, Medical Center, University of Freiburg, 79106 Freiburg, Germany
| |
Collapse
|
13
|
Azadi Moghadam P, Bashashati A, Goldenberg SL. Artificial Intelligence and Pathomics: Prostate Cancer. Urol Clin North Am 2024; 51:15-26. [PMID: 37945099 DOI: 10.1016/j.ucl.2023.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2023]
Abstract
Artificial intelligence (AI) has the potential to transform pathologic diagnosis and cancer patient management as a predictive and prognostic biomarker. AI-based systems can be used to examine digitally scanned histopathology slides and differentiate benign from malignant cells and low from high grade. Deep learning models can analyze patient data from individual or multimodal combinations and identify patterns to be used to predict the response to different therapeutic options, the risk of recurrence or progression, and the prognosis of the newly diagnosed patient. AI-based models will improve treatment planning for patients with prostate cancer and improve the efficiency and cost-effectiveness of the pathology laboratory.
Collapse
Affiliation(s)
- Puria Azadi Moghadam
- Department of Electrical and Computer Engineering, University of British Columbia, 2332 Main Mall, Vancouver, British Columbia V6T 1Z4, Canada
| | - Ali Bashashati
- School of Biomedical Engineering, University of British Columbia, 2222 Health Sciences Mall, Vancouver, British Columbia V6T 1Z3, Canada; Department of Pathology and Laboratory Medicine, University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC V6T 1Z7, Canada
| | - S Larry Goldenberg
- Department of Urologic Sciences, University of British Columbia, 2775 Laurel Street, Vancouver British Columbia V5Z 1M9, Canada.
| |
Collapse
|
14
|
Levy JJ, Davis MJ, Chacko RS, Davis MJ, Fu LJ, Goel T, Pamal A, Nafi I, Angirekula A, Suvarna A, Vempati R, Christensen BC, Hayden MS, Vaickus LJ, LeBoeuf MR. Intraoperative margin assessment for basal cell carcinoma with deep learning and histologic tumor mapping to surgical site. NPJ Precis Oncol 2024; 8:2. [PMID: 38172524 PMCID: PMC10764333 DOI: 10.1038/s41698-023-00477-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Accepted: 11/14/2023] [Indexed: 01/05/2024] Open
Abstract
Successful treatment of solid cancers relies on complete surgical excision of the tumor either for definitive treatment or before adjuvant therapy. Intraoperative and postoperative radial sectioning, the most common form of margin assessment, can lead to incomplete excision and increase the risk of recurrence and repeat procedures. Mohs Micrographic Surgery is associated with complete removal of basal cell and squamous cell carcinoma through real-time margin assessment of 100% of the peripheral and deep margins. Real-time assessment in many tumor types is constrained by tissue size, complexity, and specimen processing / assessment time during general anesthesia. We developed an artificial intelligence platform to reduce the tissue preprocessing and histological assessment time through automated grossing recommendations, mapping and orientation of tumor to the surgical specimen. Using basal cell carcinoma as a model system, results demonstrate that this approach can address surgical laboratory efficiency bottlenecks for rapid and complete intraoperative margin assessment.
Collapse
Affiliation(s)
- Joshua J Levy
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, 90048, USA.
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, 90048, USA.
- Department of Dermatology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA.
- Emerging Diagnostic and Investigative Technologies, Clinical Genomics and Advanced Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Hitchcock Medical Center, Lebanon, NH, 03756, USA.
- Department of Epidemiology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA.
- Program in Quantitative Biomedical Sciences, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA.
| | - Matthew J Davis
- Department of Dermatology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
| | | | - Michael J Davis
- Department of Dermatology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
| | - Lucy J Fu
- Geisel School of Medicine at Dartmouth, Hanover, NH, 03755, USA
| | - Tarushii Goel
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, 22312, USA
- Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Akash Pamal
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, 22312, USA
- University of Virginia, Charlottesville, VA, 22903, USA
| | - Irfan Nafi
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, 22312, USA
- Stanford University, Palo Alto, CA, 94305, USA
| | - Abhinav Angirekula
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, 22312, USA
- University of Illinois Urbana-Champaign, Champaign, IL, 61820, USA
| | - Anish Suvarna
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, 22312, USA
| | - Ram Vempati
- Thomas Jefferson High School for Science and Technology, Alexandria, VA, 22312, USA
| | - Brock C Christensen
- Department of Dermatology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
- Department of Molecular and Systems Biology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
- Department of Community and Family Medicine, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
| | - Matthew S Hayden
- Department of Dermatology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
| | - Louis J Vaickus
- Emerging Diagnostic and Investigative Technologies, Clinical Genomics and Advanced Technologies, Department of Pathology and Laboratory Medicine, Dartmouth Hitchcock Medical Center, Lebanon, NH, 03756, USA
| | - Matthew R LeBoeuf
- Department of Dermatology, Geisel School of Medicine at Dartmouth, Hanover, NH, 03756, USA
| |
Collapse
|
15
|
Liu JTC, Chow SSL, Colling R, Downes MR, Farré X, Humphrey P, Janowczyk A, Mirtti T, Verrill C, Zlobec I, True LD. Engineering the future of 3D pathology. J Pathol Clin Res 2024; 10:e347. [PMID: 37919231 PMCID: PMC10807588 DOI: 10.1002/cjp2.347] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Revised: 10/06/2023] [Accepted: 10/15/2023] [Indexed: 11/04/2023]
Abstract
In recent years, technological advances in tissue preparation, high-throughput volumetric microscopy, and computational infrastructure have enabled rapid developments in nondestructive 3D pathology, in which high-resolution histologic datasets are obtained from thick tissue specimens, such as whole biopsies, without the need for physical sectioning onto glass slides. While 3D pathology generates massive datasets that are attractive for automated computational analysis, there is also a desire to use 3D pathology to improve the visual assessment of tissue histology. In this perspective, we discuss and provide examples of potential advantages of 3D pathology for the visual assessment of clinical specimens and the challenges of dealing with large 3D datasets (of individual or multiple specimens) that pathologists have not been trained to interpret. We discuss the need for artificial intelligence triaging algorithms and explainable analysis methods to assist pathologists or other domain experts in the interpretation of these novel, often complex, large datasets.
Collapse
Affiliation(s)
- Jonathan TC Liu
- Department of Mechanical EngineeringUniversity of WashingtonSeattleWAUSA
- Department of Laboratory Medicine & PathologyUniversity of Washington School of MedicineSeattleUSA
- Department of BioengineeringUniversity of WashingtonSeattleUSA
| | - Sarah SL Chow
- Department of Mechanical EngineeringUniversity of WashingtonSeattleWAUSA
| | | | | | | | - Peter Humphrey
- Department of UrologyYale School of MedicineNew HavenCTUSA
| | - Andrew Janowczyk
- Wallace H Coulter Department of Biomedical EngineeringEmory University and Georgia Institute of TechnologyAtlantaGAUSA
- Geneva University HospitalsGenevaSwitzerland
| | - Tuomas Mirtti
- Helsinki University Hospital and University of HelsinkiHelsinkiFinland
- Emory University School of MedicineAtlantaGAUSA
| | - Clare Verrill
- John Radcliffe HospitalUniversity of OxfordOxfordUK
- NIHR Oxford Biomedical Research CentreOxford University Hospitals NHS Foundation TrustOxfordUK
| | - Inti Zlobec
- Institute for Tissue Medicine and PathologyUniversity of BernBernSwitzerland
| | - Lawrence D True
- Department of Laboratory Medicine & PathologyUniversity of Washington School of MedicineSeattleUSA
| |
Collapse
|
16
|
Erion Barner LA, Gao G, Reddi DM, Lan L, Burke W, Mahmood F, Grady WM, Liu JTC. Artificial Intelligence-Triaged 3-Dimensional Pathology to Improve Detection of Esophageal Neoplasia While Reducing Pathologist Workloads. Mod Pathol 2023; 36:100322. [PMID: 37657711 DOI: 10.1016/j.modpat.2023.100322] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Revised: 07/25/2023] [Accepted: 08/25/2023] [Indexed: 09/03/2023]
Abstract
Early detection of esophageal neoplasia via evaluation of endoscopic surveillance biopsies is the key to maximizing survival for patients with Barrett's esophagus, but it is hampered by the sampling limitations of conventional slide-based histopathology. Comprehensive evaluation of whole biopsies with 3-dimensional (3D) pathology may improve early detection of malignancies, but large 3D pathology data sets are tedious for pathologists to analyze. Here, we present a deep learning-based method to automatically identify the most critical 2-dimensional (2D) image sections within 3D pathology data sets for pathologists to review. Our method first generates a 3D heatmap of neoplastic risk for each biopsy, then classifies all 2D image sections within the 3D data set in order of neoplastic risk. In a clinical validation study, we diagnose esophageal biopsies with artificial intelligence-triaged 3D pathology (3 images per biopsy) vs standard slide-based histopathology (16 images per biopsy) and show that our method improves detection sensitivity while reducing pathologist workloads.
Collapse
Affiliation(s)
| | - Gan Gao
- Department of Mechanical Engineering, University of Washington, Seattle, Washington
| | - Deepti M Reddi
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, Washington
| | - Lydia Lan
- Department of Mechanical Engineering, University of Washington, Seattle, Washington; Department of Biology, University of Washington, Seattle, Washington
| | - Wynn Burke
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, Washington; Department of Medicine (Gastroenterology Division), University of Washington School of Medicine, Seattle, Washington
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, Massachusetts; Harvard Data Science Initiative, Harvard University, Cambridge, Massachusetts
| | - William M Grady
- Clinical Research Division, Fred Hutchinson Cancer Research Center, Seattle, Washington
| | - Jonathan T C Liu
- Department of Mechanical Engineering, University of Washington, Seattle, Washington; Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, Washington; Department of Bioengineering, University of Washington, Seattle, Washington.
| |
Collapse
|
17
|
Koyuncu C, Janowczyk A, Farre X, Pathak T, Mirtti T, Fernandez PL, Pons L, Reder NP, Serafin R, Chow SSL, Viswanathan VS, Glaser AK, True LD, Liu JTC, Madabhushi A. Visual Assessment of 2-Dimensional Levels Within 3-Dimensional Pathology Data Sets of Prostate Needle Biopsies Reveals Substantial Spatial Heterogeneity. J Transl Med 2023; 103:100265. [PMID: 37858679 PMCID: PMC10926776 DOI: 10.1016/j.labinv.2023.100265] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 10/05/2023] [Accepted: 10/10/2023] [Indexed: 10/21/2023] Open
Abstract
Prostate cancer prognostication largely relies on visual assessment of a few thinly sectioned biopsy specimens under a microscope to assign a Gleason grade group (GG). Unfortunately, the assigned GG is not always associated with a patient's outcome in part because of the limited sampling of spatially heterogeneous tumors achieved by 2-dimensional histopathology. In this study, open-top light-sheet microscopy was used to obtain 3-dimensional pathology data sets that were assessed by 4 human readers. Intrabiopsy variability was assessed by asking readers to perform Gleason grading of 5 different levels per biopsy for a total of 20 core needle biopsies (ie, 100 total images). Intrabiopsy variability (Cohen κ) was calculated as the worst pairwise agreement in GG between individual levels within each biopsy and found to be 0.34, 0.34, 0.38, and 0.43 for the 4 pathologists. These preliminary results reveal that even within a 1-mm-diameter needle core, GG based on 2-dimensional images can vary dramatically depending on the location within a biopsy being analyzed. We believe that morphologic assessment of whole biopsies in 3 dimension has the potential to enable more reliable and consistent tumor grading.
Collapse
Affiliation(s)
- Can Koyuncu
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia
| | - Andrew Janowczyk
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia; Department of Oncology, Division of Precision Oncology, University Hospital of Geneva, Geneva, Switzerland; Department of Clinical Pathology, Division of Clinical Pathology, University Hospital of Geneva, Geneva, Switzerland
| | - Xavier Farre
- Public Health Agency of Catalonia, Lleida, Catalonia, Spain
| | - Tilak Pathak
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia
| | - Tuomas Mirtti
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia; Department of Pathology, University of Helsinki and Helsinki University, Hospital, Helsinki, Finland; Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland; iCAN-Digital Precision Cancer Medicine Flagship, Helsinki, Finland
| | - Pedro L Fernandez
- Department of Pathology, Hospital Germans Trias i Pujol, IGTP, Universidad Autonoma de Barcelona, Barcelona, Spain
| | - Laura Pons
- Department of Pathology, Hospital Germans Trias i Pujol, IGTP, Barcelona, Spain
| | - Nicholas P Reder
- Department of Mechanical Engineering, University of Washington, Seattle, Washington; Department of Laboratory Medicine & Pathology, University of Washington, Seattle, Washington
| | - Robert Serafin
- Department of Mechanical Engineering, University of Washington, Seattle, Washington
| | - Sarah S L Chow
- Department of Mechanical Engineering, University of Washington, Seattle, Washington
| | - Vidya S Viswanathan
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia
| | - Adam K Glaser
- Department of Mechanical Engineering, University of Washington, Seattle, Washington
| | - Lawrence D True
- Department of Laboratory Medicine & Pathology, University of Washington, Seattle, Washington; Department of Urology, University of Washington, Seattle, Washington
| | - Jonathan T C Liu
- Department of Mechanical Engineering, University of Washington, Seattle, Washington; Department of Laboratory Medicine & Pathology, University of Washington, Seattle, Washington; Department of Bioengineering, University of Washington, Seattle, Washington
| | - Anant Madabhushi
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia; Atlanta VA Medical Center, Atlanta, Georgia.
| |
Collapse
|
18
|
Lin S, Yu X, Chen H, Chen Z, Yang Y. Clinical efficacy of prostate PI-RADS V2.1 score combined with serum PSA-related indicators in the detection of gray zone prostate cancer. Int Urol Nephrol 2023; 55:2685-2693. [PMID: 37523077 DOI: 10.1007/s11255-023-03692-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 06/22/2023] [Indexed: 08/01/2023]
Abstract
PURPOSE The purpose of this study is to improve the diagnostic accuracy of gray zone prostate cancer evaluation by combining the prostate imaging report and data system version 2.1 (PI-RADS V2.1) score with serum prostate-specific antigen (PSA). METHODS We analyzed data from 212 men suspected of having prostate cancer and compared PSA-related indicators and PI-RADS V2.1 scores between 96 patients with prostate cancer and 116 without prostate cancer. By contrasting PI-RADS V2.1 scores with serum PSA-related markers, the diagnostic precision in the detection of grey zone prostate cancer was assessed. RESULTS The median PI-RADS V2.1 scores and serum tPSA levels of patients with prostate cancer were significantly higher (P < 0.05). The PI-RADS V2.1 score correlated positively with serum tPSA, PSA density (PSAD), and prostate health index (PHI) levels (P < 0.05) and negatively correlated with fPSA/tPSA concentrations (P < 0.05). Logistic regression identified risk factors including family history, PI-RADS V2.1 score, tPSA, PSAD, and PHI, with prostate volume and fPSA/tPSA as protective factors (P < 0.05). Combining serum PSA-related indicators with the PI-RADS V2.1 score improved diagnostic accuracy for gray zone prostate cancer (AUC 0.986, specificity 99.14%, sensitivity 92.71%). CONCLUSION The presence of a family history, a high PI-RADS V2.1 score, and elevated serum PSA-related markers contribute to high prostate cancer risk and development. The combined use of these indicators offers superior predictive value in detecting prostate cancer compared to a single indicator.
Collapse
Affiliation(s)
| | - XiuXiu Yu
- Wenzhou Medical University, Wenzhou, China
| | | | | | - Yu Yang
- Wenzhou Medical University, Wenzhou, China.
| |
Collapse
|
19
|
Wang N, Zhang C, Wei X, Yan T, Zhou W, Zhang J, Kang H, Yuan Z, Chen X. Harnessing the power of optical microscopy for visualization and analysis of histopathological images. BIOMEDICAL OPTICS EXPRESS 2023; 14:5451-5465. [PMID: 37854561 PMCID: PMC10581782 DOI: 10.1364/boe.501893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 08/29/2023] [Accepted: 09/01/2023] [Indexed: 10/20/2023]
Abstract
Histopathology is the foundation and gold standard for identifying diseases, and precise quantification of histopathological images can provide the pathologist with objective clues to make a more convincing diagnosis. Optical microscopy (OM), an important branch of optical imaging technology that provides high-resolution images of tissue cytology and structural morphology, has been used in the diagnosis of histopathology and evolved into a new disciplinary direction of optical microscopic histopathology (OMH). There are a number of ex-vivo studies providing applicability of different OMH approaches, and a transfer of these techniques toward in vivo diagnosis is currently in progress. Furthermore, combined with advanced artificial intelligence algorithms, OMH allows for improved diagnostic reliability and convenience due to the complementarity of retrieval information. In this review, we cover recent advances in OMH, including the exploration of new techniques in OMH as well as their applications, and look ahead to new challenges in OMH. These typical application examples well demonstrate the application potential and clinical value of OMH techniques in histopathological diagnosis.
Collapse
Affiliation(s)
- Nan Wang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Chang Zhang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
| | - Xinyu Wei
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
| | - Tianyu Yan
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Wangting Zhou
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Jiaojiao Zhang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Huan Kang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Zhen Yuan
- Faculty of Health Sciences, University of Macau, Macau, 999078, China
| | - Xueli Chen
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
- Inovation Center for Advanced Medical Imaging and Intelligent Medicine, Guangzhou Institute of Technology, Xidian University, Guangzhou, Guangdong 510555, China
| |
Collapse
|
20
|
Reddi DM, Barner LA, Burke W, Gao G, Grady WM, Liu JTC. Nondestructive 3D Pathology Image Atlas of Barrett Esophagus With Open-Top Light-Sheet Microscopy. Arch Pathol Lab Med 2023; 147:1164-1171. [PMID: 36596255 DOI: 10.5858/arpa.2022-0133-oa] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/01/2022] [Indexed: 01/04/2023]
Abstract
CONTEXT.— Anatomic pathologists render diagnosis on tissue samples sectioned onto glass slides and viewed under a bright-field microscope. This approach is destructive to the sample, which can limit its use for ancillary assays that can inform patient management. Furthermore, the subjective interpretation of a relatively small number of 2D tissue sections per sample contributes to low interobserver agreement among pathologists for the assessment (diagnosis and grading) of various lesions. OBJECTIVE.— To evaluate 3D pathology data sets of thick formalin-fixed Barrett esophagus specimens imaged nondestructively with open-top light-sheet (OTLS) microscopy. DESIGN.— Formalin-fixed, paraffin-embedded Barrett esophagus samples (N = 15) were deparaffinized, stained with a fluorescent analog of hematoxylin-eosin, optically cleared, and imaged nondestructively with OTLS microscopy. The OTLS microscopy images were subsequently compared with archived hematoxylin-eosin histology sections from each sample. RESULTS.— Barrett esophagus samples, both small endoscopic forceps biopsies and endoscopic mucosal resections, exhibited similar resolvable structures between OTLS microscopy and conventional light microscopy with up to a ×20 objective (×200 overall magnification). The 3D histologic images generated by OTLS microscopy can enable improved discrimination of cribriform and well-formed gland morphologies. In addition, a much larger amount of tissue is visualized with OTLS microscopy, which enables improved assessment of clinical specimens exhibiting high spatial heterogeneity. CONCLUSIONS.— In esophageal specimens, OTLS microscopy can generate images comparable in quality to conventional light microscopy, with the advantages of providing 3D information for enhanced evaluation of glandular morphologies and enabling much more of the tissue specimen to be visualized nondestructively.
Collapse
Affiliation(s)
- Deepti M Reddi
- From the Department of Laboratory Medicine and Pathology (Reddi, Liu), University of Washington, Seattle
| | - Lindsey A Barner
- Department of Mechanical Engineering (Barner, Gao, Liu), University of Washington, Seattle
| | - Wynn Burke
- Department of Medicine (Burke, Grady), University of Washington, Seattle
- The Clinical Research Division, Fred Hutchinson Cancer Research Center, Seattle, Washington (Burke, Grady)
| | - Gan Gao
- Department of Mechanical Engineering (Barner, Gao, Liu), University of Washington, Seattle
| | - William M Grady
- Department of Medicine (Burke, Grady), University of Washington, Seattle
- The Clinical Research Division, Fred Hutchinson Cancer Research Center, Seattle, Washington (Burke, Grady)
| | - Jonathan T C Liu
- From the Department of Laboratory Medicine and Pathology (Reddi, Liu), University of Washington, Seattle
- Department of Mechanical Engineering (Barner, Gao, Liu), University of Washington, Seattle
- Department of Bioengineering (Liu), University of Washington, Seattle
| |
Collapse
|
21
|
Falahkheirkhah K, Mukherjee SS, Gupta S, Herrera-Hernandez L, McCarthy MR, Jimenez RE, Cheville JC, Bhargava R. Accelerating Cancer Histopathology Workflows with Chemical Imaging and Machine Learning. CANCER RESEARCH COMMUNICATIONS 2023; 3:1875-1887. [PMID: 37772992 PMCID: PMC10506535 DOI: 10.1158/2767-9764.crc-23-0226] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/21/2023] [Revised: 08/21/2023] [Accepted: 08/21/2023] [Indexed: 09/30/2023]
Abstract
Histopathology has remained a cornerstone for biomedical tissue assessment for over a century, with a resource-intensive workflow involving biopsy or excision, gross examination, sampling, tissue processing to snap frozen or formalin-fixed paraffin-embedded blocks, sectioning, staining, optical imaging, and microscopic assessment. Emerging chemical imaging approaches, including stimulated Raman scattering (SRS) microscopy, can directly measure inherent molecular composition in tissue (thereby dispensing with the need for tissue processing, sectioning, and using dyes) and can use artificial intelligence (AI) algorithms to provide high-quality images. Here we show the integration of SRS microscopy in a pathology workflow to rapidly record chemical information from minimally processed fresh-frozen prostate tissue. Instead of using thin sections, we record data from intact thick tissues and use optical sectioning to generate images from multiple planes. We use a deep learning–based processing pipeline to generate virtual hematoxylin and eosin images. Next, we extend the computational method to generate archival-quality images in minutes, which are equivalent to those obtained from hours/days-long formalin-fixed, paraffin-embedded processing. We assessed the quality of images from the perspective of enabling pathologists to make decisions, demonstrating that the virtual stained image quality was diagnostically useful and the interpathologist agreement on prostate cancer grade was not impacted. Finally, because this method does not wash away lipids and small molecules, we assessed the utility of lipid chemical composition in determining grade. Together, the combination of chemical imaging and AI provides novel capabilities for rapid assessments in pathology by reducing the complexity and burden of current workflows. SIGNIFICANCE Archival-quality (formalin-fixed paraffin-embedded), thin-section diagnostic images are obtained from thick-cut, fresh-frozen prostate tissues without dyes or stains to expedite cancer histopathology by combining SRS microscopy and machine learning.
Collapse
Affiliation(s)
- Kianoush Falahkheirkhah
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois
| | - Sudipta S. Mukherjee
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois
| | - Sounak Gupta
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota
| | | | | | - Rafael E. Jimenez
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota
| | - John C. Cheville
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota
| | - Rohit Bhargava
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Chemical and Biomolecular Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Mechanical Science and Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Cancer Center at Illinois, University of Illinois Urbana-Champaign, Urbana, Illinois
| |
Collapse
|
22
|
Bishop KW, Barner LAE, Han Q, Baraznenok E, Lan L, Poudel C, Gao G, Serafin RB, Chow SS, Glaser AK, Janowczyk A, Brenes D, Huang H, Miyasato D, True LD, Kang S, Vaughan JC, Liu JT. An end-to-end workflow for non-destructive 3D pathology. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.03.551845. [PMID: 37577615 PMCID: PMC10418226 DOI: 10.1101/2023.08.03.551845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/15/2023]
Abstract
Recent advances in 3D pathology offer the ability to image orders-of-magnitude more tissue than conventional pathology while providing a volumetric context that is lacking with 2D tissue sections, all without requiring destructive tissue sectioning. Generating high-quality 3D pathology datasets on a consistent basis is non-trivial, requiring careful attention to many details regarding tissue preparation, imaging, and data/image processing in an iterative process. Here we provide an end-to-end protocol covering all aspects of a 3D pathology workflow (using light-sheet microscopy as an illustrative imaging platform) with sufficient detail to perform well-controlled preclinical and clinical studies. While 3D pathology is compatible with diverse staining protocols and computationally generated color palettes for visual analysis, this protocol will focus on a fluorescent analog of hematoxylin and eosin (H&E), which remains the most common stain for gold-standard diagnostic determinations. We present our guidelines for a broad range of end-users (e.g., biologists, clinical researchers, and engineers) in a simple tutorial format.
Collapse
Affiliation(s)
- Kevin W. Bishop
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | | | - Qinghua Han
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Elena Baraznenok
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Lydia Lan
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
- Department of Biology, University of Washington, Seattle, Washington, USA
| | - Chetan Poudel
- Department of Chemistry, University of Washington, Seattle, Washington, USA
| | - Gan Gao
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Robert B. Serafin
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Sarah S.L. Chow
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Adam K. Glaser
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Andrew Janowczyk
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA
- Department of Oncology, Division of Precision Oncology, University Hospital of Geneva, Geneva, Switzerland
- Department of Clinical Pathology, Division of Clinical Pathology, University Hospital of Geneva, Geneva, Switzerland
| | - David Brenes
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Hongyi Huang
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Dominie Miyasato
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Lawrence D. True
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
| | - Soyoung Kang
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
| | - Joshua C. Vaughan
- Department of Chemistry, University of Washington, Seattle, Washington, USA
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington, USA
| | - Jonathan T.C. Liu
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
| |
Collapse
|
23
|
Serafin R, Koyuncu C, Xie W, Huang H, Glaser AK, Reder NP, Janowczyk A, True LD, Madabhushi A, Liu JT. Nondestructive 3D pathology with analysis of nuclear features for prostate cancer risk assessment. J Pathol 2023; 260:390-401. [PMID: 37232213 PMCID: PMC10524574 DOI: 10.1002/path.6090] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 03/16/2023] [Accepted: 04/12/2023] [Indexed: 05/27/2023]
Abstract
Prostate cancer treatment decisions rely heavily on subjective visual interpretation [assigning Gleason patterns or International Society of Urological Pathology (ISUP) grade groups] of limited numbers of two-dimensional (2D) histology sections. Under this paradigm, interobserver variance is high, with ISUP grades not correlating well with outcome for individual patients, and this contributes to the over- and undertreatment of patients. Recent studies have demonstrated improved prognostication of prostate cancer outcomes based on computational analyses of glands and nuclei within 2D whole slide images. Our group has also shown that the computational analysis of three-dimensional (3D) glandular features, extracted from 3D pathology datasets of whole intact biopsies, can allow for improved recurrence prediction compared to corresponding 2D features. Here we seek to expand on these prior studies by exploring the prognostic value of 3D shape-based nuclear features in prostate cancer (e.g. nuclear size, sphericity). 3D pathology datasets were generated using open-top light-sheet (OTLS) microscopy of 102 cancer-containing biopsies extracted ex vivo from the prostatectomy specimens of 46 patients. A deep learning-based workflow was developed for 3D nuclear segmentation within the glandular epithelium versus stromal regions of the biopsies. 3D shape-based nuclear features were extracted, and a nested cross-validation scheme was used to train a supervised machine classifier based on 5-year biochemical recurrence (BCR) outcomes. Nuclear features of the glandular epithelium were found to be more prognostic than stromal cell nuclear features (area under the ROC curve [AUC] = 0.72 versus 0.63). 3D shape-based nuclear features of the glandular epithelium were also more strongly associated with the risk of BCR than analogous 2D features (AUC = 0.72 versus 0.62). The results of this preliminary investigation suggest that 3D shape-based nuclear features are associated with prostate cancer aggressiveness and could be of value for the development of decision-support tools. © 2023 The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Robert Serafin
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Can Koyuncu
- Wallace H Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | - Weisi Xie
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Hongyi Huang
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Adam K Glaser
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Nicholas P Reder
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Andrew Janowczyk
- Wallace H Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
- Precision Oncology Center Institute of Pathology, Lausanne University Hospital (CHUV), Lausanne, Switzerland
- Department of Clinical Pathology, Lausanne University Hospital (CHUV), Lausanne, Switzerland
| | - Lawrence D True
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Anant Madabhushi
- Wallace H Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
- Atlanta Veterans Affairs Medical Center, Decatur, GA, USA
| | - Jonathan Tc Liu
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
- Department of Bioengineering, University of Washington, Seattle, WA, USA
| |
Collapse
|
24
|
Song AH, Williams M, Williamson DFK, Jaume G, Zhang A, Chen B, Serafin R, Liu JTC, Baras A, Parwani AV, Mahmood F. Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples. ARXIV 2023:arXiv:2307.14907v1. [PMID: 37547660 PMCID: PMC10402184] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
Human tissue consists of complex structures that display a diversity of morphologies, forming a tissue microenvironment that is, by nature, three-dimensional (3D). However, the current standard-of-care involves slicing 3D tissue specimens into two-dimensional (2D) sections and selecting a few for microscopic evaluation1,2, with concomitant risks of sampling bias and misdiagnosis3-6. To this end, there have been intense efforts to capture 3D tissue morphology and transition to 3D pathology, with the development of multiple high-resolution 3D imaging modalities7-18. However, these tools have had little translation to clinical practice as manual evaluation of such large data by pathologists is impractical and there is a lack of computational platforms that can efficiently process the 3D images and provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy12-14 or microcomputed tomography15,16 and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting that there is value in capturing larger extents of spatially heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.
Collapse
Affiliation(s)
- Andrew H. Song
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Mane Williams
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Drew F. K. Williamson
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Andrew Zhang
- Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Robert Serafin
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Jonathan T. C. Liu
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Alex Baras
- Department of Pathology, Johns Hopkins Hospital, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Anil V. Parwani
- Department of Pathology, The Ohio State University, Columbus, Ohio, USA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| |
Collapse
|
25
|
Liu JTC, Glaser AK, Poudel C, Vaughan JC. Nondestructive 3D Pathology with Light-Sheet Fluorescence Microscopy for Translational Research and Clinical Assays. ANNUAL REVIEW OF ANALYTICAL CHEMISTRY (PALO ALTO, CALIF.) 2023; 16:231-252. [PMID: 36854208 DOI: 10.1146/annurev-anchem-091222-092734] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In recent years, there has been a revived appreciation for the importance of spatial context and morphological phenotypes for both understanding disease progression and guiding treatment decisions. Compared with conventional 2D histopathology, which is the current gold standard of medical diagnostics, nondestructive 3D pathology offers researchers and clinicians the ability to visualize orders of magnitude more tissue within their natural volumetric context. This has been enabled by rapid advances in tissue-preparation methods, high-throughput 3D microscopy instrumentation, and computational tools for processing these massive feature-rich data sets. Here, we provide a brief overview of many of these technical advances along with remaining challenges to be overcome. We also speculate on the future of 3D pathology as applied in translational investigations, preclinical drug development, and clinical decision-support assays.
Collapse
Affiliation(s)
- Jonathan T C Liu
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA;
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Adam K Glaser
- Department of Mechanical Engineering, University of Washington, Seattle, Washington, USA;
- Allen Institute for Neural Dynamics, Seattle, Washington, USA
| | - Chetan Poudel
- Department of Chemistry, University of Washington, Seattle, Washington, USA
| | - Joshua C Vaughan
- Department of Chemistry, University of Washington, Seattle, Washington, USA
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington, USA
| |
Collapse
|
26
|
Lapierre-Landry M, Liu Y, Bayat M, Wilson DL, Jenkins MW. Digital labeling for 3D histology: segmenting blood vessels without a vascular contrast agent using deep learning. BIOMEDICAL OPTICS EXPRESS 2023; 14:2416-2431. [PMID: 37342724 PMCID: PMC10278624 DOI: 10.1364/boe.480230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/12/2023] [Accepted: 02/20/2023] [Indexed: 06/23/2023]
Abstract
Recent advances in optical tissue clearing and three-dimensional (3D) fluorescence microscopy have enabled high resolution in situ imaging of intact tissues. Using simply prepared samples, we demonstrate here "digital labeling," a method to segment blood vessels in 3D volumes solely based on the autofluorescence signal and a nuclei stain (DAPI). We trained a deep-learning neural network based on the U-net architecture using a regression loss instead of a commonly used segmentation loss to achieve better detection of small vessels. We achieved high vessel detection accuracy and obtained accurate vascular morphometrics such as vessel length density and orientation. In the future, such digital labeling approach could easily be transferred to other biological structures.
Collapse
Affiliation(s)
| | - Yehe Liu
- Department of Biomedical Engineering, Case Western Reserve University, USA
| | - Mahdi Bayat
- Department of Electrical, Computer and Systems Engineering, Case Western Reserve University, USA
| | - David L. Wilson
- Department of Biomedical Engineering, Case Western Reserve University, USA
- Department of Radiology, Case Western Reserve University, USA
| | - Michael W. Jenkins
- Department of Biomedical Engineering, Case Western Reserve University, USA
- Department of Pediatrics, School of
Medicine, Case Western Reserve University, USA
| |
Collapse
|
27
|
Laurino A, Franceschini A, Pesce L, Cinci L, Montalbano A, Mazzamuto G, Sancataldo G, Nesi G, Costantini I, Silvestri L, Pavone FS. A Guide to Perform 3D Histology of Biological Tissues with Fluorescence Microscopy. Int J Mol Sci 2023; 24:ijms24076747. [PMID: 37047724 PMCID: PMC10094801 DOI: 10.3390/ijms24076747] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 03/27/2023] [Accepted: 03/29/2023] [Indexed: 04/09/2023] Open
Abstract
The analysis of histological alterations in all types of tissue is of primary importance in pathology for highly accurate and robust diagnosis. Recent advances in tissue clearing and fluorescence microscopy made the study of the anatomy of biological tissue possible in three dimensions. The combination of these techniques with classical hematoxylin and eosin (H&E) staining has led to the birth of three-dimensional (3D) histology. Here, we present an overview of the state-of-the-art methods, highlighting the optimal combinations of different clearing methods and advanced fluorescence microscopy techniques for the investigation of all types of biological tissues. We employed fluorescence nuclear and eosin Y staining that enabled us to obtain hematoxylin and eosin pseudo-coloring comparable with the gold standard H&E analysis. The computational reconstructions obtained with 3D optical imaging can be analyzed by a pathologist without any specific training in volumetric microscopy, paving the way for new biomedical applications in clinical pathology.
Collapse
Affiliation(s)
- Annunziatina Laurino
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
| | - Alessandra Franceschini
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
| | - Luca Pesce
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
| | - Lorenzo Cinci
- Department of Experimental and Clinical Biomedical Sciences, Radiodiagnostic Unit n. 2, Careggi University Hospital, 50134 Florence, Italy
| | - Alberto Montalbano
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Neurofarba Section of Pharmacology and Toxicology, University of Florence, 50139 Florence, Italy
| | - Giacomo Mazzamuto
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
- National Research Council—National Institute of Optics (CNR-INO), 50125 Sesto Fiorentino, Italy
| | - Giuseppe Sancataldo
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
| | - Gabriella Nesi
- Department of Health Sciences, University of Florence, 50139 Florence, Italy
| | - Irene Costantini
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- National Research Council—National Institute of Optics (CNR-INO), 50125 Sesto Fiorentino, Italy
- Department of Biology, University of Florence, 50019 Florence, Italy
| | - Ludovico Silvestri
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
- National Research Council—National Institute of Optics (CNR-INO), 50125 Sesto Fiorentino, Italy
| | - Francesco Saverio Pavone
- European Laboratory for Non-linear Spectroscopy, LENS, 50019 Sesto Fiorentino, Italy
- Department of Physics and Astronomy, University of Florence, 50019 Florence, Italy
- National Research Council—National Institute of Optics (CNR-INO), 50125 Sesto Fiorentino, Italy
| |
Collapse
|
28
|
Liang J, Zhang W, Yang J, Wu M, Dai Q, Yin H, Xiao Y, Kong L. Deep learning supported discovery of biomarkers for clinical prognosis of liver cancer. NAT MACH INTELL 2023. [DOI: 10.1038/s42256-023-00635-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/05/2023]
|
29
|
Bai B, Yang X, Li Y, Zhang Y, Pillar N, Ozcan A. Deep learning-enabled virtual histological staining of biological samples. LIGHT, SCIENCE & APPLICATIONS 2023; 12:57. [PMID: 36864032 PMCID: PMC9981740 DOI: 10.1038/s41377-023-01104-7] [Citation(s) in RCA: 42] [Impact Index Per Article: 42.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 02/10/2023] [Accepted: 02/14/2023] [Indexed: 06/18/2023]
Abstract
Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to as virtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.
Collapse
Affiliation(s)
- Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Xilin Yang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA.
| |
Collapse
|
30
|
Chu TN, Wong EY, Ma R, Yang CH, Dalieh IS, Hung AJ. Exploring the Use of Artificial Intelligence in the Management of Prostate Cancer. Curr Urol Rep 2023; 24:231-240. [PMID: 36808595 PMCID: PMC10090000 DOI: 10.1007/s11934-023-01149-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/30/2023] [Indexed: 02/21/2023]
Abstract
PURPOSE OF REVIEW This review aims to explore the current state of research on the use of artificial intelligence (AI) in the management of prostate cancer. We examine the various applications of AI in prostate cancer, including image analysis, prediction of treatment outcomes, and patient stratification. Additionally, the review will evaluate the current limitations and challenges faced in the implementation of AI in prostate cancer management. RECENT FINDINGS Recent literature has focused particularly on the use of AI in radiomics, pathomics, the evaluation of surgical skills, and patient outcomes. AI has the potential to revolutionize the future of prostate cancer management by improving diagnostic accuracy, treatment planning, and patient outcomes. Studies have shown improved accuracy and efficiency of AI models in the detection and treatment of prostate cancer, but further research is needed to understand its full potential as well as limitations.
Collapse
Affiliation(s)
- Timothy N Chu
- Center for Robotic Simulation & Education, Department of Urology, USC Institute of Urology, University of Southern California, Catherine & Joseph Aresty1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA
| | - Elyssa Y Wong
- Center for Robotic Simulation & Education, Department of Urology, USC Institute of Urology, University of Southern California, Catherine & Joseph Aresty1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA
| | - Runzhuo Ma
- Center for Robotic Simulation & Education, Department of Urology, USC Institute of Urology, University of Southern California, Catherine & Joseph Aresty1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA
| | - Cherine H Yang
- Center for Robotic Simulation & Education, Department of Urology, USC Institute of Urology, University of Southern California, Catherine & Joseph Aresty1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA
| | - Istabraq S Dalieh
- Center for Robotic Simulation & Education, Department of Urology, USC Institute of Urology, University of Southern California, Catherine & Joseph Aresty1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA
| | - Andrew J Hung
- Center for Robotic Simulation & Education, Department of Urology, USC Institute of Urology, University of Southern California, Catherine & Joseph Aresty1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
| |
Collapse
|
31
|
Ragab M, Kateb F, El-Sawy EK, Binyamin SS, Al-Rabia MW, A. Mansouri R. Archimedes Optimization Algorithm with Deep Learning-Based Prostate Cancer Classification on Magnetic Resonance Imaging. Healthcare (Basel) 2023; 11:healthcare11040590. [PMID: 36833124 PMCID: PMC9957347 DOI: 10.3390/healthcare11040590] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2022] [Revised: 02/03/2023] [Accepted: 02/06/2023] [Indexed: 02/18/2023] Open
Abstract
Prostate cancer (PCa) is becoming one of the most frequently occurring cancers among men and causes an even greater number of deaths. Due to the complexity of tumor masses, radiologists find it difficult to identify PCa accurately. Over the years, several PCa-detecting methods have been formulated, but these methods cannot identify cancer efficiently. Artificial Intelligence (AI) has both information technologies that simulate natural or biological phenomena and human intelligence in addressing issues. AI technologies have been broadly implemented in the healthcare domain, including 3D printing, disease diagnosis, health monitoring, hospital scheduling, clinical decision support, classification and prediction, and medical data analysis. These applications significantly boost the cost-effectiveness and accuracy of healthcare services. This article introduces an Archimedes Optimization Algorithm with Deep Learning-based Prostate Cancer Classification (AOADLB-P2C) model on MRI images. The presented AOADLB-P2C model examines MRI images for the identification of PCa. To accomplish this, the AOADLB-P2C model performs pre-processing in two stages: adaptive median filtering (AMF)-based noise removal and contrast enhancement. Additionally, the presented AOADLB-P2C model extracts features via a densely connected network (DenseNet-161) model with a root-mean-square propagation (RMSProp) optimizer. Finally, the presented AOADLB-P2C model classifies PCa using the AOA with a least-squares support vector machine (LS-SVM) method. The simulation values of the presented AOADLB-P2C model are tested using a benchmark MRI dataset. The comparative experimental results demonstrate the improvements of the AOADLB-P2C model over other recent approaches.
Collapse
Affiliation(s)
- Mahmoud Ragab
- Information Technology Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
- Department of Mathematics, Faculty of Science, Al-Azhar University, Cairo 11884, Egypt
- Correspondence:
| | - Faris Kateb
- Information Technology Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - E. K. El-Sawy
- Faculty of Earth Sciences, King Abdulaziz University, Jeddah 21589, Saudi Arabia
- Geology Department, Faculty of Science, Al-Azhar University (Assiut branch), Assiut 71524, Egypt
| | - Sami Saeed Binyamin
- Computer and Information Technology Department, The Applied College, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Mohammed W. Al-Rabia
- Department of Medical Microbiology and Parasitolog, Faculty of Medicine, King Abdulaziz University, Jeddah 21589, Saudi Arabia
- Health Promotion Center, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Rasha A. Mansouri
- Prince Sattam Bin Abdulaziz University, Al-Kharj 11942, Saudi Arabia
- Department of Biochemistry, Faculty of Sciences, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| |
Collapse
|
32
|
Ao J, Shao X, Liu Z, Liu Q, Xia J, Shi Y, Qi L, Pan J, Ji M. Stimulated Raman Scattering Microscopy Enables Gleason Scoring of Prostate Core Needle Biopsy by a Convolutional Neural Network. Cancer Res 2023; 83:641-651. [PMID: 36594873 PMCID: PMC9929517 DOI: 10.1158/0008-5472.can-22-2146] [Citation(s) in RCA: 25] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 10/19/2022] [Accepted: 12/29/2022] [Indexed: 01/04/2023]
Abstract
Focal therapy (FT) has been proposed as an approach to eradicate clinically significant prostate cancer while preserving the normal surrounding tissues to minimize treatment-related toxicity. Rapid histology of core needle biopsies is essential to ensure the precise FT for localized lesions and to determine tumor grades. However, it is difficult to achieve both high accuracy and speed with currently available histopathology methods. Here, we demonstrated that stimulated Raman scattering (SRS) microscopy could reveal the largely heterogeneous histologic features of fresh prostatic biopsy tissues in a label-free and near real-time manner. A diagnostic convolutional neural network (CNN) built based on images from 61 patients could classify Gleason patterns of prostate cancer with an accuracy of 85.7%. An additional 22 independent cases introduced as external test dataset validated the CNN performance with 84.4% accuracy. Gleason scores of core needle biopsies from 21 cases were calculated using the deep learning SRS system and showed a 71% diagnostic consistency with grading from three pathologists. This study demonstrates the potential of a deep learning-assisted SRS platform in evaluating the tumor grade of prostate cancer, which could help simplify the diagnostic workflow and provide timely histopathology compatible with FT treatment. SIGNIFICANCE A platform combining stimulated Raman scattering microscopy and a convolutional neural network provides rapid histopathology and automated Gleason scoring on fresh prostate core needle biopsies without complex tissue processing.
Collapse
Affiliation(s)
- Jianpeng Ao
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute of Fudan University, Fudan University, Shanghai, P.R. China
| | - Xiaoguang Shao
- Department of Urology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, P.R. China
| | - Zhijie Liu
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute of Fudan University, Fudan University, Shanghai, P.R. China
| | - Qiang Liu
- Department of Pathology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, P.R. China
| | - Jun Xia
- Department of Pathology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, P.R. China
| | - Yongheng Shi
- Department of Pathology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, P.R. China
| | - Lin Qi
- Department of Radiology, Huadong Hospital Affiliated to Fudan University, Shanghai, P.R. China
| | - Jiahua Pan
- Department of Urology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, P.R. China
| | - Minbiao Ji
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute of Fudan University, Fudan University, Shanghai, P.R. China
| |
Collapse
|
33
|
Artificial Intelligence for Clinical Diagnosis and Treatment of Prostate Cancer. Cancers (Basel) 2022; 14:cancers14225595. [PMID: 36428686 PMCID: PMC9688370 DOI: 10.3390/cancers14225595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 10/29/2022] [Accepted: 11/01/2022] [Indexed: 11/16/2022] Open
Abstract
As medical science and technology progress towards the era of "big data", a multi-dimensional dataset pertaining to medical diagnosis and treatment is becoming accessible for mathematical modelling. However, these datasets are frequently inconsistent, noisy, and often characterized by a significant degree of redundancy. Thus, extensive data processing is widely advised to clean the dataset before feeding it into the mathematical model. In this context, Artificial intelligence (AI) techniques, including machine learning (ML) and deep learning (DL) algorithms based on artificial neural networks (ANNs) and their types, are being used to produce a precise and cross-sectional illustration of clinical data. For prostate cancer patients, datasets derived from the prostate-specific antigen (PSA), MRI-guided biopsies, genetic biomarkers, and the Gleason grading are primarily used for diagnosis, risk stratification, and patient monitoring. However, recording diagnoses and further stratifying risks based on such diagnostic data frequently involves much subjectivity. Thus, implementing an AI algorithm on a PC's diagnostic data can reduce the subjectivity of the process and assist in decision making. In addition, AI is used to cut down the processing time and help with early detection, which provides a superior outcome in critical cases of prostate cancer. Furthermore, this also facilitates offering the service at a lower cost by reducing the amount of human labor. Herein, the prime objective of this review is to provide a deep analysis encompassing the existing AI algorithms that are being deployed in the field of prostate cancer (PC) for diagnosis and treatment. Based on the available literature, AI-powered technology has the potential for extensive growth and penetration in PC diagnosis and treatment to ease and expedite the existing medical process.
Collapse
|
34
|
Kiemen AL, Braxton AM, Grahn MP, Han KS, Babu JM, Reichel R, Jiang AC, Kim B, Hsu J, Amoa F, Reddy S, Hong SM, Cornish TC, Thompson ED, Huang P, Wood LD, Hruban RH, Wirtz D, Wu PH. CODA: quantitative 3D reconstruction of large tissues at cellular resolution. Nat Methods 2022; 19:1490-1499. [PMID: 36280719 PMCID: PMC10500590 DOI: 10.1038/s41592-022-01650-9] [Citation(s) in RCA: 55] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 09/14/2022] [Indexed: 12/15/2022]
Abstract
A central challenge in biology is obtaining high-content, high-resolution information while analyzing tissue samples at volumes relevant to disease progression. We address this here with CODA, a method to reconstruct exceptionally large (up to multicentimeter cubed) tissues at subcellular resolution using serially sectioned hematoxylin and eosin-stained tissue sections. Here we demonstrate CODA's ability to reconstruct three-dimensional (3D) distinct microanatomical structures in pancreas, skin, lung and liver tissues. CODA allows creation of readily quantifiable tissue volumes amenable to biological research. As a testbed, we assess the microanatomy of the human pancreas during tumorigenesis within the branching pancreatic ductal system, labeling ten distinct structures to examine heterogeneity and structural transformation during neoplastic progression. We show that pancreatic precancerous lesions develop into distinct 3D morphological phenotypes and that pancreatic cancer tends to spread far from the bulk tumor along collagen fibers that are highly aligned to the 3D curves of ductal, lobular, vascular and neural structures. Thus, CODA establishes a means to transform broadly the structural study of human diseases through exploration of exhaustively labeled 3D microarchitecture.
Collapse
Affiliation(s)
- Ashley L Kiemen
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Alicia M Braxton
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Mia P Grahn
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Kyu Sang Han
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Jaanvi Mahesh Babu
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Rebecca Reichel
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ann C Jiang
- Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Bridgette Kim
- Department of Mechanical Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Jocelyn Hsu
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Falone Amoa
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Sashank Reddy
- Department of Plastic and Reconstructive Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Seung-Mo Hong
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Toby C Cornish
- Department of Pathology, University of Colorado School of Medicine, Aurora, CO, USA
| | - Elizabeth D Thompson
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Peng Huang
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, USA
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Laura D Wood
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ralph H Hruban
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Denis Wirtz
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA.
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
- Department of Materials Science and Engineering, The Johns Hopkins University, Baltimore, MD, USA.
- Johns Hopkins Physical Sciences-Oncology Center, The Johns Hopkins University, Baltimore, MD, USA.
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
| | - Pei-Hsun Wu
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA.
- Johns Hopkins Physical Sciences-Oncology Center, The Johns Hopkins University, Baltimore, MD, USA.
| |
Collapse
|
35
|
Ternes L, Lin JR, Chen YA, Gray JW, Chang YH. Computational multiplex panel reduction to maximize information retention in breast cancer tissue microarrays. PLoS Comput Biol 2022; 18:e1010505. [PMID: 36178966 PMCID: PMC9555662 DOI: 10.1371/journal.pcbi.1010505] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Revised: 10/12/2022] [Accepted: 08/21/2022] [Indexed: 01/26/2023] Open
Abstract
Recent state-of-the-art multiplex imaging techniques have expanded the depth of information that can be captured within a single tissue sample by allowing for panels with dozens of markers. Despite this increase in capacity, space on the panel is still limited due to technical artifacts, tissue loss, and long imaging acquisition time. As such, selecting which markers to include on a panel is important, since removing important markers will result in a loss of biologically relevant information, but identifying redundant markers will provide a room for other markers. To address this, we propose computational approaches to determine the amount of shared information between markers and select an optimally reduced panel that captures maximum amount of information with the fewest markers. Here we examine several panel selection approaches and evaluate them based on their ability to reconstruct the full panel images and information within breast cancer tissue microarray datasets using cyclic immunofluorescence as a proof of concept. We show that all methods perform adequately and can re-capture cell types using only 18 of 25 markers (72% of the original panel size). The correlation-based selection methods achieved the best single-cell marker mean intensity predictions with a Spearman correlation of 0.90 with the reduced panel. Using the proposed methods shown here, it is possible for researchers to design more efficient multiplex imaging panels that maximize the amount of information retained with the limited number of markers with respect to certain evaluation metrics and architecture biases.
Collapse
Affiliation(s)
- Luke Ternes
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, Oregon, United States of America
| | - Jia-Ren Lin
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Yu-An Chen
- Ludwig Center at Harvard and Laboratory of Systems Pharmacology, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Joe W. Gray
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, Oregon, United States of America
| | - Young Hwan Chang
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, Oregon, United States of America
| |
Collapse
|
36
|
Mizuno K, Beltran H. Future directions for precision oncology in prostate cancer. Prostate 2022; 82 Suppl 1:S86-S96. [PMID: 35657153 PMCID: PMC9942493 DOI: 10.1002/pros.24354] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 03/28/2022] [Indexed: 11/06/2022]
Abstract
Clinical genomic testing is becoming routine in prostate cancer, as biomarker-driven therapies such as poly-ADP ribose polymerase (PARP) inhibitors and anti-PD1 immunotherapy are now approved for select men with castration-resistant prostate cancer harboring alterations in DNA repair genes. Challenges for precision medicine in prostate cancer include an overall low prevalence of actionable genomic alterations and a still limited understanding of the impact of tumor heterogeneity and co-occurring alterations on treatment response and outcomes across diverse patient populations. Expanded tissue-based technologies such as whole-genome sequencing, transcriptome analysis, epigenetic analysis, and single-cell RNA sequencing have not yet entered the clinical realm and could potentially improve upon our understanding of how molecular features of tumors, intratumoral heterogeneity, and the tumor microenvironment impact therapy response and resistance. Blood-based technologies including cell-free DNA, circulating tumor cells (CTCs), and extracellular vesicles (EVs) are less invasive molecular profiling resources that could also help capture intraindividual tumor heterogeneity and track dynamic changes that occur in the context of specific therapies. Furthermore, molecular imaging is an important biomarker tool within the framework of prostate cancer precision medicine with a capability to detect heterogeneity across metastases and potential therapeutic targets less invasively. Here, we review recent technological advances that may help promote the future implementation and value of precision oncology testing for patients with advanced prostate cancer.
Collapse
Affiliation(s)
- Kei Mizuno
- Department of Medical Oncology, Dana Farber Cancer Institute
| | - Himisha Beltran
- Department of Medical Oncology, Dana Farber Cancer Institute
| |
Collapse
|
37
|
Viswanathan VS, Toro P, Corredor G, Mukhopadhyay S, Madabhushi A. The state of the art for artificial intelligence in lung digital pathology. J Pathol 2022; 257:413-429. [PMID: 35579955 PMCID: PMC9254900 DOI: 10.1002/path.5966] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 04/26/2022] [Accepted: 05/15/2022] [Indexed: 12/03/2022]
Abstract
Lung diseases carry a significant burden of morbidity and mortality worldwide. The advent of digital pathology (DP) and an increase in computational power have led to the development of artificial intelligence (AI)-based tools that can assist pathologists and pulmonologists in improving clinical workflow and patient management. While previous works have explored the advances in computational approaches for breast, prostate, and head and neck cancers, there has been a growing interest in applying these technologies to lung diseases as well. The application of AI tools on radiology images for better characterization of indeterminate lung nodules, fibrotic lung disease, and lung cancer risk stratification has been well documented. In this article, we discuss methodologies used to build AI tools in lung DP, describing the various hand-crafted and deep learning-based unsupervised feature approaches. Next, we review AI tools across a wide spectrum of lung diseases including cancer, tuberculosis, idiopathic pulmonary fibrosis, and COVID-19. We discuss the utility of novel imaging biomarkers for different types of clinical problems including quantification of biomarkers like PD-L1, lung disease diagnosis, risk stratification, and prediction of response to treatments such as immune checkpoint inhibitors. We also look briefly at some emerging applications of AI tools in lung DP such as multimodal data analysis, 3D pathology, and transplant rejection. Lastly, we discuss the future of DP-based AI tools, describing the challenges with regulatory approval, developing reimbursement models, planning clinical deployment, and addressing AI biases. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
| | - Paula Toro
- Department of PathologyCleveland ClinicClevelandOHUSA
| | - Germán Corredor
- Department of Biomedical EngineeringCase Western Reserve UniversityClevelandOHUSA
- Louis Stokes Cleveland VA Medical CenterClevelandOHUSA
| | | | - Anant Madabhushi
- Department of Biomedical EngineeringCase Western Reserve UniversityClevelandOHUSA
- Louis Stokes Cleveland VA Medical CenterClevelandOHUSA
| |
Collapse
|
38
|
Mertz L. New Efforts in Biomedical Imaging. IEEE Pulse 2022; 13:2-7. [PMID: 36044471 DOI: 10.1109/mpuls.2022.3191382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Since the invention of the X-ray in 1895, biomedical imaging has come a long way and the pace of advances in the field are only accelerating. Research groups are progressing on many fronts through improvements to existing technologies and the development of novel imaging devices and algorithms that not only deliver better pictures, but also more-and more useful-information from the image data.
Collapse
|
39
|
Bankhead P. Developing image analysis methods for digital pathology. J Pathol 2022; 257:391-402. [PMID: 35481680 PMCID: PMC9324951 DOI: 10.1002/path.5921] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Revised: 04/22/2022] [Accepted: 04/25/2022] [Indexed: 12/04/2022]
Abstract
The potential to use quantitative image analysis and artificial intelligence is one of the driving forces behind digital pathology. However, despite novel image analysis methods for pathology being described across many publications, few become widely adopted and many are not applied in more than a single study. The explanation is often straightforward: software implementing the method is simply not available, or is too complex, incomplete, or dataset‐dependent for others to use. The result is a disconnect between what seems already possible in digital pathology based upon the literature, and what actually is possible for anyone wishing to apply it using currently available software. This review begins by introducing the main approaches and techniques involved in analysing pathology images. I then examine the practical challenges inherent in taking algorithms beyond proof‐of‐concept, from both a user and developer perspective. I describe the need for a collaborative and multidisciplinary approach to developing and validating meaningful new algorithms, and argue that openness, implementation, and usability deserve more attention among digital pathology researchers. The review ends with a discussion about how digital pathology could benefit from interacting with and learning from the wider bioimage analysis community, particularly with regard to sharing data, software, and ideas. © 2022 The Author. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Peter Bankhead
- Edinburgh Pathology, Institute of Genetics and Cancer, University of Edinburgh, Edinburgh, UK.,Centre for Genomic & Experimental Medicine, Institute of Genetics and Cancer, University of Edinburgh, Edinburgh, UK.,Cancer Research UK Edinburgh Centre, Institute of Genetics and Cancer, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
40
|
Kramer CJH, Vreeswijk MPG, Thijssen B, Bosse T, Wesseling J. Beyond the snapshot: optimizing prognostication and prediction by moving from fixed to functional multidimensional cancer pathology. J Pathol 2022; 257:403-412. [PMID: 35438188 PMCID: PMC9324156 DOI: 10.1002/path.5915] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Revised: 04/11/2022] [Accepted: 04/13/2022] [Indexed: 11/10/2022]
Abstract
The role of pathology in patient management has evolved over time from the retrospective review of cells, tissue, and disease (‘what happened’) to a prospective outlook (‘what will happen’). Examination of a static, two‐dimensional hematoxylin and eosin (H&E)‐stained tissue slide has traditionally been the pathologist's primary task, but novel ancillary techniques enabled by technological breakthroughs have supported pathologists in their increasing ability to predict disease status and behaviour. Nevertheless, the informational limits of 2D, fixed tissue are now being reached and technological innovation is urgently needed to ensure that our understanding of disease entities continues to support improved individualized treatment options. Here we review pioneering work currently underway in the field of cancer pathology that has the potential to capture information beyond the current basic snapshot. A selection of exciting new technologies is discussed that promise to facilitate integration of the functional and multidimensional (space and time) information needed to optimize the prognostic and predictive value of cancer pathology. Learning how to analyse, interpret, and apply the wealth of data acquired by these new approaches will challenge the knowledge and skills of the pathology community. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- C J H Kramer
- Department of Pathology, Leiden University Medical Center, Leiden, The Netherlands
| | - M P G Vreeswijk
- Department of Human Genetics, Leiden University Medical Center, Leiden, The Netherlands
| | - B Thijssen
- Division of Molecular Carcinogenesis, Oncode Institute, Netherlands Cancer Institute, Amsterdam, The Netherlands.,Department of Biomolecular Health Sciences, Faculty of Veterinary Medicine, Utrecht University, Utrecht, The Netherlands
| | - T Bosse
- Department of Pathology, Leiden University Medical Center, Leiden, The Netherlands
| | - J Wesseling
- Department of Pathology, Leiden University Medical Center, Leiden, The Netherlands.,Department of Pathology, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands.,Division of Molecular Pathology, Netherlands Cancer Institute, Amsterdam, The Netherlands
| |
Collapse
|
41
|
Bishop KW, Maitland KC, Rajadhyaksha M, Liu JTC. In vivo microscopy as an adjunctive tool to guide detection, diagnosis, and treatment. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-220032-PER. [PMID: 35478042 PMCID: PMC9043840 DOI: 10.1117/1.jbo.27.4.040601] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Accepted: 04/05/2022] [Indexed: 05/05/2023]
Abstract
SIGNIFICANCE There have been numerous academic and commercial efforts to develop high-resolution in vivo microscopes for a variety of clinical use cases, including early disease detection and surgical guidance. While many high-profile studies, commercialized products, and publications have resulted from these efforts, mainstream clinical adoption has been relatively slow other than for a few clinical applications (e.g., dermatology). AIM Here, our goals are threefold: (1) to introduce and motivate the need for in vivo microscopy (IVM) as an adjunctive tool for clinical detection, diagnosis, and treatment, (2) to discuss the key translational challenges facing the field, and (3) to propose best practices and recommendations to facilitate clinical adoption. APPROACH We will provide concrete examples from various clinical domains, such as dermatology, oral/gastrointestinal oncology, and neurosurgery, to reinforce our observations and recommendations. RESULTS While the incremental improvement and optimization of IVM technologies should and will continue to occur, future translational efforts would benefit from the following: (1) integrating clinical and industry partners upfront to define and maintain a compelling value proposition, (2) identifying multimodal/multiscale imaging workflows, which are necessary for success in most clinical scenarios, and (3) developing effective artificial intelligence tools for clinical decision support, tempered by a realization that complete adoption of such tools will be slow. CONCLUSIONS The convergence of imaging modalities, academic-industry-clinician partnerships, and new computational capabilities has the potential to catalyze rapid progress and adoption of IVM in the next few decades.
Collapse
Affiliation(s)
- Kevin W. Bishop
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
| | - Kristen C. Maitland
- Texas A&M University, Department of Biomedical Engineering, College Station, Texas, United States
| | - Milind Rajadhyaksha
- Memorial Sloan Kettering Cancer Center, Dermatology Service, New York, New York, United States
| | - Jonathan T. C. Liu
- University of Washington, Department of Bioengineering, Seattle, Washington, United States
- University of Washington, Department of Mechanical Engineering, Seattle, Washington, United States
- University of Washington, Department of Laboratory Medicine and Pathology, Seattle, Washington, United States
- Address all correspondence to Jonathan T.C. Liu,
| |
Collapse
|