1
|
Li Y, Pillar N, Li J, Liu T, Wu D, Sun S, Ma G, de Haan K, Huang L, Zhang Y, Hamidi S, Urisman A, Keidar Haran T, Wallace WD, Zuckerman JE, Ozcan A. Virtual histological staining of unlabeled autopsy tissue. Nat Commun 2024; 15:1684. [PMID: 38396004 PMCID: PMC10891155 DOI: 10.1038/s41467-024-46077-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2023] [Accepted: 02/09/2024] [Indexed: 02/25/2024] Open
Abstract
Traditional histochemical staining of post-mortem samples often confronts inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, and such chemical staining procedures covering large tissue areas demand substantial labor, cost and time. Here, we demonstrate virtual staining of autopsy tissue using a trained neural network to rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images, matching hematoxylin and eosin (H&E) stained versions of the same samples. The trained model can effectively accentuate nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining fails to provide consistent staining quality. This virtual autopsy staining technique provides a rapid and resource-efficient solution to generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.
Collapse
Affiliation(s)
- Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Jingxi Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Tairan Liu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Di Wu
- Computer Science Department, University of California, Los Angeles, CA, 90095, USA
| | - Songyu Sun
- Computer Science Department, University of California, Los Angeles, CA, 90095, USA
| | - Guangdong Ma
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- School of Physics, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Kevin de Haan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Luzhe Huang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Sepehr Hamidi
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine, University of California, Los Angeles, CA, 90095, USA
| | - Anatoly Urisman
- Department of Pathology, University of California, San Francisco, CA, 94143, USA
| | - Tal Keidar Haran
- Department of Pathology, Hadassah Hebrew University Medical Center, Jerusalem, 91120, Israel
| | - William Dean Wallace
- Department of Pathology, Keck School of Medicine, University of Southern California, Los Angeles, CA, 90033, USA
| | - Jonathan E Zuckerman
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine, University of California, Los Angeles, CA, 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA.
- Department of Surgery, University of California, Los Angeles, CA, 90095, USA.
| |
Collapse
|
2
|
Ferreira MFS, Brambilla G, Thévenaz L, Feng X, Zhang L, Sumetsky M, Jones C, Pedireddy S, Vollmer F, Dragic PD, Henderson-Sapir O, Ottaway DJ, Strupiechonski E, Hernandez-Cardoso GG, Hernandez-Serrano AI, González FJ, Castro Camus E, Méndez A, Saccomandi P, Quan Q, Xie Z, Reinhard BM, Diem M. Roadmap on optical sensors. JOURNAL OF OPTICS (2010) 2024; 26:013001. [PMID: 38116399 PMCID: PMC10726224 DOI: 10.1088/2040-8986/ad0e85] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 06/09/2023] [Accepted: 11/21/2023] [Indexed: 12/21/2023]
Abstract
Optical sensors and sensing technologies are playing a more and more important role in our modern world. From micro-probes to large devices used in such diverse areas like medical diagnosis, defence, monitoring of industrial and environmental conditions, optics can be used in a variety of ways to achieve compact, low cost, stand-off sensing with extreme sensitivity and selectivity. Actually, the challenges to the design and functioning of an optical sensor for a particular application requires intimate knowledge of the optical, material, and environmental properties that can affect its performance. This roadmap on optical sensors addresses different technologies and application areas. It is constituted by twelve contributions authored by world-leading experts, providing insight into the current state-of-the-art and the challenges their respective fields face. Two articles address the area of optical fibre sensors, encompassing both conventional and specialty optical fibres. Several other articles are dedicated to laser-based sensors, micro- and nano-engineered sensors, whispering-gallery mode and plasmonic sensors. The use of optical sensors in chemical, biological and biomedical areas is discussed in some other papers. Different approaches required to satisfy applications at visible, infrared and THz spectral regions are also discussed.
Collapse
Affiliation(s)
| | | | | | - Xian Feng
- Jiangsu Normal University, People’s Republic of China
| | - Lei Zhang
- Zhejiang University, People’s Republic of China
| | - Misha Sumetsky
- Aston Institute of Photonic Technologies, Aston University, Birmingham, United Kingdom
| | - Callum Jones
- Department of Physics and Astronomy, Living Systems Institute, University of Exeter, United Kingdom
| | - Srikanth Pedireddy
- Department of Physics and Astronomy, Living Systems Institute, University of Exeter, United Kingdom
| | - Frank Vollmer
- Department of Physics and Astronomy, Living Systems Institute, University of Exeter, United Kingdom
| | - Peter D Dragic
- University of Illinois at Urbana-Champaign, United States of America
| | - Ori Henderson-Sapir
- Department of Physics and Institute of Photonics and Advanced Sensing, The University of Adelaide, SA, Australia
- OzGrav, University of Adelaide, Adelaide, SA, Australia
- Mirage Photonics, Oaklands Park, SA, Australia
| | - David J Ottaway
- Department of Physics and Institute of Photonics and Advanced Sensing, The University of Adelaide, SA, Australia
- OzGrav, University of Adelaide, Adelaide, SA, Australia
| | | | | | | | | | | | | | - Paola Saccomandi
- Department of Mechanical Engineering, Politecnico di Milano, Italy
| | - Qimin Quan
- NanoMosaic Inc., United States of America
| | - Zhongcong Xie
- Massachusetts General Hospital and Harvard Medical School, United States of America
| | - Björn M Reinhard
- Department of Chemistry and The Photonics Center, Boston University, United States of America
| | - Max Diem
- Northeastern University and CIRECA LLC, United States of America
| |
Collapse
|
3
|
Falahkheirkhah K, Mukherjee SS, Gupta S, Herrera-Hernandez L, McCarthy MR, Jimenez RE, Cheville JC, Bhargava R. Accelerating Cancer Histopathology Workflows with Chemical Imaging and Machine Learning. CANCER RESEARCH COMMUNICATIONS 2023; 3:1875-1887. [PMID: 37772992 PMCID: PMC10506535 DOI: 10.1158/2767-9764.crc-23-0226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/21/2023] [Revised: 08/21/2023] [Accepted: 08/21/2023] [Indexed: 09/30/2023]
Abstract
Histopathology has remained a cornerstone for biomedical tissue assessment for over a century, with a resource-intensive workflow involving biopsy or excision, gross examination, sampling, tissue processing to snap frozen or formalin-fixed paraffin-embedded blocks, sectioning, staining, optical imaging, and microscopic assessment. Emerging chemical imaging approaches, including stimulated Raman scattering (SRS) microscopy, can directly measure inherent molecular composition in tissue (thereby dispensing with the need for tissue processing, sectioning, and using dyes) and can use artificial intelligence (AI) algorithms to provide high-quality images. Here we show the integration of SRS microscopy in a pathology workflow to rapidly record chemical information from minimally processed fresh-frozen prostate tissue. Instead of using thin sections, we record data from intact thick tissues and use optical sectioning to generate images from multiple planes. We use a deep learning–based processing pipeline to generate virtual hematoxylin and eosin images. Next, we extend the computational method to generate archival-quality images in minutes, which are equivalent to those obtained from hours/days-long formalin-fixed, paraffin-embedded processing. We assessed the quality of images from the perspective of enabling pathologists to make decisions, demonstrating that the virtual stained image quality was diagnostically useful and the interpathologist agreement on prostate cancer grade was not impacted. Finally, because this method does not wash away lipids and small molecules, we assessed the utility of lipid chemical composition in determining grade. Together, the combination of chemical imaging and AI provides novel capabilities for rapid assessments in pathology by reducing the complexity and burden of current workflows. SIGNIFICANCE Archival-quality (formalin-fixed paraffin-embedded), thin-section diagnostic images are obtained from thick-cut, fresh-frozen prostate tissues without dyes or stains to expedite cancer histopathology by combining SRS microscopy and machine learning.
Collapse
Affiliation(s)
- Kianoush Falahkheirkhah
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois
| | - Sudipta S. Mukherjee
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois
| | - Sounak Gupta
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota
| | | | | | - Rafael E. Jimenez
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota
| | - John C. Cheville
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota
| | - Rohit Bhargava
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Chemical and Biomolecular Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Mechanical Science and Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois
- Cancer Center at Illinois, University of Illinois Urbana-Champaign, Urbana, Illinois
| |
Collapse
|
4
|
Bhargava R. Digital Histopathology by Infrared Spectroscopic Imaging. ANNUAL REVIEW OF ANALYTICAL CHEMISTRY (PALO ALTO, CALIF.) 2023; 16:205-230. [PMID: 37068745 PMCID: PMC10408309 DOI: 10.1146/annurev-anchem-101422-090956] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Infrared (IR) spectroscopic imaging records spatially resolved molecular vibrational spectra, enabling a comprehensive measurement of the chemical makeup and heterogeneity of biological tissues. Combining this novel contrast mechanism in microscopy with the use of artificial intelligence can transform the practice of histopathology, which currently relies largely on human examination of morphologic patterns within stained tissue. First, this review summarizes IR imaging instrumentation especially suited to histopathology, analyses of its performance, and major trends. Second, an overview of data processing methods and application of machine learning is given, with an emphasis on the emerging use of deep learning. Third, a discussion on workflows in pathology is provided, with four categories proposed based on the complexity of methods and the analytical performance needed. Last, a set of guidelines, termed experimental and analytical specifications for spectroscopic imaging in histopathology, are proposed to help standardize the diversity of approaches in this emerging area.
Collapse
Affiliation(s)
- Rohit Bhargava
- Department of Bioengineering; Department of Electrical and Computer Engineering; Department of Mechanical Science and Engineering; Department of Chemical and Biomolecular Engineering; Department of Chemistry; Cancer Center at Illinois; and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA;
| |
Collapse
|
5
|
Khan U, Koivukoski S, Valkonen M, Latonen L, Ruusuvuori P. The effect of neural network architecture on virtual H&E staining: Systematic assessment of histological feasibility. PATTERNS (NEW YORK, N.Y.) 2023; 4:100725. [PMID: 37223268 PMCID: PMC10201298 DOI: 10.1016/j.patter.2023.100725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 12/23/2022] [Accepted: 03/08/2023] [Indexed: 05/25/2023]
Abstract
Conventional histopathology has relied on chemical staining for over a century. The staining process makes tissue sections visible to the human eye through a tedious and labor-intensive procedure that alters the tissue irreversibly, preventing repeated use of the sample. Deep learning-based virtual staining can potentially alleviate these shortcomings. Here, we used standard brightfield microscopy on unstained tissue sections and studied the impact of increased network capacity on the resulting virtually stained H&E images. Using the generative adversarial neural network model pix2pix as a baseline, we observed that replacing simple convolutions with dense convolution units increased the structural similarity score, peak signal-to-noise ratio, and nuclei reproduction accuracy. We also demonstrated highly accurate reproduction of histology, especially with increased network capacity, and demonstrated applicability to several tissues. We show that network architecture optimization can improve the image translation accuracy of virtual H&E staining, highlighting the potential of virtual staining in streamlining histopathological analysis.
Collapse
Affiliation(s)
- Umair Khan
- University of Turku, Institute of Biomedicine, Turku 20014, Finland
| | - Sonja Koivukoski
- University of Eastern Finland, Institute of Biomedicine, Kuopio 70211, Finland
| | - Mira Valkonen
- Tampere University, Faculty of Medicine and Health Technology, Tampere 33100, Finland
| | - Leena Latonen
- University of Eastern Finland, Institute of Biomedicine, Kuopio 70211, Finland
- Foundation for the Finnish Cancer Institute, Helsinki 00290, Finland
| | - Pekka Ruusuvuori
- University of Turku, Institute of Biomedicine, Turku 20014, Finland
- Tampere University, Faculty of Medicine and Health Technology, Tampere 33100, Finland
- FICAN West Cancer Centre, Cancer Research Unit, Turku University Hospital, Turku 20500, Finland
| |
Collapse
|
6
|
Bai B, Yang X, Li Y, Zhang Y, Pillar N, Ozcan A. Deep learning-enabled virtual histological staining of biological samples. LIGHT, SCIENCE & APPLICATIONS 2023; 12:57. [PMID: 36864032 PMCID: PMC9981740 DOI: 10.1038/s41377-023-01104-7] [Citation(s) in RCA: 33] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 02/10/2023] [Accepted: 02/14/2023] [Indexed: 06/18/2023]
Abstract
Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to as virtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.
Collapse
Affiliation(s)
- Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Xilin Yang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA.
| |
Collapse
|
7
|
Pillar N, Ozcan A. Virtual tissue staining in pathology using machine learning. Expert Rev Mol Diagn 2022; 22:987-989. [PMID: 36440487 DOI: 10.1080/14737159.2022.2153040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 11/25/2022] [Indexed: 11/29/2022]
Affiliation(s)
- Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, USA
- Bioengineering Department, University of California, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, USA
- Bioengineering Department, University of California, Los Angeles, CA, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
- Department of Surgery, University of California, Los Angeles, CA, USA
| |
Collapse
|
8
|
Cameron JM, Rinaldi C, Rutherford SH, Sala A, G Theakstone A, Baker MJ. Clinical Spectroscopy: Lost in Translation? APPLIED SPECTROSCOPY 2022; 76:393-415. [PMID: 34041957 DOI: 10.1177/00037028211021846] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
This Focal Point Review paper discusses the developments of biomedical Raman and infrared spectroscopy, and the recent strive towards these technologies being regarded as reliable clinical tools. The promise of vibrational spectroscopy in the field of biomedical science, alongside the development of computational methods for spectral analysis, has driven a plethora of proof-of-concept studies which convey the potential of various spectroscopic approaches. Here we report a brief review of the literature published over the past few decades, with a focus on the current technical, clinical, and economic barriers to translation, namely the limitations of many of the early studies, and the lack of understanding of clinical pathways, health technology assessments, regulatory approval, clinical feasibility, and funding applications. The field of biomedical vibrational spectroscopy must acknowledge and overcome these hurdles in order to achieve clinical efficacy. Current prospects have been overviewed with comment on the advised future direction of spectroscopic technologies, with the aspiration that many of these innovative approaches can ultimately reach the frontier of medical diagnostics and many clinical applications.
Collapse
Affiliation(s)
| | - Christopher Rinaldi
- WestCHEM, Department of Pure and Applied Chemistry, Technology and Innovation Centre, Glasgow, UK
| | - Samantha H Rutherford
- WestCHEM, Department of Pure and Applied Chemistry, Technology and Innovation Centre, Glasgow, UK
| | - Alexandra Sala
- WestCHEM, Department of Pure and Applied Chemistry, Technology and Innovation Centre, Glasgow, UK
| | - Ashton G Theakstone
- WestCHEM, Department of Pure and Applied Chemistry, Technology and Innovation Centre, Glasgow, UK
| | | |
Collapse
|
9
|
He Y, Li J, Shen S, Liu K, Wong KK, He T, Wong STC. Image-to-image translation of label-free molecular vibrational images for a histopathological review using the UNet+/seg-cGAN model. BIOMEDICAL OPTICS EXPRESS 2022; 13:1924-1938. [PMID: 35519236 PMCID: PMC9045908 DOI: 10.1364/boe.445319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Revised: 12/23/2021] [Accepted: 02/09/2022] [Indexed: 06/14/2023]
Abstract
Translating images generated by label-free microscopy imaging, such as Coherent Anti-Stokes Raman Scattering (CARS), into more familiar clinical presentations of histopathological images will help the adoption of real-time, spectrally resolved label-free imaging in clinical diagnosis. Generative adversarial networks (GAN) have made great progress in image generation and translation, but have been criticized for lacking precision. In particular, GAN has often misinterpreted image information and identified incorrect content categories during image translation of microscopy scans. To alleviate this problem, we developed a new Pix2pix GAN model that simultaneously learns classifying contents in the images from a segmentation dataset during the image translation training. Our model integrates UNet+ with seg-cGAN, conditional generative adversarial networks with partial regularization of segmentation. Technical innovations of the UNet+/seg-cGAN model include: (1) replacing UNet with UNet+ as the Pix2pix cGAN's generator to enhance pattern extraction and richness of the gradient, and (2) applying the partial regularization strategy to train a part of the generator network as the segmentation sub-model on a separate segmentation dataset, thus enabling the model to identify correct content categories during image translation. The quality of histopathological-like images generated based on label-free CARS images has been improved significantly.
Collapse
Affiliation(s)
- Yunjie He
- Translational Biophotonics Laboratory,
Systems Medicine and Bioengineering Department, Houston
Methodist Cancer Center, Houston, USA
| | - Jiasong Li
- Translational Biophotonics Laboratory,
Systems Medicine and Bioengineering Department, Houston
Methodist Cancer Center, Houston, USA
| | - Steven Shen
- Pathology and Genome Medicine Department,
Houston Methodist Hospital, Weill Cornell
Medicine, Houston, USA
| | - Kai Liu
- Translational Biophotonics Laboratory,
Systems Medicine and Bioengineering Department, Houston
Methodist Cancer Center, Houston, USA
| | - Kelvin K. Wong
- Translational Biophotonics Laboratory,
Systems Medicine and Bioengineering Department, Houston
Methodist Cancer Center, Houston, USA
- T.T. and W. F. Chao Center for BRAIN,
Houston Methodist Academic Institute,
USA
| | - Tiancheng He
- Translational Biophotonics Laboratory,
Systems Medicine and Bioengineering Department, Houston
Methodist Cancer Center, Houston, USA
| | - Stephen T. C. Wong
- Translational Biophotonics Laboratory,
Systems Medicine and Bioengineering Department, Houston
Methodist Cancer Center, Houston, USA
- Pathology and Genome Medicine Department,
Houston Methodist Hospital, Weill Cornell
Medicine, Houston, USA
- T.T. and W. F. Chao Center for BRAIN,
Houston Methodist Academic Institute,
USA
| |
Collapse
|
10
|
Goertzen N, Pappesch R, Fassunke J, Brüning T, Ko YD, Schmidt J, Großerueschkamp F, Buettner R, Gerwert K. Quantum Cascade Laser-Based Infrared Imaging as a Label-Free and Automated Approach to Determine Mutations in Lung Adenocarcinoma. THE AMERICAN JOURNAL OF PATHOLOGY 2021; 191:1269-1280. [PMID: 34004158 DOI: 10.1016/j.ajpath.2021.04.013] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 04/09/2021] [Accepted: 04/22/2021] [Indexed: 12/28/2022]
Abstract
Therapeutic decisions in lung cancer critically depend on the determination of histologic types and oncogene mutations. Therefore, tumor samples are subjected to standard histologic and immunohistochemical analyses and examined for relevant mutations using comprehensive molecular diagnostics. In this study, an alternative diagnostic approach for automatic and label-free detection of mutations in lung adenocarcinoma tissue using quantum cascade laser-based infrared imaging is presented. For this purpose, a five-step supervised classification algorithm was developed, which was not only able to detect tissue types and tumor lesions, but also the tumor type and mutation status of adenocarcinomas. Tumor detection was verified on a data set of 214 patient samples with a specificity of 97% and a sensitivity of 95%. Furthermore, histology typing was verified on samples from 203 of the 214 patients with a specificity of 97% and a sensitivity of 94% for adenocarcinoma. The most frequently occurring mutations in adenocarcinoma (KRAS, EGFR, and TP53) were differentiated by this technique. Detection of mutations was verified in 60 patient samples from the data set with a sensitivity and specificity of 95% for each mutation. This demonstrates that quantum cascade laser infrared imaging can be used to analyze morphologic differences as well as molecular changes. Therefore, this single, one-step measurement provides comprehensive diagnostics of lung cancer histology types and most frequent mutations.
Collapse
Affiliation(s)
- Nina Goertzen
- Center for Protein Diagnostics, Biospectroscopy, Germany; Department of Biophysics, Faculty of Biology and Biotechnology, Ruhr University Bochum, Bochum, Germany
| | | | - Jana Fassunke
- Institut für Pathologie, Universitätsklinikum Köln, Germany
| | - Thomas Brüning
- Institute for Prevention and Occupational Medicine of the German Social Accident Insurance, Institute of the Ruhr University Bochum, Bochum, Germany
| | - Yon-Dschun Ko
- Department of Internal Medicine, Johanniter-Kliniken Bonn GmbH, Johanniter Krankenhaus, Bonn, Germany
| | - Joachim Schmidt
- Lung Cancer Center Bonn, Department of Thoracic Surgery, Helios Klinikum Bonn/Rhein-Sieg and Department of Surgery, Division of Thoracic Surgery, Universitätsklinikum Bonn, Germany
| | - Frederik Großerueschkamp
- Center for Protein Diagnostics, Biospectroscopy, Germany; Department of Biophysics, Faculty of Biology and Biotechnology, Ruhr University Bochum, Bochum, Germany
| | | | - Klaus Gerwert
- Center for Protein Diagnostics, Biospectroscopy, Germany; Department of Biophysics, Faculty of Biology and Biotechnology, Ruhr University Bochum, Bochum, Germany.
| |
Collapse
|
11
|
Enhancing hyperspectral imaging. NAT MACH INTELL 2021. [DOI: 10.1038/s42256-021-00336-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
12
|
Infrared Spectral Microscopy: A Primer for the Interventional Radiologist. J Vasc Interv Radiol 2021; 32:878-881.e1. [PMID: 33771714 DOI: 10.1016/j.jvir.2021.03.524] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2020] [Revised: 03/03/2021] [Accepted: 03/14/2021] [Indexed: 11/20/2022] Open
|
13
|
A versatile deep learning architecture for classification and label-free prediction of hyperspectral images. NAT MACH INTELL 2021; 3:306-315. [PMID: 34676358 PMCID: PMC8528004 DOI: 10.1038/s42256-021-00309-y] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.
Collapse
|
14
|
Tiwari S, Kajdacsy-Balla A, Whiteley J, Cheng G, Hewitt SM, Bhargava R. INFORM: INFrared-based ORganizational Measurements of tumor and its microenvironment to predict patient survival. SCIENCE ADVANCES 2021; 7:7/6/eabb8292. [PMID: 33536203 PMCID: PMC7857685 DOI: 10.1126/sciadv.abb8292] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2020] [Accepted: 12/11/2020] [Indexed: 05/14/2023]
Abstract
The structure and organization of a tumor and its microenvironment are often associated with cancer outcomes due to spatially varying molecular composition and signaling. A persistent challenge is to use this physical and chemical spatial organization to understand cancer progression. Here, we present a high-definition infrared imaging-based organizational measurement framework (INFORM) that leverages intrinsic chemical contrast of tissue to label unique components of the tumor and its microenvironment. Using objective and automated computational methods, further, we determine organization characteristics important for prediction. We show that the tumor spatial organization assessed with this framework is predictive of overall survival in colon cancer that adds to capability from clinical variables such as stage and grade, approximately doubling the risk of death in high-risk individuals. Our results open an all-digital avenue for measuring and studying the association between tumor spatial organization and disease progression.
Collapse
Affiliation(s)
- Saumya Tiwari
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA
| | - Andre Kajdacsy-Balla
- Department of Pathology, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Joshua Whiteley
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA
| | | | - Stephen M Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Rohit Bhargava
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA.
- Departments of Electrical and Computer Engineering, Mechanical Science and Engineering, Chemical and Biomolecular Engineering and Chemistry, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA
- Cancer Center at Illinois, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA
| |
Collapse
|
15
|
Horisaki R, Nishizaki Y, Kitaguchi K, Saito M, Tanida J. Three-dimensional deeply generated holography [Invited]. APPLIED OPTICS 2021; 60:A323-A328. [PMID: 33690416 DOI: 10.1364/ao.404151] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 10/27/2020] [Indexed: 05/28/2023]
Abstract
In this paper, we present a noniterative method for 3D computer-generated holography based on deep learning. A convolutional neural network is adapted for directly generating a hologram to reproduce a 3D intensity pattern in a given class. We experimentally demonstrated the proposed method with optical reproductions of multiple layers based on phase-only Fourier holography. Our method is noniterative, but it achieves a reproduction quality comparable with that of iterative methods for a given class.
Collapse
|
16
|
Phal Y, Yeh K, Bhargava R. Concurrent Vibrational Circular Dichroism Measurements with Infrared Spectroscopic Imaging. Anal Chem 2021; 93:1294-1303. [PMID: 33320538 PMCID: PMC9993326 DOI: 10.1021/acs.analchem.0c00323] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
Abstract
Vibrational circular dichroism (VCD) spectroscopy has emerged as a powerful platform to quantify chirality, a vital biological property that performs a pivotal role in the metabolism of life organisms. With a photoelastic modulator (PEM) integrated into an infrared spectrometer, the differential response of a sample to the direction of circularly polarized light can be used to infer conformation handedness. However, these optical components inherently exhibit chromatic behavior and are typically optimized at discrete spectral frequencies. Advancements of discrete frequency infrared (DFIR) spectroscopic microscopes in spectral image quality and data throughput are promising for use toward analytical VCD measurements. Utilizing the PEM advantages incorporated into a custom-built QCL microscope, we demonstrate a point scanning VCD instrument capable of acquiring spectra rapidly across all fingerprint region wavelengths in transmission configuration. Moreover, for the first time, we also demonstrate the VCD imaging performance of our instrument for site-specific chirality mapping of biological tissue samples. This study offers some insight into future possibilities of examining small, localized changes in tissue that have major implications for systemic diseases and their progression, while also laying the groundwork for additional modeling and validation in advancing the capability of VCD spectroscopy and imaging.
Collapse
|
17
|
Tuck M, Blanc L, Touti R, Patterson NH, Van Nuffel S, Villette S, Taveau JC, Römpp A, Brunelle A, Lecomte S, Desbenoit N. Multimodal Imaging Based on Vibrational Spectroscopies and Mass Spectrometry Imaging Applied to Biological Tissue: A Multiscale and Multiomics Review. Anal Chem 2020; 93:445-477. [PMID: 33253546 DOI: 10.1021/acs.analchem.0c04595] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Affiliation(s)
- Michael Tuck
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| | - Landry Blanc
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| | - Rita Touti
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| | - Nathan Heath Patterson
- Mass Spectrometry Research Center, Department of Biochemistry, Vanderbilt University, Nashville, Tennessee 37232-8575, United States
| | - Sebastiaan Van Nuffel
- Materials Research Institute, The Pennsylvania State University, University Park, Pennsylvania 16802, United States
| | - Sandrine Villette
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| | - Jean-Christophe Taveau
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| | - Andreas Römpp
- Bioanalytical Sciences and Food Analysis, University of Bayreuth, Universitätsstraße 30, 95440 Bayreuth, Germany
| | - Alain Brunelle
- Laboratoire d'Archéologie Moléculaire et Structurale, LAMS UMR 8220, CNRS, Sorbonne Université, 4 Place Jussieu, 75005 Paris, France
| | - Sophie Lecomte
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| | - Nicolas Desbenoit
- Institut de Chimie & Biologie des Membranes & des Nano-objets, CBMN UMR 5248, CNRS, Université de Bordeaux, 1 Allée Geoffroy Saint-Hilaire, 33600 Pessac, France
| |
Collapse
|
18
|
Rivenson Y, de Haan K, Wallace WD, Ozcan A. Emerging Advances to Transform Histopathology Using Virtual Staining. BME FRONTIERS 2020; 2020:9647163. [PMID: 37849966 PMCID: PMC10521663 DOI: 10.34133/2020/9647163] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 07/28/2020] [Indexed: 10/19/2023] Open
Abstract
In an age where digitization is widespread in clinical and preclinical workflows, pathology is still predominantly practiced by microscopic evaluation of stained tissue specimens affixed on glass slides. Over the last decade, new high throughput digital scanning microscopes have ushered in the era of digital pathology that, along with recent advances in machine vision, have opened up new possibilities for Computer-Aided-Diagnoses. Despite these advances, the high infrastructural costs related to digital pathology and the perception that the digitization process is an additional and nondirectly reimbursable step have challenged its widespread adoption. Here, we discuss how emerging virtual staining technologies and machine learning can help to disrupt the standard histopathology workflow and create new avenues for the diagnostic paradigm that will benefit patients and healthcare systems alike via digital pathology.
Collapse
Affiliation(s)
- Yair Rivenson
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, USA
- Bioengineering Department, University of California, Los Angeles, CA, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Kevin de Haan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, USA
- Bioengineering Department, University of California, Los Angeles, CA, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - W. Dean Wallace
- Department of Pathology and Laboratory Medicine, Keck School of Medicine of USC, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, USA
- Bioengineering Department, University of California, Los Angeles, CA, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
- Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles, CA, USA
| |
Collapse
|
19
|
Suryadevara V, Nazeer SS, Sreedhar H, Adelaja O, Kajdacsy-Balla A, Natarajan V, Walsh MJ. Infrared spectral microscopy as a tool to monitor lung fibrosis development in a model system. BIOMEDICAL OPTICS EXPRESS 2020; 11:3996-4007. [PMID: 33014581 PMCID: PMC7510888 DOI: 10.1364/boe.394730] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/09/2020] [Revised: 06/17/2020] [Accepted: 06/18/2020] [Indexed: 06/11/2023]
Abstract
Tissue fibrosis is a progressive and destructive disease process that can occur in many different organs including the liver, kidney, skin, and lungs. Fibrosis is typically initiated by inflammation as a result of chronic insults such as infection, chemicals and autoimmune diseases. Current approaches to examine organ fibrosis are limited to radiological and histological analyses. Infrared spectroscopic imaging offers a potential alternative approach to gain insight into biochemical changes associated with fibrosis progression. In this study, we demonstrate that IR imaging of a mouse model of pulmonary fibrosis can identify biochemical changes observed with fibrosis progression and the beginning of resolution using K-means analysis, spectral ratios and multivariate data analysis. This study demonstrates that IR imaging may be a useful approach to understand the biochemical events associated with fibrosis initiation, progression and resolution for both the clinical setting and for assessing novel anti-fibrotic drugs in a model system.
Collapse
Affiliation(s)
- Vidyani Suryadevara
- Department of Bioengineering, University of Illinois at Chicago, Chicago, IL 60607, USA
| | - Shaiju S. Nazeer
- Department of Pathology, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Hari Sreedhar
- Department of Pathology, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Oluwatobi Adelaja
- Department of Pathology, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - André Kajdacsy-Balla
- Department of Pathology, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Viswanathan Natarajan
- Department of Pharmacology, University of Illinois at Chicago, Chicago, IL 60612, USA
- Contributed equally as senior co-authors
| | - Michael J. Walsh
- Department of Bioengineering, University of Illinois at Chicago, Chicago, IL 60607, USA
- Department of Pathology, University of Illinois at Chicago, Chicago, IL 60612, USA
- Contributed equally as senior co-authors
| |
Collapse
|
20
|
Kallenbach-Thieltges A, Großerueschkamp F, Jütte H, Kuepper C, Reinacher-Schick A, Tannapfel A, Gerwert K. Label-free, automated classification of microsatellite status in colorectal cancer by infrared imaging. Sci Rep 2020; 10:10161. [PMID: 32576892 PMCID: PMC7311536 DOI: 10.1038/s41598-020-67052-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Accepted: 05/21/2020] [Indexed: 12/12/2022] Open
Abstract
Challenging histopathological diagnostics in cancer include microsatellite instability-high (MSI-H) colorectal cancer (CRC), which occurs in 15% of early-stage CRC and is caused by a deficiency in the mismatch repair system. The diagnosis of MSI-H cannot be reliably achieved by visual inspection of a hematoxylin and eosin stained thin section alone, but additionally requires subsequent molecular analysis. Time- and sample-intensive immunohistochemistry with subsequent fragment length analysis is used. The aim of the presented feasibility study is to test the ability of quantum cascade laser (QCL)-based infrared (IR) imaging as an alternative diagnostic tool for MSI-H in CRC. We analyzed samples from 100 patients with sporadic CRC UICC stage II and III. Forty samples were used to develop the random forest classifier and 60 samples to verify the results on an independent blinded dataset. Specifically, 100% sensitivity and 93% specificity were achieved based on the independent 30 MSI-H- and 30 microsatellite stable (MSS)-patient validation cohort. This showed that QCL-based IR imaging is able to distinguish between MSI-H and MSS for sporadic CRC - a question that goes beyond morphological features - based on the use of spatially resolved infrared spectra used as biomolecular fingerprints.
Collapse
Affiliation(s)
- Angela Kallenbach-Thieltges
- Ruhr University Bochum, Center for Protein Diagnostics (ProDi), Biospectroscopy, Bochum, Germany.,Ruhr University Bochum, Faculty of Biology and Biotechnology, Department of Biophysics, Bochum, Germany
| | - Frederik Großerueschkamp
- Ruhr University Bochum, Center for Protein Diagnostics (ProDi), Biospectroscopy, Bochum, Germany.,Ruhr University Bochum, Faculty of Biology and Biotechnology, Department of Biophysics, Bochum, Germany
| | - Hendrik Jütte
- Institute of Pathology, Ruhr University Bochum, Bochum, Germany
| | - Claus Kuepper
- Ruhr University Bochum, Center for Protein Diagnostics (ProDi), Biospectroscopy, Bochum, Germany.,Ruhr University Bochum, Faculty of Biology and Biotechnology, Department of Biophysics, Bochum, Germany
| | - Anke Reinacher-Schick
- Department of Hematology, Oncology and Palliative Care, St. Josef Hospital, Ruhr University Bochum, Bochum, Germany
| | | | - Klaus Gerwert
- Ruhr University Bochum, Center for Protein Diagnostics (ProDi), Biospectroscopy, Bochum, Germany. .,Ruhr University Bochum, Faculty of Biology and Biotechnology, Department of Biophysics, Bochum, Germany.
| |
Collapse
|
21
|
Abstract
Hematological analysis, via a complete blood count (CBC) and microscopy, is critical for screening, diagnosing, and monitoring blood conditions and diseases but requires complex equipment, multiple chemical reagents, laborious system calibration and procedures, and highly trained personnel for operation. Here we introduce a hematological assay based on label-free molecular imaging with deep-ultraviolet microscopy that can provide fast quantitative information of key hematological parameters to facilitate and improve hematological analysis. We demonstrate that this label-free approach yields 1) a quantitative five-part white blood cell differential, 2) quantitative red blood cell and hemoglobin characterization, 3) clear identification of platelets, and 4) detailed subcellular morphology. Analysis of tens of thousands of live cells is achieved in minutes without any sample preparation. Finally, we introduce a pseudocolorization scheme that accurately recapitulates the appearance of cells under conventional staining protocols for microscopic analysis of blood smears and bone marrow aspirates. Diagnostic efficacy is evaluated by a panel of hematologists performing a blind analysis of blood smears from healthy donors and thrombocytopenic and sickle cell disease patients. This work has significant implications toward simplifying and improving CBC and blood smear analysis, which is currently performed manually via bright-field microscopy, and toward the development of a low-cost, easy-to-use, and fast hematological analyzer as a point-of-care device and for low-resource settings.
Collapse
|
22
|
Horisaki R, Okamoto Y, Tanida J. Deeply coded aperture for lensless imaging. OPTICS LETTERS 2020; 45:3131-3134. [PMID: 32479477 DOI: 10.1364/ol.390810] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/19/2020] [Accepted: 05/03/2020] [Indexed: 06/11/2023]
Abstract
In this Letter, we present a method for jointly designing a coded aperture and a convolutional neural network for reconstructing an object from a single-shot lensless measurement. The coded aperture and the reconstruction network are connected with a deep learning framework in which the coded aperture is placed as a first convolutional layer. Our co-optimization method was experimentally demonstrated with a fully convolutional network, and its performance was compared to a coded aperture with a modified uniformly redundant array.
Collapse
|
23
|
Zhang Y, de Haan K, Rivenson Y, Li J, Delis A, Ozcan A. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. LIGHT, SCIENCE & APPLICATIONS 2020; 9:78. [PMID: 32411363 PMCID: PMC7203145 DOI: 10.1038/s41377-020-0315-y] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2020] [Revised: 04/14/2020] [Accepted: 04/15/2020] [Indexed: 05/16/2023]
Abstract
Histological staining is a vital step in diagnosing various diseases and has been used for more than a century to provide contrast in tissue sections, rendering the tissue constituents visible for microscopic analysis by medical experts. However, this process is time consuming, labour intensive, expensive and destructive to the specimen. Recently, the ability to virtually stain unlabelled tissue sections, entirely avoiding the histochemical staining step, has been demonstrated using tissue-stain-specific deep neural networks. Here, we present a new deep-learning-based framework that generates virtually stained images using label-free tissue images, in which different stains are merged following a micro-structure map defined by the user. This approach uses a single deep neural network that receives two different sources of information as its input: (1) autofluorescence images of the label-free tissue sample and (2) a "digital staining matrix", which represents the desired microscopic map of the different stains to be virtually generated in the same tissue section. This digital staining matrix is also used to virtually blend existing stains, digitally synthesizing new histological stains. We trained and blindly tested this virtual-staining network using unlabelled kidney tissue sections to generate micro-structured combinations of haematoxylin and eosin (H&E), Jones' silver stain, and Masson's trichrome stain. Using a single network, this approach multiplexes the virtual staining of label-free tissue images with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created in the same tissue cross section, which is currently not feasible with standard histochemical staining methods.
Collapse
Affiliation(s)
- Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Bioengineering Department, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095 USA
| | - Kevin de Haan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Bioengineering Department, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095 USA
| | - Yair Rivenson
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Bioengineering Department, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095 USA
| | - Jingxi Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Bioengineering Department, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095 USA
| | - Apostolos Delis
- Department of Computer Science, University of California, Los Angeles, CA 90095 USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Bioengineering Department, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095 USA
- Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles, CA 90095 USA
| |
Collapse
|
24
|
Infrared Spectroscopic Imaging Visualizes a Prognostic Extracellular Matrix-Related Signature in Breast Cancer. Sci Rep 2020; 10:5442. [PMID: 32214177 PMCID: PMC7096505 DOI: 10.1038/s41598-020-62403-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2019] [Accepted: 02/27/2020] [Indexed: 12/22/2022] Open
Abstract
Molecular analysis techniques such as gene expression analysis and proteomics have contributed greatly to our understanding of cancer heterogeneity. In prior studies, gene expression analysis was shown to stratify patient outcome on the basis of tumor-microenvironment associated genes. A specific gene expression profile, referred to as ECM3 (Extracellular Matrix Cluster 3), indicated poorer survival in patients with grade III tumors. In this work, we aimed to visualize the downstream effects of this gene expression profile onto the tissue, thus providing a spatial context to altered gene expression profiles. Using infrared spectroscopic imaging, we identified spectral patterns specific to the ECM3 gene expression profile, achieving a high spectral classification performance of 0.87 as measured by the area under the curve of the receiver operating characteristic curve. On a patient level, we correctly identified 20 out of 22 ECM3 group patients and 19 out of 20 non-ECM3 group patients by using this spectroscopic imaging-based classifier. By comparing pixels that were identified as ECM3 or non-ECM3 with H&E and IHC images, we were also able to observe an association between tissue morphology and the gene expression clusters, showing the ability of our method to capture broad outcome associated features from infrared images.
Collapse
|
25
|
Abstract
Optical microscopy for biomedical samples requires expertise in staining to visualize structure and composition. Midinfrared (mid-IR) spectroscopic imaging offers label-free molecular recording and virtual staining by probing fundamental vibrational modes of molecular components. This quantitative signal can be combined with machine learning to enable microscopy in diverse fields from cancer diagnoses to forensics. However, absorption of IR light by common optical imaging components makes mid-IR light incompatible with modern optical microscopy and almost all biomedical research and clinical workflows. Here we conceptualize an IR-optical hybrid (IR-OH) approach that sensitively measures molecular composition based on an optical microscope with wide-field interferometric detection of absorption-induced sample expansion. We demonstrate that IR-OH exceeds state-of-the-art IR microscopy in coverage (10-fold), spatial resolution (fourfold), and spectral consistency (by mitigating the effects of scattering). The combined impact of these advances allows full slide infrared absorption images of unstained breast tissue sections on a visible microscope platform. We further show that automated histopathologic segmentation and generation of computationally stained (stainless) images is possible, resolving morphological features in both color and spatial detail comparable to current pathology protocols but without stains or human interpretation. IR-OH is compatible with clinical and research pathology practice and could make for a cost-effective alternative to conventional stain-based protocols for stainless, all-digital pathology.
Collapse
|
26
|
Deep Learning for Hyperspectral Image Analysis, Part II: Applications to Remote Sensing and Biomedicine. HYPERSPECTRAL IMAGE ANALYSIS 2020. [DOI: 10.1007/978-3-030-38617-7_4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
|
27
|
Abstract
Fourier transform-infrared spectroscopy (FT-IR) represents an attractive molecular diagnostic modality for translation to the clinic, where comprehensive chemical profiling of biological samples may revolutionize a myriad of pathways in clinical settings. Principally, FT-IR provides a rapid, cost-effective platform to obtain a molecular fingerprint of clinical samples based on vibrational transitions of chemical bonds upon interaction with infrared light. To date, considerable research activities have demonstrated competitive to superior performance of FT-IR strategies in comparison to conventional techniques, with particular promise for earlier, accessible disease diagnostics, thereby improving patient outcomes. However, amidst the changing healthcare landscape in times of aging populations and increased prevalence of cancer and chronic disease, routine adoption of FT-IR within clinical laboratories has remained elusive. Hence, this perspective shall outline the significant clinical potential of FT-IR diagnostics and subsequently address current barriers to translation from the perspective of all stakeholders, in the context of biofluid, histopathology, cytology, microbiology, and biomarker discovery frameworks. Thereafter, future perspectives of FT-IR for healthcare will be discussed, with consideration of recent technological advances that may facilitate future clinical translation.
Collapse
Affiliation(s)
- Duncan Finlayson
- Centre for Doctoral Training in Medical Devices and Health Technologies, Department of Biomedical Engineering , University of Strathclyde , Wolfson Centre, 106 Rottenrow , Glasgow G4 0NW , U.K.,WestCHEM , Department of Pure and Applied Chemistry , Technology and Innovation Centre, 99 George Street , Glasgow G1 1RD , U.K
| | - Christopher Rinaldi
- Centre for Doctoral Training in Medical Devices and Health Technologies, Department of Biomedical Engineering , University of Strathclyde , Wolfson Centre, 106 Rottenrow , Glasgow G4 0NW , U.K.,WestCHEM , Department of Pure and Applied Chemistry , Technology and Innovation Centre, 99 George Street , Glasgow G1 1RD , U.K
| | - Matthew J Baker
- WestCHEM , Department of Pure and Applied Chemistry , Technology and Innovation Centre, 99 George Street , Glasgow G1 1RD , U.K.,ClinSpec Diagnostics Ltd. , Technology and Innovation Centre, 99 George Street , Glasgow G11RD , U.K
| |
Collapse
|
28
|
Fast stimulated Raman and second harmonic generation imaging for intraoperative gastro-intestinal cancer detection. Sci Rep 2019; 9:10052. [PMID: 31296917 PMCID: PMC6624250 DOI: 10.1038/s41598-019-46489-x] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2019] [Accepted: 06/25/2019] [Indexed: 01/26/2023] Open
Abstract
Conventional haematoxylin, eosin and saffron (HES) histopathology, currently the 'gold-standard' for pathological diagnosis of cancer, requires extensive sample preparations that are achieved within time scales that are not compatible with intra-operative situations where quick decisions must be taken. Providing to pathologists a close to real-time technology revealing tissue structures at the cellular level with HES histologic quality would provide an invaluable tool for surgery guidance with evident clinical benefit. Here, we specifically develop a stimulated Raman imaging based framework that demonstrates gastro-intestinal (GI) cancer detection of unprocessed human surgical specimens. The generated stimulated Raman histology (SRH) images combine chemical and collagen information to mimic conventional HES histopathology staining. We report excellent agreements between SRH and HES images acquire on the same patients for healthy, pre-cancerous and cancerous colon and pancreas tissue sections. We also develop a novel fast SRH imaging modality that captures at the pixel level all the information necessary to provide instantaneous SRH images. These developments pave the way for instantaneous label free GI histology in an intra-operative context.
Collapse
|
29
|
Kumar N, Uppala P, Duddu K, Sreedhar H, Varma V, Guzman G, Walsh M, Sethi A. Hyperspectral Tissue Image Segmentation Using Semi-Supervised NMF and Hierarchical Clustering. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:1304-1313. [PMID: 30489266 PMCID: PMC6548328 DOI: 10.1109/tmi.2018.2883301] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
Hyperspectral imaging (HSI) of tissue samples in the mid-infrared (mid-IR) range provides spectro-chemical and tissue structure information at sub-cellular spatial resolution. Disease states can be directly assessed by analyzing the mid-IR spectra of different cell types (e.g., epithelial cells) and sub-cellular components (e.g., nuclei), provided that we can accurately classify the pixels belonging to these components. The challenge is to extract information from hundreds of noisy mid-IR bands at each pixel, where each band is not very informative in itself, making annotations of unstained tissue HSI images particularly tricky. Because the tissue structure is not necessarily identical between the two sections, only a few regions in unstained HSI image can be annotated with high confidence, even when serial (or adjacent) hematoxylin and eosin stained section is used as a visual guide. In order to completely use both labeled and unlabeled pixels in training images, we have developed an HSI pixel classification method that uses semi-supervised learning for both spectral dimension reduction and hierarchical pixel clustering. Compared to the supervised classifiers, the proposed method was able to account for the vast differences in the spectra of sub-cellular components of the same cell type and to achieve an F1 score of 71.18% on twofold cross-validation across 20 tissue images. To generate further interest in this promising modality, we have released our source code and also showed that disease classification is straightforward after HSI image segmentation.
Collapse
|
30
|
Lotfollahi M, Berisha S, Daeinejad D, Mayerich D. Digital Staining of High-Definition Fourier Transform Infrared (FT-IR) Images Using Deep Learning. APPLIED SPECTROSCOPY 2019; 73:556-564. [PMID: 30657342 PMCID: PMC6499711 DOI: 10.1177/0003702818819857] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosis and research. While these labels offer a high degree of specificity, throughput is limited by the need for multiple samples. Traditional histology stains, such as immunohistochemical labels, also rely only on protein expression and cannot quantify small molecules and metabolites that may aid in diagnosis. Finally, chemical stains and dyes permanently alter the tissue, making downstream analysis impossible. Fourier transform infrared (FT-IR) spectroscopic imaging has shown promise for label-free characterization of important tissue phenotypes and can bypass the need for many chemical labels. Fourier transform infrared classification commonly leverages supervised learning, requiring human annotation that is tedious and prone to errors. One alternative is digital staining, which leverages machine learning to map IR spectra to a corresponding chemical stain. This replaces human annotation with computer-aided alignment. Previous work relies on alignment of adjacent serial tissue sections. Since the tissue samples are not identical at the cellular level, this technique cannot be applied to high-definition FT-IR images. In this paper, we demonstrate that cellular-level mapping can be accomplished using identical samples for both FT-IR and chemical labels. In addition, higher-resolution results can be achieved using a deep convolutional neural network that integrates spatial and spectral features.
Collapse
Affiliation(s)
- Mahsa Lotfollahi
- Department of Electrical and Computer Engineering, University of Houston
| | - Sebastian Berisha
- Department of Electrical and Computer Engineering, University of Houston
| | - Davar Daeinejad
- Department of Electrical and Computer Engineering, University of Houston
| | - David Mayerich
- Department of Electrical and Computer Engineering, University of Houston
| |
Collapse
|
31
|
Berisha S, Lotfollahi M, Jahanipour J, Gurcan I, Walsh M, Bhargava R, Van Nguyen H, Mayerich D. Deep learning for FTIR histology: leveraging spatial and spectral features with convolutional neural networks. Analyst 2019; 144:1642-1653. [PMID: 30644947 DOI: 10.1039/c8an01495g] [Citation(s) in RCA: 45] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
Current methods for cancer detection rely on tissue biopsy, chemical labeling/staining, and examination of the tissue by a pathologist. Though these methods continue to remain the gold standard, they are non-quantitative and susceptible to human error. Fourier transform infrared (FTIR) spectroscopic imaging has shown potential as a quantitative alternative to traditional histology. However, identification of histological components requires reliable classification based on molecular spectra, which are susceptible to artifacts introduced by noise and scattering. Several tissue types, particularly in heterogeneous tissue regions, tend to confound traditional classification methods. Convolutional neural networks (CNNs) are the current state-of-the-art in image classification, providing the ability to learn spatial characteristics of images. In this paper, we demonstrate that CNNs with architectures designed to process both spectral and spatial information can significantly improve classifier performance over per-pixel spectral classification. We report classification results after applying CNNs to data from tissue microarrays (TMAs) to identify six major cellular and acellular constituents of tissue, namely adipocytes, blood, collagen, epithelium, necrosis, and myofibroblasts. Experimental results show that the use of spatial information in addition to the spectral information brings significant improvements in the classifier performance and allows classification of cellular subtypes, such as adipocytes, that exhibit minimal chemical information but have distinct spatial characteristics. This work demonstrates the application and efficiency of deep learning algorithms in improving the diagnostic techniques in clinical and research activities related to cancer.
Collapse
Affiliation(s)
- Sebastian Berisha
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA.
| | | | | | | | | | | | | | | |
Collapse
|
32
|
Berisha S, Chang S, Saki S, Daeinejad D, He Z, Mankar R, Mayerich D. SIproc: an open-source biomedical data processing platform for large hyperspectral images. Analyst 2018; 142:1350-1357. [PMID: 27924319 DOI: 10.1039/c6an02082h] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit designed to perform out-of-core processing of hyperspectral images. By taking advantage of graphical processing unit (GPU) computing combined with adaptive data streaming, our software alleviates common workstation memory limitations while achieving better performance than existing applications.
Collapse
Affiliation(s)
- Sebastian Berisha
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA.
| | | | | | | | | | | | | |
Collapse
|
33
|
Neumann EK, Comi TJ, Spegazzini N, Mitchell JW, Rubakhin SS, Gillette MU, Bhargava R, Sweedler JV. Multimodal Chemical Analysis of the Brain by High Mass Resolution Mass Spectrometry and Infrared Spectroscopic Imaging. Anal Chem 2018; 90:11572-11580. [PMID: 30188687 DOI: 10.1021/acs.analchem.8b02913] [Citation(s) in RCA: 50] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
The brain functions through chemical interactions between many different cell types, including neurons and glia. Acquiring comprehensive information on complex, heterogeneous systems requires multiple analytical tools, each of which have unique chemical specificity and spatial resolution. Multimodal imaging generates complementary chemical information via spatially localized molecular maps, ideally from the same sample, but requires method enhancements that span from data acquisition to interpretation. We devised a protocol for performing matrix-assisted laser desorption/ionization (MALDI)-Fourier transform ion cyclotron resonance-mass spectrometry imaging (MSI), followed by infrared (IR) spectroscopic imaging on the same specimen. Multimodal measurements from the same tissue provide precise spatial alignment between modalities, enabling more advanced image processing such as image fusion and sharpening. Performing MSI first produces higher quality data from each technique compared to performing IR imaging before MSI. The difference is likely due to fixing the tissue section during MALDI matrix removal, thereby preventing analyte degradation occurring during IR imaging from an unfixed specimen. Leveraging the unique capabilities of each modality, we utilized pan sharpening of MS (mass spectrometry) ion images with selected bands from IR spectroscopy and midlevel data fusion. In comparison to sharpening with histological images, pan sharpening can employ a plethora of IR bands, producing sharpened MS images while retaining the fidelity of the initial ion images. Using Laplacian pyramid sharpening, we determine the localization of several lipids present within the hippocampus with high mass accuracy at 5 μm pixel widths. Further, through midlevel data fusion of the imaging data sets combined with k-means clustering, the combined data set discriminates between additional anatomical structures unrecognized by the individual imaging approaches. Significant differences between molecular ion abundances are detected between relevant structures within the hippocampus, such as the CA1 and CA3 regions. Our methodology provides high quality multiplex and multimodal chemical imaging of the same tissue sample, enabling more advanced data processing and analysis routines.
Collapse
|
34
|
Pahlow S, Weber K, Popp J, Wood BR, Kochan K, Rüther A, Perez-Guaita D, Heraud P, Stone N, Dudgeon A, Gardner B, Reddy R, Mayerich D, Bhargava R. Application of Vibrational Spectroscopy and Imaging to Point-of-Care Medicine: A Review. APPLIED SPECTROSCOPY 2018; 72:52-84. [PMID: 30265133 PMCID: PMC6524782 DOI: 10.1177/0003702818791939] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Affiliation(s)
- Susanne Pahlow
- Friedrich Schiller University Jena, Institute of Physical Chemistry and Abbe Center of Photonics, Jena, Germany
- InfectoGnostics Research Campus Jena, Centre for Applied Research, Jena, Germany
| | - Karina Weber
- Friedrich Schiller University Jena, Institute of Physical Chemistry and Abbe Center of Photonics, Jena, Germany
- InfectoGnostics Research Campus Jena, Centre for Applied Research, Jena, Germany
- Leibniz Institute of Photonic Technology-Leibniz Health Technologies, Jena, Germany
| | - Jürgen Popp
- Friedrich Schiller University Jena, Institute of Physical Chemistry and Abbe Center of Photonics, Jena, Germany
- InfectoGnostics Research Campus Jena, Centre for Applied Research, Jena, Germany
- Leibniz Institute of Photonic Technology-Leibniz Health Technologies, Jena, Germany
| | - Bayden R. Wood
- Centre for Biospectroscopy, School of Chemistry, Monash University, Clayton, Victoria, Australia
| | - Kamila Kochan
- Centre for Biospectroscopy, School of Chemistry, Monash University, Clayton, Victoria, Australia
| | - Anja Rüther
- Centre for Biospectroscopy, School of Chemistry, Monash University, Clayton, Victoria, Australia
| | - David Perez-Guaita
- Centre for Biospectroscopy, School of Chemistry, Monash University, Clayton, Victoria, Australia
| | - Philip Heraud
- Centre for Biospectroscopy, School of Chemistry, Monash University, Clayton, Victoria, Australia
| | - Nick Stone
- University of Exeter, School of Physics and Astronomy, Exeter, UK
| | - Alex Dudgeon
- University of Exeter, School of Physics and Astronomy, Exeter, UK
| | - Ben Gardner
- University of Exeter, School of Physics and Astronomy, Exeter, UK
| | - Rohith Reddy
- Department of Electrical Engineering, University of Houston, Houston, USA
| | - David Mayerich
- Department of Electrical Engineering, University of Houston, Houston, USA
| | - Rohit Bhargava
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana Champaign, Departments of Mechanical Engineering, Bioengineering, Chemical and Biomolecular Engineering, Electrical and Computer Engineering, and Chemistry, University of Illinois at Urbana-Champaign, Urbana, USA
| |
Collapse
|
35
|
Simultaneous cancer and tumor microenvironment subtyping using confocal infrared microscopy for all-digital molecular histopathology. Proc Natl Acad Sci U S A 2018; 115:E5651-E5660. [PMID: 29866827 PMCID: PMC6016804 DOI: 10.1073/pnas.1719551115] [Citation(s) in RCA: 79] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023] Open
Abstract
Cancer alters both the morphological and the biochemical properties of multiple cell types in a tissue. Generally, the morphology of epithelial cells is practical for routine disease diagnoses. Here, infrared spectroscopic imaging biochemically characterizes breast cancer, both epithelial cells and the tumor-associated microenvironment. Unfortunately, conventional spectral analyses are slow. Hence, we designed and built a laser confocal microscope that demonstrates a high signal-to-noise ratio for confident diagnoses. The instrument cuts down imaging time from days to minutes, making the technology feasible for research and clinical translation. Finally, automated human breast cancer biopsy imaging is reported in ∼1 hour, paving the way for routine research into the total tumor (epithelial plus microenvironment) properties and rapid, label-free diagnoses. Histopathology based on spatial patterns of epithelial cells is the gold standard for clinical diagnoses and research in carcinomas; although known to be important, the tissue microenvironment is not readily used due to complex and subjective interpretation with existing tools. Here, we demonstrate accurate subtyping from molecular properties of epithelial cells using emerging high-definition Fourier transform infrared (HD FT-IR) spectroscopic imaging combined with machine learning algorithms. In addition to detecting four epithelial subtypes, we simultaneously delineate three stromal subtypes that characterize breast tumors. While FT-IR imaging data enable fully digital pathology with rich information content, the long spectral scanning times required for signal averaging and processing make the technology impractical for routine research or clinical use. Hence, we developed a confocal design in which refractive IR optics are designed to provide high-definition, rapid spatial scanning and discrete spectral tuning using a quantum cascade laser (QCL) source. This instrument provides simultaneously high resolving power (2-μm pixel size) and high signal-to-noise ratio (SNR) (>1,300), providing a speed increase of ∼50-fold for obtaining classified results compared with present imaging spectrometers. We demonstrate spectral fidelity and interinstrument operability of our developed instrument by accurate analysis of a 100-case breast tissue set that was analyzed in a day, considerably speeding research. Clinical breast biopsies typical of a patients’ caseload are analyzed in ∼1 hour. This study paves the way for comprehensive tumor-microenvironment analyses in feasible time periods, presenting a critical step in practical label-free molecular histopathology.
Collapse
|
36
|
Mankar R, Walsh MJ, Bhargava R, Prasad S, Mayerich D. Selecting optimal features from Fourier transform infrared spectroscopy for discrete-frequency imaging. Analyst 2018; 143:1147-1156. [PMID: 29404544 PMCID: PMC5860915 DOI: 10.1039/c7an01888f] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Tissue histology utilizing chemical and immunohistochemical labels plays an important role in biomedicine and disease diagnosis. Recent research suggests that mid-infrared (IR) spectroscopic imaging may augment histology by providing quantitative molecular information. One of the major barriers to this approach is long acquisition time using Fourier-transform infrared (FTIR) spectroscopy. Recent advances in discrete frequency sources, particularly quantum cascade lasers (QCLs), may mitigate this problem by allowing selective sampling of the absorption spectrum. However, DFIR imaging only provides a significant advantage when the number of spectral samples is minimized, requiring a priori knowledge of important spectral features. In this paper, we demonstrate the use of a GPU-based genetic algorithm (GA) using linear discriminant analysis (LDA) for DFIR feature selection. Our proposed method relies on pre-acquired broadband FTIR images for feature selection. Based on user-selected criteria for classification accuracy, our algorithm provides a minimal set of features that can be used with DFIR in a time-frame more practical for clinical diagnosis.
Collapse
Affiliation(s)
- Rupali Mankar
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA.
| | | | | | | | | |
Collapse
|
37
|
Wrobel TP, Bhargava R. Infrared Spectroscopic Imaging Advances as an Analytical Technology for Biomedical Sciences. Anal Chem 2018; 90:1444-1463. [PMID: 29281255 PMCID: PMC6421863 DOI: 10.1021/acs.analchem.7b05330] [Citation(s) in RCA: 63] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Affiliation(s)
- Tomasz P. Wrobel
- Beckman Institute for Advanced Science and Technology, Urbana, Illinois 61801, United States
| | - Rohit Bhargava
- Beckman Institute for Advanced Science and Technology, Urbana, Illinois 61801, United States
- Departments of Bioengineering, Electrical and Computer Engineering, Mechanical Science and Engineering, Chemical and Biomolecular Engineering, and Chemistry, University of Illinois at Urbana−Champaign, Urbana, Illinois 61801, United States
| |
Collapse
|
38
|
Singh DK, Gholamalamdari O, Jadaliha M, Ling Li X, Lin YC, Zhang Y, Guang S, Hashemikhabir S, Tiwari S, Zhu YJ, Khan A, Thomas A, Chakraborty A, Macias V, Balla AK, Bhargava R, Janga SC, Ma J, Prasanth SG, Lal A, Prasanth KV. PSIP1/p75 promotes tumorigenicity in breast cancer cells by promoting the transcription of cell cycle genes. Carcinogenesis 2017. [PMID: 28633434 DOI: 10.1093/carcin/bgx062] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Breast cancer (BC) is a highly heterogeneous disease, both at the pathological and molecular level, and several chromatin-associated proteins play crucial roles in BC initiation and progression. Here, we demonstrate the role of PSIP1 (PC4 and SF2 interacting protein)/p75 (LEDGF) in BC progression. PSIP1/p75, previously identified as a chromatin-adaptor protein, is found to be upregulated in basal-like/triple negative breast cancer (TNBC) patient samples and cell lines. Immunohistochemistry in tissue arrays showed elevated levels of PSIP1 in metastatic invasive ductal carcinoma. Survival data analyses revealed that the levels of PSIP1 showed a negative association with TNBC patient survival. Depletion of PSIP1/p75 significantly reduced the tumorigenicity and metastatic properties of TNBC cell lines while its over-expression promoted tumorigenicity. Further, gene expression studies revealed that PSIP1 regulates the expression of genes controlling cell-cycle progression, cell migration and invasion. Finally, by interacting with RNA polymerase II, PSIP1/p75 facilitates the association of RNA pol II to the promoter of cell cycle genes and thereby regulates their transcription. Our findings demonstrate an important role of PSIP1/p75 in TNBC tumorigenicity by promoting the expression of genes that control the cell cycle and tumor metastasis.
Collapse
Affiliation(s)
- Deepak K Singh
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Omid Gholamalamdari
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Mahdieh Jadaliha
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Xiao Ling Li
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Yo-Chuen Lin
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Yang Zhang
- Department of Bioengineering, Beckman Institute of Advanced Science and Technology, UIUC, Urbana, IL 61801, USA
| | - Shuomeng Guang
- Department of Bioengineering, Beckman Institute of Advanced Science and Technology, UIUC, Urbana, IL 61801, USA
| | - Seyedsasan Hashemikhabir
- Department of Biohealth Informatics, School of Informatics and Computing, Indiana University Purdue University Indianapolis, Indianapolis, IN 46202,USA
| | - Saumya Tiwari
- Department of Bioengineering, Beckman Institute of Advanced Science and Technology, UIUC, Urbana, IL 61801, USA
| | - Yuelin J Zhu
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Abid Khan
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | | | - Arindam Chakraborty
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Virgilia Macias
- Department of Pathology, College of Medicine, University of Illinois at Chicago,Chicago, IL 60612, USA
| | - Andre K Balla
- Department of Pathology, College of Medicine, University of Illinois at Chicago,Chicago, IL 60612, USA
| | - Rohit Bhargava
- Department of Bioengineering, Beckman Institute of Advanced Science and Technology, UIUC, Urbana, IL 61801, USA.,Departments of Electrical and Computer Engineering, Mechanical Science and Engineering, Chemical and Biomolecular Engineering and Chemistry, UIUC, Urbana, IL, USA
| | - Sarath Chandra Janga
- Department of Biohealth Informatics, School of Informatics and Computing, Indiana University Purdue University Indianapolis, Indianapolis, IN 46202,USA
| | - Jian Ma
- Department of Bioengineering, Beckman Institute of Advanced Science and Technology, UIUC, Urbana, IL 61801, USA.,School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Supriya G Prasanth
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Ashish Lal
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| | - Kannanganattu V Prasanth
- Department of Cell and Developmental Biology, University of Illinois at Urbana-Champaign, IL 61801,USA, Genetics Branch, Center for Cancer Research, National Cancer Institute, Bethesda, MD 20892, USA
| |
Collapse
|
39
|
Bhargava R, Madabhushi A. Emerging Themes in Image Informatics and Molecular Analysis for Digital Pathology. Annu Rev Biomed Eng 2017; 18:387-412. [PMID: 27420575 DOI: 10.1146/annurev-bioeng-112415-114722] [Citation(s) in RCA: 86] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
Pathology is essential for research in disease and development, as well as for clinical decision making. For more than 100 years, pathology practice has involved analyzing images of stained, thin tissue sections by a trained human using an optical microscope. Technological advances are now driving major changes in this paradigm toward digital pathology (DP). The digital transformation of pathology goes beyond recording, archiving, and retrieving images, providing new computational tools to inform better decision making for precision medicine. First, we discuss some emerging innovations in both computational image analytics and imaging instrumentation in DP. Second, we discuss molecular contrast in pathology. Molecular DP has traditionally been an extension of pathology with molecularly specific dyes. Label-free, spectroscopic images are rapidly emerging as another important information source, and we describe the benefits and potential of this evolution. Third, we describe multimodal DP, which is enabled by computational algorithms and combines the best characteristics of structural and molecular pathology. Finally, we provide examples of application areas in telepathology, education, and precision medicine. We conclude by discussing challenges and emerging opportunities in this area.
Collapse
Affiliation(s)
- Rohit Bhargava
- Departments of Bioengineering, Chemical and Biomolecular Engineering, Electrical and Computer Engineering, Mechanical Science and Engineering, and Chemistry, and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801;
| | - Anant Madabhushi
- Center for Computational Imaging and Personalized Diagnostics; Departments of Biomedical Engineering, Urology, Pathology, Radiology, Radiation Oncology, General Medical Sciences, Electrical Engineering, and Computer Science; and Case Comprehensive Cancer Center, Case Western Reserve University, Cleveland, Ohio 44106;
| |
Collapse
|
40
|
Li Z, Aleshire K, Kuno M, Hartland GV. Super-Resolution Far-Field Infrared Imaging by Photothermal Heterodyne Imaging. J Phys Chem B 2017; 121:8838-8846. [DOI: 10.1021/acs.jpcb.7b06065] [Citation(s) in RCA: 81] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Zhongming Li
- Department of Chemistry and
Biochemistry, University of Notre Dame, Notre Dame, Indiana 46556, United States
| | - Kyle Aleshire
- Department of Chemistry and
Biochemistry, University of Notre Dame, Notre Dame, Indiana 46556, United States
| | - Masaru Kuno
- Department of Chemistry and
Biochemistry, University of Notre Dame, Notre Dame, Indiana 46556, United States
| | - Gregory V. Hartland
- Department of Chemistry and
Biochemistry, University of Notre Dame, Notre Dame, Indiana 46556, United States
| |
Collapse
|
41
|
Accurate and interpretable classification of microspectroscopy pixels using artificial neural networks. Med Image Anal 2017; 37:37-45. [PMID: 28131075 DOI: 10.1016/j.media.2017.01.001] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2016] [Revised: 12/19/2016] [Accepted: 01/05/2017] [Indexed: 11/20/2022]
Abstract
This paper addresses the problem of classifying materials from microspectroscopy at a pixel level. The challenges lie in identifying discriminatory spectral features and obtaining accurate and interpretable models relating spectra and class labels. We approach the problem by designing a supervised classifier from a tandem of Artificial Neural Network (ANN) models that identify relevant features in raw spectra and achieve high classification accuracy. The tandem of ANN models is meshed with classification rule extraction methods to lower the model complexity and to achieve interpretability of the resulting model. The contribution of the work is in designing each ANN model based on the microspectroscopy hypothesis about a discriminatory feature of a certain target class being composed of a linear combination of spectra. The novelty lies in meshing ANN and decision rule models into a tandem configuration to achieve accurate and interpretable classification results. The proposed method was evaluated using a set of broadband coherent anti-Stokes Raman scattering (BCARS) microscopy cell images (600 000 pixel-level spectra) and a reference four-class rule-based model previously created by biochemical experts. The generated classification rule-based model was on average 85% accurate measured by the DICE pixel label similarity metric, and on average 96% similar to the reference rules measured by the vector cosine metric.
Collapse
|
42
|
Tiwari S, Raman J, Reddy V, Ghetler A, Tella RP, Han Y, Moon CR, Hoke CD, Bhargava R. Towards Translation of Discrete Frequency Infrared Spectroscopic Imaging for Digital Histopathology of Clinical Biopsy Samples. Anal Chem 2016; 88:10183-10190. [PMID: 27626947 DOI: 10.1021/acs.analchem.6b02754] [Citation(s) in RCA: 38] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
Fourier transform infrared (FT-IR) spectroscopic imaging has been widely tested as a tool for stainless digital histology of biomedical specimens, including for the identification of infiltration and fibrosis in endomyocardial biopsy samples to assess transplant rejection. A major barrier in clinical translation has been the slow speed of imaging. To address this need, we tested and report here the viability of using high speed discrete frequency infrared (DFIR) imaging to obtain stain-free biochemical imaging in cardiovascular samples collected from patients. Images obtained by this method were classified with high accuracy by a Bayesian classification algorithm trained on FT-IR imaging data as well as on DFIR data. A single spectral feature correlated with instances of fibrosis, as identified by the pathologist, highlights the advantage of the DFIR imaging approach for rapid detection. The speed of digital pathologic recognition was at least 16 times faster than the fastest FT-IR imaging instrument. These results indicate that a fast, on-site identification of fibrosis using IR imaging has potential for real time assistance during surgeries. Further, the work describes development and applications of supervised classifiers on DFIR imaging data, comparing classifiers developed on FT-IR and DFIR imaging modalities and identifying specific spectral features for accurate identification of fibrosis. This addresses a topic of much debate on the use of training data and cross-modality validity of IR measurements. Together, the work is a step toward addressing a clinical diagnostic need at acquisition time scales that make IR imaging technology practical for medical use.
Collapse
Affiliation(s)
- Saumya Tiwari
- Department of Bioengineering and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign , Urbana, Illinois 61801, United States
| | - Jai Raman
- Knight Cardiovascular Institute, Oregon Health & Science University , 3181 SW Sam Jackson Park Road, Portland, Oregon 97201, United States
| | - Vijaya Reddy
- Department of Pathology, Rush University Medical Center , 1725 West Harrison Street, Chicago, Illinois 60612, United States
| | - Andrew Ghetler
- California Research Center, Spectroscopy and Vacuum Solutions Division, Agilent Technologies, Inc. , 5301 Stevens Creek Blvd., Santa Clara, California 95051 United States
| | - Richard P Tella
- California Research Center, Spectroscopy and Vacuum Solutions Division, Agilent Technologies, Inc. , 5301 Stevens Creek Blvd., Santa Clara, California 95051 United States
| | - Yang Han
- California Research Center, Spectroscopy and Vacuum Solutions Division, Agilent Technologies, Inc. , 5301 Stevens Creek Blvd., Santa Clara, California 95051 United States
| | - Christopher R Moon
- California Research Center, Spectroscopy and Vacuum Solutions Division, Agilent Technologies, Inc. , 5301 Stevens Creek Blvd., Santa Clara, California 95051 United States
| | - Charles D Hoke
- California Research Center, Spectroscopy and Vacuum Solutions Division, Agilent Technologies, Inc. , 5301 Stevens Creek Blvd., Santa Clara, California 95051 United States
| | - Rohit Bhargava
- Department of Bioengineering and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign , Urbana, Illinois 61801, United States.,Departments of Chemistry, Mechanical Science and Engineering, Chemical and Biomolecular Engineering, Electrical and Computer Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign , Urbana, Illinois 61801, United States
| |
Collapse
|
43
|
Pounder FN, Reddy RK, Bhargava R. Development of a practical spatial-spectral analysis protocol for breast histopathology using Fourier transform infrared spectroscopic imaging. Faraday Discuss 2016; 187:43-68. [PMID: 27095431 PMCID: PMC5515302 DOI: 10.1039/c5fd00199d] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
Breast cancer screening provides sensitive tumor identification, but low specificity implies that a vast majority of biopsies are not ultimately diagnosed as cancer. Automated techniques to evaluate biopsies can prevent errors, reduce pathologist workload and provide objective analysis. Fourier transform infrared (FT-IR) spectroscopic imaging provides both molecular signatures and spatial information that may be applicable for pathology. Here, we utilize both the spectral and spatial information to develop a combined classifier that provides rapid tissue assessment. First, we evaluated the potential of IR imaging to provide a diagnosis using spectral data alone. While highly accurate histologic [epithelium, stroma] recognition could be achieved, the same was not possible for disease [cancer, no-cancer] due to the diversity of spectral signals. Hence, we employed spatial data, developing and evaluating increasingly complex models, to detect cancers. Sub-mm tumors could be very confidently predicted as indicated by the quantitative measurement of accuracy via receiver operating characteristic (ROC) curve analyses. The developed protocol was validated with a small set and statistical performance used to develop a model that predicts study design for a large scale, definitive validation. The results of evaluation on different instruments, at higher noise levels, under a coarser spectral resolution and two sampling modes [transmission and transflection], indicate that the protocol is highly accurate under a variety of conditions. The study paves the way to validating IR imaging for rapid breast tumor detection, its statistical validation and potential directions for optimization of the speed and sampling for clinical deployment.
Collapse
Affiliation(s)
- F Nell Pounder
- Department of Bioengineering and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
| | - Rohith K Reddy
- Department of Bioengineering and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
| | - Rohit Bhargava
- Department of Bioengineering and Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA. and Departments of Chemical & Biomolecular Engineering, Electrical & Computer Engineering, Mechanical Science & Engineering and Chemistry, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| |
Collapse
|
44
|
Lee W, Kabir MM, Emmadi R, Toussaint KC. Third-harmonic generation imaging of breast tissue biopsies. J Microsc 2016; 264:175-181. [PMID: 27229847 DOI: 10.1111/jmi.12427] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2015] [Revised: 04/11/2016] [Accepted: 04/28/2016] [Indexed: 01/15/2023]
Abstract
We demonstrate for the first time the imaging of unstained breast tissue biopsies using third-harmonic generation (THG) microscopy. As a label-free imaging technique, THG microscopy is compared to phase contrast and polarized light microscopy which are standard imaging methods for breast tissues. A simple feature detection algorithm is applied to detect tumour-associated lymphocyte rich regions in unstained breast biopsy tissue and compared with corresponding regions identified by a pathologist from bright-field images of hematoxylin and eosin stained breast tissue. Our results suggest that THG imaging holds potential as a complementary technique for analysing breast tissue biopsies.
Collapse
Affiliation(s)
- Woowon Lee
- Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, U.S.A
| | - Mohammad M Kabir
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, U.S.A
| | - Rajyasree Emmadi
- Department of Pathology, University of Illinois at Chicago, Chicago, Illinois, U.S.A
| | - Kimani C Toussaint
- Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, U.S.A.. .,Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign and Bioengineering, Urbana, Illinois, U.S.A..
| |
Collapse
|
45
|
Misra SK, Mukherjee P, Ohoka A, Schwartz-Duval AS, Tiwari S, Bhargava R, Pan D. Vibrational spectroscopy and imaging for concurrent cellular trafficking of co-localized doxorubicin and deuterated phospholipid vesicles. NANOSCALE 2016; 8:2826-31. [PMID: 26763407 PMCID: PMC4868062 DOI: 10.1039/c5nr07975f] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
Simultaneous tracking of nanoparticles and encapsulated payload is of great importance and visualizing their activity is arduous. Here we use vibrational spectroscopy to study the in vitro tracking of co-localized lipid nanoparticles and encapsulated drug employing a model system derived from doxorubicin-encapsulated deuterated phospholipid (dodecyl phosphocholine-d38) single tailed phospholipid vesicles.
Collapse
Affiliation(s)
- S K Misra
- Departments of Bioengineering and Materials Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Carle Foundation Hospital, 502 N. Busey St., Urbana, IL 61801, USA.
| | - P Mukherjee
- Department of Bioengineering, Electrical and Computer Engineering, Chemical and Biomolecular Engineering, Chemistry, and Mechanical Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, 1304 W. Springfield Ave, Urbana, IL 61801, USA.
| | - A Ohoka
- Departments of Bioengineering and Materials Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Carle Foundation Hospital, 502 N. Busey St., Urbana, IL 61801, USA.
| | - A S Schwartz-Duval
- Departments of Bioengineering and Materials Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Carle Foundation Hospital, 502 N. Busey St., Urbana, IL 61801, USA.
| | - S Tiwari
- Department of Bioengineering, Electrical and Computer Engineering, Chemical and Biomolecular Engineering, Chemistry, and Mechanical Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, 1304 W. Springfield Ave, Urbana, IL 61801, USA.
| | - R Bhargava
- Department of Bioengineering, Electrical and Computer Engineering, Chemical and Biomolecular Engineering, Chemistry, and Mechanical Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, 1304 W. Springfield Ave, Urbana, IL 61801, USA.
| | - D Pan
- Departments of Bioengineering and Materials Science and Engineering, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Carle Foundation Hospital, 502 N. Busey St., Urbana, IL 61801, USA.
| |
Collapse
|
46
|
Varma VK, Kajdacsy-Balla A, Akkina SK, Setty S, Walsh MJ. A label-free approach by infrared spectroscopic imaging for interrogating the biochemistry of diabetic nephropathy progression. Kidney Int 2016; 89:1153-1159. [PMID: 26924056 DOI: 10.1016/j.kint.2015.11.027] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2015] [Revised: 11/16/2015] [Accepted: 11/19/2015] [Indexed: 10/22/2022]
Abstract
Routine histology, the current gold standard, involves staining for specific biomolecules. However, untapped biochemical information in tissue can be gathered using biochemical imaging. Infrared spectroscopy is an emerging modality that allows label-free chemical imaging to derive biochemical information (such as protein, lipids, DNA, collagen) from tissues. Here we employed this technology in order to better predict the development of diabetic nephropathy. Using human primary kidney biopsies or nephrectomies, we obtained tissue from 4 histologically normal kidneys, 4 histologically normal kidneys from diabetic subjects, and 5 kidneys with evidence of diabetic nephropathy. A biochemical signature of diabetic nephropathy was derived that enabled prediction of nephropathy based on the ratio of only 2 spectral frequencies. Nonetheless, using the entire spectrum of biochemical information, we were able to detect renal disease with near-perfect accuracy. Additionally, study of sequential protocol biopsies from 3 transplanted kidneys showed biochemical changes even prior to clinical manifestation of diabetic nephropathy. Thus, infrared imaging can identify critical biochemical alterations that precede morphologic changes, potentially allowing for earlier intervention.
Collapse
Affiliation(s)
- Vishal K Varma
- Department of Bioengineering, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Andre Kajdacsy-Balla
- Department of Pathology, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Sanjeev K Akkina
- Department of Nephrology, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Suman Setty
- Department of Pathology, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Michael J Walsh
- Department of Bioengineering, University of Illinois at Chicago, Chicago, Illinois, USA; Department of Pathology, University of Illinois at Chicago, Chicago, Illinois, USA.
| |
Collapse
|
47
|
Yang C, Niedieker D, Grosserüschkamp F, Horn M, Tannapfel A, Kallenbach-Thieltges A, Gerwert K, Mosig A. Fully automated registration of vibrational microspectroscopic images in histologically stained tissue sections. BMC Bioinformatics 2015; 16:396. [PMID: 26607812 PMCID: PMC4659215 DOI: 10.1186/s12859-015-0804-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2015] [Accepted: 10/29/2015] [Indexed: 01/01/2023] Open
Abstract
Background In recent years, hyperspectral microscopy techniques such as infrared or Raman microscopy have been applied successfully for diagnostic purposes. In many of the corresponding studies, it is common practice to measure one and the same sample under different types of microscopes. Any joint analysis of the two image modalities requires to overlay the images, so that identical positions in the sample are located at the same coordinate in both images. This step, commonly referred to as image registration, has typically been performed manually in the lack of established automated computational registration tools. Results We propose a corresponding registration algorithm that addresses this registration problem, and demonstrate the robustness of our approach in different constellations of microscopes. First, we deal with subregion registration of Fourier Transform Infrared (FTIR) microscopic images in whole-slide histopathological staining images. Second, we register FTIR imaged cores of tissue microarrays in their histopathologically stained counterparts, and finally perform registration of Coherent anti-Stokes Raman spectroscopic (CARS) images within histopathological staining images. Conclusions Our validation involves a large variety of samples obtained from colon, bladder, and lung tissue on three different types of microscopes, and demonstrates that our proposed method works fully automated and highly robust in different constellations of microscopes involving diverse types of tissue samples. Electronic supplementary material The online version of this article (doi:10.1186/s12859-015-0804-9) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Chen Yang
- Department of Biophysics, CAS-MPG Partner Institute and Key Laboratory for Computational Biology, 320 Yueyang Road, Shanghai, 200031, China.
| | - Daniel Niedieker
- Department of Biophysics, Ruhr-University Bochum, Universitätsstraße 150, Bochum, 44780, Germany.
| | - Frederik Grosserüschkamp
- Department of Biophysics, Ruhr-University Bochum, Universitätsstraße 150, Bochum, 44780, Germany.
| | - Melanie Horn
- Department of Biophysics, Ruhr-University Bochum, Universitätsstraße 150, Bochum, 44780, Germany.
| | - Andrea Tannapfel
- Institute of Pathology, Ruhr-University Bochum, Bochum, Germany, Bürkle-de-la-Camp-Platz 1, Bochum, 44789, Germany.
| | | | - Klaus Gerwert
- Department of Biophysics, CAS-MPG Partner Institute and Key Laboratory for Computational Biology, 320 Yueyang Road, Shanghai, 200031, China. .,Department of Biophysics, Ruhr-University Bochum, Universitätsstraße 150, Bochum, 44780, Germany.
| | - Axel Mosig
- Department of Biophysics, Ruhr-University Bochum, Universitätsstraße 150, Bochum, 44780, Germany.
| |
Collapse
|
48
|
Leslie LS, Wrobel TP, Mayerich D, Bindra S, Emmadi R, Bhargava R. High definition infrared spectroscopic imaging for lymph node histopathology. PLoS One 2015; 10:e0127238. [PMID: 26039216 PMCID: PMC4454651 DOI: 10.1371/journal.pone.0127238] [Citation(s) in RCA: 42] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2015] [Accepted: 04/14/2015] [Indexed: 11/19/2022] Open
Abstract
Chemical imaging is a rapidly emerging field in which molecular information within samples can be used to predict biological function and recognize disease without the use of stains or manual identification. In Fourier transform infrared (FT-IR) spectroscopic imaging, molecular absorption contrast provides a large signal relative to noise. Due to the long mid-IR wavelengths and sub-optimal instrument design, however, pixel sizes have historically been much larger than cells. This limits both the accuracy of the technique in identifying small regions, as well as the ability to visualize single cells. Here we obtain data with micron-sized sampling using a tabletop FT-IR instrument, and demonstrate that the high-definition (HD) data lead to accurate identification of multiple cells in lymph nodes that was not previously possible. Highly accurate recognition of eight distinct classes - naïve and memory B cells, T cells, erythrocytes, connective tissue, fibrovascular network, smooth muscle, and light and dark zone activated B cells was achieved in healthy, reactive, and malignant lymph node biopsies using a random forest classifier. The results demonstrate that cells currently identifiable only through immunohistochemical stains and cumbersome manual recognition of optical microscopy images can now be distinguished to a similar level through a single IR spectroscopic image from a lymph node biopsy.
Collapse
Affiliation(s)
- L. Suzanne Leslie
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| | - Tomasz P. Wrobel
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| | - David Mayerich
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, United States America
| | - Snehal Bindra
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| | - Rajyasree Emmadi
- Department of Pathology, University of Illinois at Chicago, Chicago, Illinois, United States of America
| | - Rohit Bhargava
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Chemical and Biomolecular Engineering, University of Illinois at Urbana-Champaign, Illinois, United States of America
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Illinois, United States of America
- Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Illinois, United States of America
- Department of Chemistry, University of Illinois at Urbana-Champaign, Illinois, United States of America
- * E-mail:
| |
Collapse
|
49
|
Tiwari S, Bhargava R. Extracting knowledge from chemical imaging data using computational algorithms for digital cancer diagnosis. THE YALE JOURNAL OF BIOLOGY AND MEDICINE 2015; 88:131-43. [PMID: 26029012 PMCID: PMC4445435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Fourier transform infrared (FTIR) spectroscopic imaging is an emerging microscopy modality for clinical histopathologic diagnoses as well as for biomedical research. Spectral data recorded in this modality are indicative of the underlying, spatially resolved biochemical composition but need computerized algorithms to digitally recognize and transform this information to a diagnostic tool to identify cancer or other physiologic conditions. Statistical pattern recognition forms the backbone of these recognition protocols and can be used for highly accurate results. Aided by biochemical correlations with normal and diseased states and the power of modern computer-aided pattern recognition, this approach is capable of combating many standing questions of traditional histology-based diagnosis models. For example, a simple diagnostic test can be developed to determine cell types in tissue. As a more advanced application, IR spectral data can be integrated with patient information to predict risk of cancer, providing a potential road to precision medicine and personalized care in cancer treatment. The IR imaging approach can be implemented to complement conventional diagnoses, as the samples remain unperturbed and are not destroyed. Despite high potential and utility of this approach, clinical implementation has not yet been achieved due to practical hurdles like speed of data acquisition and lack of optimized computational procedures for extracting clinically actionable information rapidly. The latter problem has been addressed by developing highly efficient ways to process IR imaging data but remains one that has considerable scope for progress. Here, we summarize the major issues and provide practical considerations in implementing a modified Bayesian classification protocol for digital molecular pathology. We hope to familiarize readers with analysis methods in IR imaging data and enable researchers to develop methods that can lead to the use of this promising technique for digital diagnosis of cancer.
Collapse
Affiliation(s)
- Saumya Tiwari
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana Champaign, Urbana, Illinois,Department of Bioengineering, University of Illinois at Urbana Champaign, Urbana, Illinois
| | - Rohit Bhargava
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana Champaign, Urbana, Illinois,Department of Bioengineering, University of Illinois at Urbana Champaign, Urbana, Illinois,Departments of Chemical & Biomolecular Engineering, Electrical & Computer Engineering, Mechanical Science & Engineering and Chemistry, University of Illinois at Urbana Champaign, Urbana, Illinois,To whom all correspondence should be addressed: Rohit Bhargava, Beckman Institute for Advanced Science and Technology, 405 N. Mathews Ave, Urbana, IL 61801; Tele: 217-265-6596; Fax: 217-265-0246;
| |
Collapse
|