1
|
Mubarak HK, Zhou X, Palsgrove D, Sumer BD, Chen AY, Fei B. An Ensemble Learning Method for Detection of Head and Neck Squamous Cell Carcinoma Using Polarized Hyperspectral Microscopic Imaging. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12933:129330P. [PMID: 38711533 PMCID: PMC11073817 DOI: 10.1117/12.3007869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.
Collapse
Affiliation(s)
- Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University, Atlanta, GA
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
2
|
Zhou X, Ma L, Mubarak HK, Palsgrove D, Sumer BD, Chen AY, Fei B. Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:016005. [PMID: 38239390 PMCID: PMC10795499 DOI: 10.1117/1.jbo.29.1.016005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 12/04/2023] [Accepted: 12/20/2023] [Indexed: 01/22/2024]
Abstract
Significance Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications. Aim The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides. Approach We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: S 0 , S 1 , S 2 , and S 3 . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors. Results By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides. Conclusions With the spatial and spectral information from the Stokes vector data cubes (S 0 , S 1 , S 2 , and S 3 ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Hasan K. Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Doreen Palsgrove
- The University of Texas Southwestern Medical Center, Department of Pathology, Dallas, Texas, United States
| | - Baran D. Sumer
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
3
|
Yang J, Xue Q, Li J, Han B, Wang Y, Bai H. Deep ultraviolet high-resolution microscopic hyperspectral imager and its biological tissue detection. APPLIED OPTICS 2023; 62:3310-3319. [PMID: 37132831 DOI: 10.1364/ao.485387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Ultraviolet (UV) hyperspectral imaging technology is commonly used in the field of atmospheric remote sensing. In recent years, some in-laboratory research has been carried out for substance detection and identification. In this paper, UV hyperspectral imaging technology is introduced into microscopy to better utilize the obvious absorption characteristics of components, such as proteins and nucleic acids in biological tissues in the ultraviolet band. A deep UV microscopic hyperspectral imager based on the Offner structure with F # 2.5, low spectral keystone and smile is designed and developed. A 0.68 numerical aperture microscope objective is designed. The spectral range of the system is from 200 nm to 430 nm; the spectral resolution is better than 0.5 nm; and the spatial resolution is better than 1.3 µm. The K562 cells can be distinguished by transmission spectrum of nucleus. The UV microscopic hyperspectral image of the unstained mouse liver slices showed similar results to the microscopic image after hematoxylin and eosin staining, which could help to simplify the pathological examination process. Both results show a great performance in spatial and spectral detecting capabilities of our instrument, which has the potential for biomedical research and diagnosis.
Collapse
|
4
|
Tran MH, Fei B. Compact and ultracompact spectral imagers: technology and applications in biomedical imaging. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:040901. [PMID: 37035031 PMCID: PMC10075274 DOI: 10.1117/1.jbo.28.4.040901] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 02/27/2023] [Indexed: 05/18/2023]
Abstract
Significance Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging. Aim With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications. Approach We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition. Results There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings. Conclusions Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.
Collapse
Affiliation(s)
- Minh H. Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- Address all correspondence to Baowei Fei,
| |
Collapse
|
5
|
Stergar J, Hren R, Milanič M. Design and Validation of a Custom-Made Hyperspectral Microscope Imaging System for Biomedical Applications. SENSORS (BASEL, SWITZERLAND) 2023; 23:2374. [PMID: 36904578 PMCID: PMC10007032 DOI: 10.3390/s23052374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 02/10/2023] [Accepted: 02/19/2023] [Indexed: 06/18/2023]
Abstract
Hyperspectral microscope imaging (HMI) is an emerging modality that integrates spatial information collected by standard laboratory microscopy and the spectral-based contrast obtained by hyperspectral imaging and may be instrumental in establishing novel quantitative diagnostic methodologies, particularly in histopathology. Further expansion of HMI capabilities hinges upon the modularity and versatility of systems and their proper standardization. In this report, we describe the design, calibration, characterization, and validation of the custom-made laboratory HMI system based on a Zeiss Axiotron fully motorized microscope and a custom-developed Czerny-Turner-type monochromator. For these important steps, we rely on a previously designed calibration protocol. Validation of the system demonstrates a performance comparable to classic spectrometry laboratory systems. We further demonstrate validation against a laboratory hyperspectral imaging system for macroscopic samples, enabling future comparison of spectral imaging results across length scales. An example of the utility of our custom-made HMI system on a standard hematoxylin and eosin-stained histology slide is also shown.
Collapse
Affiliation(s)
- Jošt Stergar
- Jožef Stefan Institute, Jamova Cesta 39, SI-1000 Ljubljana, Slovenia
- Faculty of Mathematics and Physics, University of Ljubljana, Jadranska ulica 19, SI-1000 Ljubljana, Slovenia
| | - Rok Hren
- Faculty of Mathematics and Physics, University of Ljubljana, Jadranska ulica 19, SI-1000 Ljubljana, Slovenia
| | - Matija Milanič
- Jožef Stefan Institute, Jamova Cesta 39, SI-1000 Ljubljana, Slovenia
- Faculty of Mathematics and Physics, University of Ljubljana, Jadranska ulica 19, SI-1000 Ljubljana, Slovenia
| |
Collapse
|
6
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Ortega S, Callicó GM, Medina EA, Brimhall BB, Whitted M, Fei B. Polarized Hyperspectral Microscopic Imaging for White Blood Cells on Wright-Stained Blood Smear Slides. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238207. [PMID: 38486823 PMCID: PMC10938460 DOI: 10.1117/12.2655708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
White blood cells, also called leukocytes, are hematopoietic cells of the immune system that are involved in protecting the body against both infectious diseases and foreign materials. The abnormal development and uncontrolled proliferation of these cells can lead to devastating cancers. Their timely recognition in the peripheral blood is critical to diagnosis and treatment. In this study, we developed a microscopic imaging system for improving the visualization of white blood cells on Wright's stained blood smear slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3) of five types of white blood cells (neutrophil, eosinophil, basophil, lymphocyte, and monocyte), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP), and the degree of circular polarization (DOCP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of granules in granulocytes (neutrophils, eosinophils, and basophils). Furthermore, Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of surface structures (protein patterns) of lymphocytes enabling subclassification of lymphocyte subpopulations. Finally, S2, S3, and DOCP could enhance the morphologic visualization of monocyte nucleus. We also demonstrated that the polarized hyperspectral imaging setup could provide complementary spectral information to the spatial information on different Stokes vector parameters of white blood cells. This work demonstrates that polarized light imaging & polarized hyperspectral imaging has the potential to become a strong imaging tool in the diagnosis of disorders arising from white blood cells.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Samuel Ortega
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Canary Islands, Spain
- Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Gustavo Marrero Callicó
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Canary Islands, Spain
| | - Edward A Medina
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX
| | - Bradley B Brimhall
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX
| | - Marisa Whitted
- Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
7
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Sumer BD, Fei B. Polarized hyperspectral microscopic imaging for collagen visualization on pathologic slides of head and neck squamous cell carcinoma. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238204. [PMID: 38481487 PMCID: PMC10932728 DOI: 10.1117/12.2655831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
8
|
Imaging perfusion changes in oncological clinical applications by hyperspectral imaging: a literature review. Radiol Oncol 2022; 56:420-429. [PMID: 36503709 PMCID: PMC9784371 DOI: 10.2478/raon-2022-0051] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 11/02/2022] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Hyperspectral imaging (HSI) is a promising imaging modality that uses visible light to obtain information about blood flow. It has the distinct advantage of being noncontact, nonionizing, and noninvasive without the need for a contrast agent. Among the many applications of HSI in the medical field are the detection of various types of tumors and the evaluation of their blood flow, as well as the healing processes of grafts and wounds. Since tumor perfusion is one of the critical factors in oncology, we assessed the value of HSI in quantifying perfusion changes during interventions in clinical oncology through a systematic review of the literature. MATERIALS AND METHODS The PubMed and Web of Science electronic databases were searched using the terms "hyperspectral imaging perfusion cancer" and "hyperspectral imaging resection cancer". The inclusion criterion was the use of HSI in clinical oncology, meaning that all animal, phantom, ex vivo, experimental, research and development, and purely methodological studies were excluded. RESULTS Twenty articles met the inclusion criteria. The anatomic locations of the neoplasms in the selected articles were as follows: kidneys (1 article), breasts (2 articles), eye (1 article), brain (4 articles), entire gastrointestinal (GI) tract (1 article), upper GI tract (5 articles), and lower GI tract (6 articles). CONCLUSIONS HSI is a potentially attractive imaging modality for clinical application in oncology, with assessment of mastectomy skin flap perfusion after reconstructive breast surgery and anastomotic perfusion during reconstruction of gastrointenstinal conduit as the most promising at present.
Collapse
|
9
|
Ma L, Rathgeb A, Mubarak H, Tran M, Fei B. Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-220039GRR. [PMID: 35578386 PMCID: PMC9110022 DOI: 10.1117/1.jbo.27.5.056502] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 04/28/2022] [Indexed: 06/15/2023]
Abstract
SIGNIFICANCE Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space. AIM WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics. APPROACH High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection. RESULTS Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results. CONCLUSIONS The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.
Collapse
Affiliation(s)
- Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Tianjin University, State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin, China
| | - Armand Rathgeb
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Hasan Mubarak
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Minh Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
10
|
Hoyeck MP, Matteo G, MacFarlane EM, Perera I, Bruin JE. Persistent organic pollutants and β-cell toxicity: a comprehensive review. Am J Physiol Endocrinol Metab 2022; 322:E383-E413. [PMID: 35156417 PMCID: PMC9394781 DOI: 10.1152/ajpendo.00358.2021] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/09/2021] [Revised: 12/20/2021] [Accepted: 02/07/2022] [Indexed: 01/09/2023]
Abstract
Persistent organic pollutants (POPs) are a diverse family of contaminants that show widespread global dispersion and bioaccumulation. Humans are continuously exposed to POPs through diet, air particles, and household and commercial products; POPs are consistently detected in human tissues, including the pancreas. Epidemiological studies show a modest but consistent correlation between exposure to POPs and increased diabetes risk. The goal of this review is to provide an overview of epidemiological evidence and an in-depth evaluation of the in vivo and in vitro evidence that POPs cause β-cell toxicity. We review evidence for six classes of POPs: dioxins, polychlorinated biphenyls (PCBs), organochlorine pesticides (OCPs), organophosphate pesticides (OPPs), flame retardants, and per- and polyfluoroalkyl substances (PFAS). The available data provide convincing evidence implicating POPs as a contributing factor driving impaired glucose homeostasis, β-cell dysfunction, and altered metabolic and oxidative stress pathways in islets. These findings support epidemiological data showing that POPs increase diabetes risk and emphasize the need to consider the endocrine pancreas in toxicity assessments. Our review also highlights significant gaps in the literature assessing islet-specific endpoints after both in vivo and in vitro POP exposure. In addition, most rodent studies do not consider the impact of biological sex or secondary metabolic stressors in mediating the effects of POPs on glucose homeostasis and β-cell function. We discuss key gaps and limitations that should be assessed in future studies.
Collapse
Affiliation(s)
- Myriam P Hoyeck
- Department of Biology and Institute of Biochemistry, Carleton University, Ottawa, Ontario, Canada
| | - Geronimo Matteo
- Department of Biology and Institute of Biochemistry, Carleton University, Ottawa, Ontario, Canada
- Department of Biology, University of Ottawa, Ottawa, Ontario, Canada
- Environmental Health Science and Research Bureau, Health Canada, Ottawa, Ontario, Canada
| | - Erin M MacFarlane
- Department of Biology and Institute of Biochemistry, Carleton University, Ottawa, Ontario, Canada
| | - Ineli Perera
- Department of Biology and Institute of Biochemistry, Carleton University, Ottawa, Ontario, Canada
| | - Jennifer E Bruin
- Department of Biology and Institute of Biochemistry, Carleton University, Ottawa, Ontario, Canada
| |
Collapse
|
11
|
Ma L, Little JV, Chen AY, Myers L, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-210399GR. [PMID: 35484692 PMCID: PMC9050479 DOI: 10.1117/1.jbo.27.4.046501] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 04/05/2022] [Indexed: 06/14/2023]
Abstract
SIGNIFICANCE Automatic, fast, and accurate identification of cancer on histologic slides has many applications in oncologic pathology. AIM The purpose of this study is to investigate hyperspectral imaging (HSI) for automatic detection of head and neck cancer nuclei in histologic slides, as well as cancer region identification based on nuclei detection. APPROACH A customized hyperspectral microscopic imaging system was developed and used to scan histologic slides from 20 patients with squamous cell carcinoma (SCC). Hyperspectral images and red, green, and blue (RGB) images of the histologic slides with the same field of view were obtained and registered. A principal component analysis-based nuclei segmentation method was developed to extract nuclei patches from the hyperspectral images and the coregistered RGB images. Spectra-based support vector machine and patch-based convolutional neural networks (CNNs) were implemented for nuclei classification. The CNNs were trained with RGB patches (RGB-CNN) and hyperspectral patches (HSI-CNN) of the segmented nuclei and the utility of the extra spectral information provided by HSI was evaluated. Furthermore, cancer region identification was implemented by image-wise classification based on the percentage of cancerous nuclei detected in each image. RESULTS RGB-CNN, which mainly used the spatial information of nuclei, resulted in a 0.81 validation accuracy and 0.74 testing accuracy. HSI-CNN, which utilized the spatial and spectral features of the nuclei, showed significant improvement in classification performance and achieved 0.89 validation accuracy as well as 0.82 testing accuracy. Furthermore, the image-wise cancer region identification based on nuclei detection could generally improve the cancer detection rate. CONCLUSIONS We demonstrated that the morphological and spectral information contribute to SCC nuclei differentiation and that the spectral information within hyperspectral images could improve classification performance.
Collapse
Affiliation(s)
- Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Tianjin University, State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin, China
| | - James V. Little
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, Georgia, United States
| | - Amy Y. Chen
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Larry Myers
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Baran D. Sumer
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, Texas, United States
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
12
|
Zhou X, Ma L, Mubarak HK, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12039:120390G. [PMID: 36798940 PMCID: PMC9930132 DOI: 10.1117/12.2614624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
The study is to incorporate polarized hyperspectral imaging (PHSI) with deep learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we firstly collected the Stokes vector data cubes (S0, S1, S2, and S3) of histologic slides from 17 patients with SCC by the PHSI microscope, under the wavelength range from 467 nm to 750 nm. Secondly, we generated the synthetic RGB images from the original Stokes vector data cubes. Thirdly, we cropped the synthetic RGB images into image patches at the image size of 96×96 pixels, and then set up a ResNet50-based convolutional neural network (CNN) to classify the image patches of the four Stokes vector parameters (S0, S1, S2, and S3) by application of transfer learning. To test the performances of the model, each time we trained the model based on the image patches (S0, S1, S2, and S3) of 16 patients out of 17 patients, and used the trained model to calculate the testing accuracy based on the image patches of the rest 1 patient (S0, S1, S2, and S3). We repeated the process for 6 times and obtained 24 testing accuracies (S0, S1, S2, and S3) from 6 different patients out of the 17 patients. The preliminary results showed that the average testing accuracy (84.2%) on S3 outperformed the average testing accuracy (83.5%) on S0. Furthermore, 4 of 6 testing accuracies of S3 (96.0%, 87.3%, 82.8%, and 86.7%) outperformed the testing accuracies of S0 (93.3%, 85.2%, 80.2%, and 79.0%). The study demonstrated the potential of using polarized hyperspectral imaging and deep learning for automatic detection of head and neck SCC on pathologic slides.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - James V. Little
- Emory University, Department of Pathology and Laboratory Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, GA
| | - Larry L. Myers
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baran D. Sumer
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX
- Univ. of Texas Southwestern Medical Center, Dept. of Radiology, Dallas, TX
| |
Collapse
|
13
|
Leitch K, Halicek M, Shahedi M, Little JV, Chen AY, Fei B. Detecting Aggressive Papillary Thyroid Carcinoma Using Hyperspectral Imaging and Radiomic Features. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12033:1203322. [PMID: 36798628 PMCID: PMC9929637 DOI: 10.1117/12.2611842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Hyperspectral imaging (HSI) and radiomics have the potential to improve the accuracy of tumor malignancy prediction and assessment. In this work, we extracted radiomic features of fresh surgical papillary thyroid carcinoma (PTC) specimen that were imaged with HSI. A total of 107 unique radiomic features were extracted. This study includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. With the dilated hyperspectral images, the shape feature of least axis length was able to predict the tumor aggressiveness with a high accuracy. The HSI-based radiomic method may provide a useful tool to aid oncologists in determining tumors with intermediate to high risk and in clinical decision making.
Collapse
Affiliation(s)
- Ka’Toria Leitch
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
14
|
Ma L, Shahedi M, Shi T, Halicek M, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Pixel-level Tumor Margin Assessment of Surgical Specimen with Hyperspectral Imaging and Deep Learning Classification. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11598:1159811. [PMID: 35755403 PMCID: PMC9232191 DOI: 10.1117/12.2581046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Surgery is a major treatment method for squamous cell carcinoma (SCC). During surgery, insufficient tumor margin may lead to local recurrence of cancer. Hyperspectral imaging (HSI) is a promising optical imaging technique for in vivo cancer detection and tumor margin assessment. In this study, a fully convolutional network (FCN) was implemented for tumor classification and margin assessment on hyperspectral images of SCC. The FCN was trained and validated with hyperspectral images of 25 ex vivo SCC surgical specimens from 20 different patients. The network was evaluated per patient and achieved pixel-level tissue classification with an average area under the curve (AUC) of 0.88, as well as 0.83 accuracy, 0.84 sensitivity, and 0.70 specificity across all the 20 patients. The 95% Hausdorff distance of assessed tumor margin in 17 patients was less than 2 mm, and the classification time of each tissue specimen took less than 10 seconds. The proposed methods can potentially facilitate intraoperative tumor margin assessment and improve surgical outcomes.
Collapse
Affiliation(s)
- Ling Ma
- Department of Bioengineering, University of Texas at Dallas
- State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin University
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas
| | - Ted Shi
- Department of Bioengineering, University of Texas at Dallas
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University
| | - Larry L. Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center
- Department of Radiology, University of Texas Southwestern Medical Center
| |
Collapse
|
15
|
Zhou X, Ma L, Brown W, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and machine learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11603:116030Q. [PMID: 34955584 PMCID: PMC8699168 DOI: 10.1117/12.2582330] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
The aim of this study is to incorporate polarized hyperspectral imaging (PHSI) with machine learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we imaged 20 H&E stained tissue slides from 10 patients with SCC of the larynx by the PHSI microscope. Several machine learning algorithms, including support vector machine (SVM), random forest, Gaussian naive Bayes, and logistic regression, were applied to the collected image data for the automatic detection of SCC on the H&E stained tissue slides. The performance of these methods was compared among the collected PHSI data, the pseudo-RGB images generated from the PHSI data, and the PHSI data after applying the principal component analysis (PCA) transformation. The results suggest that SVM is a superior classifier for the classification task based on the PHSI data cubes compared to the other three classifiers. The incorporate of four Stokes vector parameters improved the classification accuracy. Finally, the PCA transformed image data did not improve the accuracy as it might lose some important information from the original PHSI data. The preliminary results show that polarized hyperspectral imaging can have many potential applications in digital pathology.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
| | - William Brown
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
| | - James V. Little
- Emory University, Department of Pathology and Laboratory
Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta,
GA
| | - Larry L. Myers
- Univ. of Texas Southwestern Medical Center, Dept. of
Otolaryngology, Dallas, TX
| | - Baran D. Sumer
- Univ. of Texas Southwestern Medical Center, Dept. of
Otolaryngology, Dallas, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Advanced
Imaging Research Center, Dallas, TX
- Univ. of Texas Southwestern Medical Center, Dept. of
Radiology, Dallas, TX
| |
Collapse
|
16
|
Ma L, Zhou X, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Hyperspectral Microscopic Imaging for the Detection of Head and Neck Squamous Cell Carcinoma on Histologic Slides. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11603:116030P. [PMID: 35783088 PMCID: PMC9248908 DOI: 10.1117/12.2581970] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
The purpose of this study is to investigate hyperspectral microscopic imaging and deep learning methods for automatic detection of head and neck squamous cell carcinoma (SCC) on histologic slides. Hyperspectral imaging (HSI) cubes were acquired from pathologic slides of 18 patients with SCC of the larynx, hypopharynx, and buccal mucosa. An Inception-based two-dimensional convolutional neural network (CNN) was trained and validated for the HSI data. The automatic deep learning method was tested with independent data of human patients. This study demonstrated the feasibility of using hyperspectral microscopic imaging and deep learning classification to aid pathologists in detecting SCC on histologic slides.
Collapse
Affiliation(s)
- Ling Ma
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas
- State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin University
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University
| | - Larry L. Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baowei Fei
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center
- Department of Radiology, University of Texas Southwestern Medical Center
| |
Collapse
|
17
|
Edwards K, Halicek M, Little JV, Chen AY, Fei B. Multiparametric Radiomics for Predicting the Aggressiveness of Papillary Thyroid Carcinoma Using Hyperspectral Images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11597:1159728. [PMID: 35756897 PMCID: PMC9232190 DOI: 10.1117/12.2582147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Papillary thyroid carcinoma (PTC) is primarily treated by surgical resection. During surgery, surgeons often need intraoperative frozen analysis and pathologic consultation in order to detect PTC. In some cases pathologists cannot determine if the tumor is aggressive until the operation has been completed. In this work, we have taken tumor classification a step further by determining the tumor aggressiveness of fresh surgical specimens. We employed hyperspectral imaging (HSI) in combination with multiparametric radiomic features to complete this task. The study cohort includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. A total of 67 features were extracted from this data. Using machine learning classification methods, we were able to achieve an AUC of 0.85. Our study shows that hyperspectral imaging and multiparametric radiomic features could aid in the pathological detection of tumor aggressiveness using fresh surgical spemens obtained during surgery.
Collapse
Affiliation(s)
- Ka’Toria Edwards
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|