1
|
Mubarak HK, Zhou X, Palsgrove D, Sumer BD, Chen AY, Fei B. An Ensemble Learning Method for Detection of Head and Neck Squamous Cell Carcinoma Using Polarized Hyperspectral Microscopic Imaging. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12933:129330P. [PMID: 38711533 PMCID: PMC11073817 DOI: 10.1117/12.3007869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.
Collapse
Affiliation(s)
- Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University, Atlanta, GA
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
2
|
Zhou X, Ma L, Mubarak HK, Palsgrove D, Sumer BD, Chen AY, Fei B. Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:016005. [PMID: 38239390 PMCID: PMC10795499 DOI: 10.1117/1.jbo.29.1.016005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 12/04/2023] [Accepted: 12/20/2023] [Indexed: 01/22/2024]
Abstract
Significance Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications. Aim The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides. Approach We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: S 0 , S 1 , S 2 , and S 3 . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors. Results By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides. Conclusions With the spatial and spectral information from the Stokes vector data cubes (S 0 , S 1 , S 2 , and S 3 ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Hasan K. Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Doreen Palsgrove
- The University of Texas Southwestern Medical Center, Department of Pathology, Dallas, Texas, United States
| | - Baran D. Sumer
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
3
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Ortega S, Callicó GM, Medina EA, Brimhall BB, Whitted M, Fei B. Polarized Hyperspectral Microscopic Imaging for White Blood Cells on Wright-Stained Blood Smear Slides. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238207. [PMID: 38486823 PMCID: PMC10938460 DOI: 10.1117/12.2655708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
White blood cells, also called leukocytes, are hematopoietic cells of the immune system that are involved in protecting the body against both infectious diseases and foreign materials. The abnormal development and uncontrolled proliferation of these cells can lead to devastating cancers. Their timely recognition in the peripheral blood is critical to diagnosis and treatment. In this study, we developed a microscopic imaging system for improving the visualization of white blood cells on Wright's stained blood smear slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3) of five types of white blood cells (neutrophil, eosinophil, basophil, lymphocyte, and monocyte), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP), and the degree of circular polarization (DOCP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of granules in granulocytes (neutrophils, eosinophils, and basophils). Furthermore, Stokes vector derived parameters (DOP, DOLP, and DOCP) could improve the visualization of surface structures (protein patterns) of lymphocytes enabling subclassification of lymphocyte subpopulations. Finally, S2, S3, and DOCP could enhance the morphologic visualization of monocyte nucleus. We also demonstrated that the polarized hyperspectral imaging setup could provide complementary spectral information to the spatial information on different Stokes vector parameters of white blood cells. This work demonstrates that polarized light imaging & polarized hyperspectral imaging has the potential to become a strong imaging tool in the diagnosis of disorders arising from white blood cells.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Samuel Ortega
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Canary Islands, Spain
- Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Gustavo Marrero Callicó
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Canary Islands, Spain
| | - Edward A Medina
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX
| | - Bradley B Brimhall
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX
| | - Marisa Whitted
- Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
4
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Sumer BD, Fei B. Polarized hyperspectral microscopic imaging for collagen visualization on pathologic slides of head and neck squamous cell carcinoma. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238204. [PMID: 38481487 PMCID: PMC10932728 DOI: 10.1117/12.2655831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
5
|
Zhou X, Ma L, Mubarak HK, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12039:120390G. [PMID: 36798940 PMCID: PMC9930132 DOI: 10.1117/12.2614624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
The study is to incorporate polarized hyperspectral imaging (PHSI) with deep learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we firstly collected the Stokes vector data cubes (S0, S1, S2, and S3) of histologic slides from 17 patients with SCC by the PHSI microscope, under the wavelength range from 467 nm to 750 nm. Secondly, we generated the synthetic RGB images from the original Stokes vector data cubes. Thirdly, we cropped the synthetic RGB images into image patches at the image size of 96×96 pixels, and then set up a ResNet50-based convolutional neural network (CNN) to classify the image patches of the four Stokes vector parameters (S0, S1, S2, and S3) by application of transfer learning. To test the performances of the model, each time we trained the model based on the image patches (S0, S1, S2, and S3) of 16 patients out of 17 patients, and used the trained model to calculate the testing accuracy based on the image patches of the rest 1 patient (S0, S1, S2, and S3). We repeated the process for 6 times and obtained 24 testing accuracies (S0, S1, S2, and S3) from 6 different patients out of the 17 patients. The preliminary results showed that the average testing accuracy (84.2%) on S3 outperformed the average testing accuracy (83.5%) on S0. Furthermore, 4 of 6 testing accuracies of S3 (96.0%, 87.3%, 82.8%, and 86.7%) outperformed the testing accuracies of S0 (93.3%, 85.2%, 80.2%, and 79.0%). The study demonstrated the potential of using polarized hyperspectral imaging and deep learning for automatic detection of head and neck SCC on pathologic slides.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - James V. Little
- Emory University, Department of Pathology and Laboratory Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, GA
| | - Larry L. Myers
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baran D. Sumer
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX
- Univ. of Texas Southwestern Medical Center, Dept. of Radiology, Dallas, TX
| |
Collapse
|
6
|
Abdlaty R, Abbass MA, Awadallah AM. High Precision Monitoring of Radiofrequency Ablation for Liver Using Hyperspectral Imaging. Ann Biomed Eng 2021; 49:2430-2440. [PMID: 34075450 DOI: 10.1007/s10439-021-02797-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2021] [Accepted: 05/17/2021] [Indexed: 02/03/2023]
Abstract
Minimally invasive procedures are achieving better satisfaction for treating liver cancers. Energy-based techniques were studied as prospective alternatives to the gold standard of liver transplantation. Among these techniques, radiofrequency (RF) was investigated for the selective ablation of liver tissue. In addition to optimizing the RF settings for the purpose of overcoming tissue perforation or inadequate ablation, an instrument collecting quantitative data regarding the intraoperative tissue status can aid the treatment procedure. This study demonstrates an innovative noninvasive technique using hyperspectral imaging (HSI) for monitoring RF ablative therapy in ex-vivo liver tissue. The cubic data generated by HSI provides spectral as well as spatial properties of the liver tissue included in each pixel of the field of view. In our study, the applied statistical analysis saves the computational burdens of multivariate analysis techniques. For this purpose, spectral angle mapper, logistic regression algorithm, and principal component analysis were applied. Of all spectral bands captured by the HSI camera, bands centered at 760 and 960 nm were identified for predicting the ablated area. Based on statistical analysis, the threshold for predicting the ablated area of the liver samples was determined, provided that the specificity is kept at 90%.
Collapse
Affiliation(s)
- Ramy Abdlaty
- Department of Biomedical Engineering, Military Technical College, Cairo, Egypt.
| | - Mohamed A Abbass
- Department of Biomedical Engineering, Military Technical College, Cairo, Egypt
| | - Ahmed M Awadallah
- Department of Biomedical Engineering, Military Technical College, Cairo, Egypt
| |
Collapse
|