1
|
Tomanic T, Bozic T, Markelc B, Stergar J, Sersa G, Milanic M. Hyperspectral imaging of 4T1 mammary carcinomas grown in dorsal skinfold window chambers: spectral renormalization and fluorescence modeling. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:093504. [PMID: 39040986 PMCID: PMC11262746 DOI: 10.1117/1.jbo.29.9.093504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Revised: 06/28/2024] [Accepted: 07/02/2024] [Indexed: 07/24/2024]
Abstract
Significance Hyperspectral imaging (HSI) of murine tumor models grown in dorsal skinfold window chambers (DSWCs) offers invaluable insight into the tumor microenvironment. However, light loss in a glass coverslip is often overlooked, and particular tissue characteristics are improperly modeled, leading to errors in tissue properties extracted from hyperspectral images. Aim We highlight the significance of spectral renormalization in HSI of DSWC models and demonstrate the benefit of incorporating enhanced green fluorescent protein (EGFP) excitation and emission in the skin tissue model for tumors expressing genes to produce EGFP. Approach We employed an HSI system for intravital imaging of mice with 4T1 mammary carcinoma in a DSWC over 14 days. We performed spectral renormalization of hyperspectral images based on the measured reflectance spectra of glass coverslips and utilized an inverse adding-doubling (IAD) algorithm with a two-layer murine skin model, to extract tissue parameters, such as total hemoglobin concentration and tissue oxygenation (StO 2 ). The model was upgraded to consider EGFP fluorescence excitation and emission. Moreover, we conducted additional experiments involving tissue phantoms, human forearm skin imaging, and numerical simulations. Results Hyperspectral image renormalization and the addition of EGFP fluorescence in the murine skin model reduced the mean absolute percentage errors (MAPEs) of fitted and measured spectra by up to 10% in tissue phantoms, 0.55% to 1.5% in the human forearm experiment and numerical simulations, and up to 0.7% in 4T1 tumors. Similarly, the MAPEs for tissue parameters extracted by IAD were reduced by up to 3% in human forearms and numerical simulations. For some parameters, statistically significant differences ( p < 0.05 ) were observed in 4T1 tumors. Ultimately, we have shown that fluorescence emission could be helpful for 4T1 tumor segmentation. Conclusions The results contribute to improving intravital monitoring of DWSC models using HSI and pave the way for more accurate and precise quantitative imaging.
Collapse
Affiliation(s)
- Tadej Tomanic
- University of Ljubljana, Faculty of Mathematics and Physics, Ljubljana, Slovenia
| | - Tim Bozic
- Institute of Oncology Ljubljana, Department of Experimental Oncology, Ljubljana, Slovenia
| | - Bostjan Markelc
- Institute of Oncology Ljubljana, Department of Experimental Oncology, Ljubljana, Slovenia
| | - Jost Stergar
- University of Ljubljana, Faculty of Mathematics and Physics, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| | - Gregor Sersa
- Institute of Oncology Ljubljana, Department of Experimental Oncology, Ljubljana, Slovenia
- University of Ljubljana, Faculty of Health Sciences, Ljubljana, Slovenia
| | - Matija Milanic
- University of Ljubljana, Faculty of Mathematics and Physics, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| |
Collapse
|
2
|
Tran MH, Ma L, Mubarak H, Gomez O, Yu J, Bryarly M, Fei B. Detection and margin assessment of thyroid carcinoma with microscopic hyperspectral imaging using transformer networks. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:093505. [PMID: 39050615 PMCID: PMC11268383 DOI: 10.1117/1.jbo.29.9.093505] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Revised: 06/21/2024] [Accepted: 06/28/2024] [Indexed: 07/27/2024]
Abstract
Significance Hyperspectral imaging (HSI) is an emerging imaging modality for oncological applications and can improve cancer detection with digital pathology. Aim The study aims to highlight the increased accuracy and sensitivity of detecting the margin of thyroid carcinoma in hematoxylin and eosin (H&E)-stained histological slides using HSI and data augmentation methods. Approach Using an automated microscopic imaging system, we captured 2599 hyperspectral images from 65 H&E-stained human thyroid slides. Images were then preprocessed into 153,906 image patches of dimension 250 × 250 × 84 pixels . We modified the TimeSformer network architecture, which used alternating spectral attention and spatial attention layers. We implemented several data augmentation methods for HSI based on the RandAugment algorithm. We compared the performances of TimeSformer on HSI against the performances of pretrained ConvNext and pretrained vision transformers (ViT) networks on red, green, and blue (RGB) images. Finally, we applied attention unrolling techniques on the trained TimeSformer network to identify the biological features to which the network paid attention. Results In the testing dataset, TimeSformer achieved an accuracy of 90.87%, a weightedF 1 score of 89.79%, a sensitivity of 91.50%, and an area under the receiving operator characteristic curve (AU-ROC) score of 97.04%. Additionally, TimeSformer produced thyroid carcinoma tumor margins with an average Jaccard score of 0.76 mm. Without data augmentation, TimeSformer achieved an accuracy of 88.23%, a weightedF 1 score of 86.46%, a sensitivity of 85.53%, and an AU-ROC score of 94.94%. In comparison, the ViT network achieved an 89.98% accuracy, an 88.14% weightedF 1 score, an 84.77% sensitivity, and a 96.17% AU-ROC. Our visualization results showed that the network paid attention to biological features. Conclusions The TimeSformer model trained with hyperspectral histological data consistently outperformed conventional RGB-based models, highlighting the superiority of HSI in this context. Our proposed augmentation methods improved the accuracy, theF 1 score, and the sensitivity score.
Collapse
Affiliation(s)
- Minh Ha Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Hasan Mubarak
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ofelia Gomez
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - James Yu
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| | - Michelle Bryarly
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
3
|
Kunhoth S, Al-Maadeed S. An Analytical Study on the Utility of RGB and Multispectral Imagery with Band Selection for Automated Tumor Grading. Diagnostics (Basel) 2024; 14:1625. [PMID: 39125501 PMCID: PMC11312293 DOI: 10.3390/diagnostics14151625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2024] [Revised: 07/20/2024] [Accepted: 07/25/2024] [Indexed: 08/12/2024] Open
Abstract
The implementation of tumor grading tasks with image processing and machine learning techniques has progressed immensely over the past several years. Multispectral imaging enabled us to capture the sample as a set of image bands corresponding to different wavelengths in the visible and infrared spectrums. The higher dimensional image data can be well exploited to deliver a range of discriminative features to support the tumor grading application. This paper compares the classification accuracy of RGB and multispectral images, using a case study on colorectal tumor grading with the QU-Al Ahli Dataset (dataset I). Rotation-invariant local phase quantization (LPQ) features with an SVM classifier resulted in 80% accuracy for the RGB images compared to 86% accuracy with the multispectral images in dataset I. However, the higher dimensionality elevates the processing time. We propose a band-selection strategy using mutual information between image bands. This process eliminates redundant bands and increases classification accuracy. The results show that our band-selection method provides better results than normal RGB and multispectral methods. The band-selection algorithm was also tested on another colorectal tumor dataset, the Texas University Dataset (dataset II), to further validate the results. The proposed method demonstrates an accuracy of more than 94% with 10 bands, compared to using the whole set of 16 multispectral bands. Our research emphasizes the advantages of multispectral imaging over the RGB imaging approach and proposes a band-selection method to address the higher computational demands of multispectral imaging.
Collapse
|
4
|
Li Q, Zhang X, Zhang J, Huang H, Li L, Guo C, Li W, Guo Y. Deep learning-based hyperspectral technique identifies metastatic lymph nodes in oral squamous cell carcinoma-A pilot study. Oral Dis 2024. [PMID: 39005220 DOI: 10.1111/odi.15067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 05/31/2024] [Accepted: 07/02/2024] [Indexed: 07/16/2024]
Abstract
AIMS To establish a system based on hyperspectral imaging and deep learning for the detection of cancer cells in metastatic lymph nodes. MAIN METHODS The continuous sections of metastatic lymph nodes from 45 oral squamous cell carcinoma (OSCC) patients were collected. An improved ResUNet algorithm was established for deep learning to analyze the spectral curve differences between cancer cells and lymphocytes, and that between tumor tissue and normal tissue. KEY FINDINGS It was found that cancer cells, lymphocytes, and erythrocytes in the metastatic lymph nodes could be distinguished basing hyperspectral image, with overall accuracy (OA) as 87.30% and average accuracy (AA) as 85.46%. Cancerous area could be recognized by hyperspectral image and deep learning, and the average intersection over union (IOU) and accuracy were 0.6253 and 0.7692, respectively. SIGNIFICANCE This study indicated that deep learning-based hyperspectral techniques can identify tumor tissue in OSCC metastatic lymph nodes, achieving high accuracy of pathological diagnosis, high work efficiency, and reducing work burden. But these are preliminary results limited to a small sample.
Collapse
Affiliation(s)
- Qingxiang Li
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Xueyu Zhang
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Jianyun Zhang
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
- Department of Oral Pathology, Peking University School and Hospital of Stomatology, Beijing, China
| | - Hongyuan Huang
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Liangliang Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Chuanbin Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| | - Wei Li
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
- Beijing Key Laboratory of Fractional Signals and Systems, Beijing, China
| | - Yuxing Guo
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology, Beijing, China
- National Center for Stomatology, Beijing, China
- National Clinical Research Center for Oral Diseases, Beijing, China
- National Engineering Research Center of Oral Biomaterials and Digital Medical Devices, Beijing, China
| |
Collapse
|
5
|
İnam O, El-Baz A, Kaplan HJ, Tezel TH. Colorimetric Analyses of the Optic Nerve Head and Retina Indicate Increased Blood Flow After Vitrectomy. Transl Vis Sci Technol 2024; 13:12. [PMID: 39007833 PMCID: PMC467108 DOI: 10.1167/tvst.13.7.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Accepted: 06/03/2024] [Indexed: 07/16/2024] Open
Abstract
Purpose The purpose of this study was to evaluate the impact of vitrectomy and posterior hyaloid (PH) peeling on color alteration of optic nerve head (ONH) and retina as a surrogate biomarker of induced perfusion changes. Methods Masked morphometric and colorimetric analyses were conducted on preoperative (<1 month) and postoperative (<18 months) color fundus photographs of 54 patients undergoing vitrectomy, either with (44) or without (10) PH peeling and 31 years of age and gender-matched control eyes. Images were calibrated according to the hue and saturation values of the parapapillary venous blood column. Chromatic spectra of the retinal pigment epithelium and choroid were subtracted to avoid color aberrations. Red, green, and blue (RGB) bit values over the ONH and retina were plotted within the constructed RGB color space to analyze vitrectomy-induced color shift. Vitrectomy-induced parapapillary vein caliber changes were also computed morphometrically. Results A significant post-vitrectomy red hue shift was noted on the ONH (37.1 degrees ± 10.9 degrees vs. 4.1 degrees ± 17.7 degrees, P < 0.001), which indicates a 2.8-fold increase in blood perfusion compared to control (2.6 ± 1.9 vs. 0.9 ± 1.8, P < 0.001). A significant post-vitrectomy increase in the retinal vein diameter was also noticed (6.8 ± 6.4% vs. 0.1 ± 0.3%, P < 0.001), which was more pronounced with PH peeling (7.9 ± 6.6% vs. 3.1 ± 4.2%, P = 0.002). Conclusions Vitrectomy and PH peeling increase ONH and retinal blood flow. Colorimetric and morphometric analyses offer valuable insights for future artificial intelligence and deep learning applications in this field. Translational Relevance The methodology described herein can easily be applied in different clinical settings and may enlighten the beneficial effects of vitrectomy in several retinal vascular diseases.
Collapse
Affiliation(s)
- Onur İnam
- Department of Ophthalmology, Edward S. Harkness Eye Institute, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY, USA
- Department of Biophysics, Faculty of Medicine, Gazi University, Ankara, Turkey
| | - Ayman El-Baz
- Bioengineering Department, University of Louisville, Louisville, KY, USA
| | - Henry J. Kaplan
- Department of Ophthalmology & Visual Sciences, Kentucky Lions Eye Center, University of Louisville School of Medicine, Louisville, KY, USA
- Department of Ophthalmology, Saint Louis University, School of Medicine, St. Louis, MO, USA
| | - Tongalp H. Tezel
- Department of Ophthalmology, Edward S. Harkness Eye Institute, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY, USA
- Department of Ophthalmology & Visual Sciences, Kentucky Lions Eye Center, University of Louisville School of Medicine, Louisville, KY, USA
| |
Collapse
|
6
|
Ortega S, Quintana-Quintana L, Leon R, Fabelo H, Plaza MDLL, Camacho R, Callico GM. Histological Hyperspectral Glioblastoma Dataset (HistologyHSI-GB). Sci Data 2024; 11:681. [PMID: 38914542 PMCID: PMC11196658 DOI: 10.1038/s41597-024-03510-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Accepted: 06/10/2024] [Indexed: 06/26/2024] Open
Abstract
Hyperspectral (HS) imaging (HSI) technology combines the main features of two existing technologies: imaging and spectroscopy. This allows to analyse simultaneously the morphological and chemical attributes of the objects captured by a HS camera. In recent years, the use of HSI provides valuable insights into the interaction between light and biological tissues, and makes it possible to detect patterns, cells, or biomarkers, thus, being able to identify diseases. This work presents the HistologyHSI-GB dataset, which contains 469 HS images from 13 patients diagnosed with brain tumours, specifically glioblastoma. The slides were stained with haematoxylin and eosin (H&E) and captured using a microscope at 20× power magnification. Skilled histopathologists diagnosed the slides and provided image-level annotations. The dataset was acquired using custom HSI instrumentation, consisting of a microscope equipped with an HS camera covering the spectral range from 400 to 1000 nm.
Collapse
Affiliation(s)
- Samuel Ortega
- Seafood Industry Department, Norwegian Institute of Food, Fisheries and Aquaculture Research (Nofima), Tromsø, Norway.
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain.
- Department of Mathematics and Statistics, UiT The Arctic University of Norway, Tromsø, Norway.
| | - Laura Quintana-Quintana
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Raquel Leon
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Himar Fabelo
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
- Fundación Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), Las Palmas de Gran Canaria, Spain
- Research Unit, Hospital Universitario de Gran Canaria Doctor Negrín, Las Palmas de Gran Canaria, Spain
| | - María de la Luz Plaza
- Department of Pathological Anatomy, Hospital Universitario de Gran Canaria Doctor Negrín, Las Palmas de Gran Canaria, Spain
| | - Rafael Camacho
- Department of Pathological Anatomy, Hospital Universitario de Gran Canaria Doctor Negrín, Las Palmas de Gran Canaria, Spain
| | - Gustavo M Callico
- Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| |
Collapse
|
7
|
Norose T, Ohike N, Nakaya D, Kamiya K, Sugiura Y, Takatsuki M, Koizumi H, Okawa C, Ohya A, Sasaki M, Aoki R, Nakahara K, Kobayashi S, Tateishi K, Koike J. Investigation of the usefulness of a bile duct biopsy and bile cytology using a hyperspectral camera and machine learning. Pathol Int 2024; 74:337-345. [PMID: 38787324 DOI: 10.1111/pin.13438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 04/15/2024] [Accepted: 05/04/2024] [Indexed: 05/25/2024]
Abstract
To improve the efficiency of pathological diagnoses, the development of automatic pathological diagnostic systems using artificial intelligence (AI) is progressing; however, problems include the low interpretability of AI technology and the need for large amounts of data. We herein report the usefulness of a general-purpose method that combines a hyperspectral camera with machine learning. As a result of analyzing bile duct biopsy and bile cytology specimens, which are especially difficult to determine as benign or malignant, using multiple machine learning models, both were able to identify benign or malignant cells with an accuracy rate of more than 80% (93.3% for bile duct biopsy specimens and 83.2% for bile cytology specimens). This method has the potential to contribute to the diagnosis and treatment of bile duct cancer and is expected to be widely applied and utilized in general pathological diagnoses.
Collapse
Affiliation(s)
- Tomoko Norose
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Nobuyuki Ohike
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | | | | | - Yoshiya Sugiura
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Misato Takatsuki
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Hirotaka Koizumi
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Chie Okawa
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Aya Ohya
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Miyu Sasaki
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Ruka Aoki
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Kazunari Nakahara
- Department of Gastroenterology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Shinjiro Kobayashi
- Department of Gastroenterological and General Surgery, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Keisuke Tateishi
- Department of Gastroenterology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Junki Koike
- Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| |
Collapse
|
8
|
Campbell JM, Gosnell M, Agha A, Handley S, Knab A, Anwer AG, Bhargava A, Goldys EM. Label-Free Assessment of Key Biological Autofluorophores: Material Characteristics and Opportunities for Clinical Applications. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024:e2403761. [PMID: 38775184 DOI: 10.1002/adma.202403761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Revised: 05/04/2024] [Indexed: 06/13/2024]
Abstract
Autofluorophores are endogenous fluorescent compounds that naturally occur in the intra and extracellular spaces of all tissues and organs. Most have vital biological functions - like the metabolic cofactors NAD(P)H and FAD+, as well as the structural protein collagen. Others are considered to be waste products - like lipofuscin and advanced glycation end products - which accumulate with age and are associated with cellular dysfunction. Due to their natural fluorescence, these materials have great utility for enabling non-invasive, label-free assays with direct ties to biological function. Numerous technologies, with different advantages and drawbacks, are applied to their assessment, including fluorescence lifetime imaging microscopy, hyperspectral microscopy, and flow cytometry. Here, the applications of label-free autofluorophore assessment are reviewed for clinical and health-research applications, with specific attention to biomaterials, disease detection, surgical guidance, treatment monitoring, and tissue assessment - fields that greatly benefit from non-invasive methodologies capable of continuous, in vivo characterization.
Collapse
Affiliation(s)
- Jared M Campbell
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| | | | - Adnan Agha
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| | - Shannon Handley
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| | - Aline Knab
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| | - Ayad G Anwer
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| | - Akanksha Bhargava
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| | - Ewa M Goldys
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW, 2033, Australia
| |
Collapse
|
9
|
Zhang Y, Dong J. SAHIS-Net: a spectral attention and feature enhancement network for microscopic hyperspectral cholangiocarcinoma image segmentation. BIOMEDICAL OPTICS EXPRESS 2024; 15:3147-3162. [PMID: 38855697 PMCID: PMC11161366 DOI: 10.1364/boe.519090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 04/03/2024] [Accepted: 04/09/2024] [Indexed: 06/11/2024]
Abstract
Cholangiocarcinoma (CCA) poses a significant clinical challenge due to its aggressive nature and poor prognosis. While traditional diagnosis relies on color-based histopathology, hyperspectral imaging (HSI) offers rich, high-dimensional data holding potential for more accurate diagnosis. However, extracting meaningful insights from this data remains challenging. This work investigates the application of deep learning for CCA segmentation in microscopic HSI images, and introduces two novel neural networks: (1) Histogram Matching U-Net (HM-UNet) for efficient image pre-processing, and (2) Spectral Attention based Hyperspectral Image Segmentation Net (SAHIS-Net) for CCA segmentation. SAHIS-Net integrates a novel Spectral Attention (SA) module for adaptively weighing spectral information, an improved attention-aware feature enhancement (AFE) mechanism for better providing the model with more discriminative features, and a multi-loss training strategy for effective early stage feature extraction. We compare SAHIS-Net against several general and CCA-specific models, demonstrating its superior performance in segmenting CCA regions. These results highlight the potential of our approach for segmenting medical HSI images.
Collapse
Affiliation(s)
- Yunchu Zhang
- School of Future Science and Engineering, Soochow University, Suzhou 215222, Jiangsu, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou 215163, Jiangsu, China
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, Jiangsu, China
| | - Jianfei Dong
- School of Future Science and Engineering, Soochow University, Suzhou 215222, Jiangsu, China
| |
Collapse
|
10
|
Abraham TM, Levenson R. Current Landscape of Advanced Imaging Tools for Pathology Diagnostics. Mod Pathol 2024; 37:100443. [PMID: 38311312 DOI: 10.1016/j.modpat.2024.100443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 12/13/2023] [Accepted: 01/26/2024] [Indexed: 02/10/2024]
Abstract
Histopathology relies on century-old workflows of formalin fixation, paraffin embedding, sectioning, and staining tissue specimens on glass slides. Despite being robust, this conventional process is slow, labor-intensive, and limited to providing two-dimensional views. Emerging technologies promise to enhance and accelerate histopathology. Slide-free microscopy allows rapid imaging of fresh, unsectioned specimens, overcoming slide preparation delays. Methods such as fluorescence confocal microscopy, multiphoton microscopy, along with more recent innovations including microscopy with UV surface excitation and fluorescence-imitating brightfield imaging can generate images resembling conventional histology directly from the surface of tissue specimens. Slide-free microscopy enable applications such as rapid intraoperative margin assessment and, with appropriate technology, three-dimensional histopathology. Multiomics profiling techniques, including imaging mass spectrometry and Raman spectroscopy, provide highly multiplexed molecular maps of tissues, although clinical translation remains challenging. Artificial intelligence is aiding the adoption of new imaging modalities via virtual staining, which converts methods such as slide-free microscopy into synthetic brightfield-like or even molecularly informed images. Although not yet commonplace, these emerging technologies collectively demonstrate the potential to modernize histopathology. Artificial intelligence-assisted workflows will ease the transition to new imaging modalities. With further validation, these advances may transform the century-old conventional histopathology pipeline to better serve 21st-century medicine. This review provides an overview of these enabling technology platforms and discusses their potential impact.
Collapse
Affiliation(s)
- Tanishq Mathew Abraham
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Richard Levenson
- Department of Pathology and Laboratory Medicine, UC Davis Health, Sacramento, California.
| |
Collapse
|
11
|
Pertzborn D, Bali A, Mühlig A, von Eggeling F, Guntinas-Lichius O. Hyperspectral imaging and evaluation of surgical margins: where do we stand? Curr Opin Otolaryngol Head Neck Surg 2024; 32:96-104. [PMID: 38193544 DOI: 10.1097/moo.0000000000000957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2024]
Abstract
PURPOSE OF REVIEW To highlight the recent literature on the use of hyperspectral imaging (HSI) for cancer margin evaluation ex vivo, for head and neck cancer pathology and in vivo during head and neck cancer surgery. RECENT FINDINGS HSI can be used ex vivo on unstained and stained tissue sections to analyze head and neck tissue and tumor cells in combination with machine learning approaches to analyze head and neck cancer cell characteristics and to discriminate the tumor border from normal tissue. Data on in vivo applications during head and neck cancer surgery are preliminary and limited. Even now an accuracy of 80% for tumor versus nonneoplastic tissue classification can be achieved for certain tasks, within the current in vivo settings. SUMMARY Significant progress has been made to introduce HSI for ex vivo head and neck cancer pathology evaluation and for an intraoperative use to define the tumor margins. To optimize the accuracy for in vivo use, larger HSI databases with annotations for head and neck cancer are needed.
Collapse
Affiliation(s)
- David Pertzborn
- Department of Otorhinolaryngology, Jena University Hospital, Jena, Germany
| | | | | | | | | |
Collapse
|
12
|
Shugar AL, Konger RL, Rohan CA, Travers JB, Kim YL. Mapping cutaneous field carcinogenesis of nonmelanoma skin cancer using mesoscopic imaging of pro-inflammation cues. Exp Dermatol 2024; 33:e15076. [PMID: 38610095 PMCID: PMC11034840 DOI: 10.1111/exd.15076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2023] [Revised: 03/24/2024] [Accepted: 03/30/2024] [Indexed: 04/14/2024]
Abstract
Nonmelanoma skin cancers remain the most widely diagnosed types of cancers globally. Thus, for optimal patient management, it has become imperative that we focus our efforts on the detection and monitoring of cutaneous field carcinogenesis. The concept of field cancerization (or field carcinogenesis), introduced by Slaughter in 1953 in the context of oral cancer, suggests that invasive cancer may emerge from a molecularly and genetically altered field affecting a substantial area of underlying tissue including the skin. A carcinogenic field alteration, present in precancerous tissue over a relatively large area, is not easily detected by routine visualization. Conventional dermoscopy and microscopy imaging are often limited in assessing the entire carcinogenic landscape. Recent efforts have suggested the use of noninvasive mesoscopic (between microscopic and macroscopic) optical imaging methods that can detect chronic inflammatory features to identify pre-cancerous and cancerous angiogenic changes in tissue microenvironments. This concise review covers major types of mesoscopic optical imaging modalities capable of assessing pro-inflammatory cues by quantifying blood haemoglobin parameters and hemodynamics. Importantly, these imaging modalities demonstrate the ability to detect angiogenesis and inflammation associated with actinically damaged skin. Representative experimental preclinical and human clinical studies using these imaging methods provide biological and clinical relevance to cutaneous field carcinogenesis in altered tissue microenvironments in the apparently normal epidermis and dermis. Overall, mesoscopic optical imaging modalities assessing chronic inflammatory hyperemia can enhance the understanding of cutaneous field carcinogenesis, offer a window of intervention and monitoring for actinic keratoses and nonmelanoma skin cancers and maximise currently available treatment options.
Collapse
Affiliation(s)
- Andrea L. Shugar
- Department of Pharmacology & Toxicology, Wright State University Boonshoft School of Medicine, Dayton, Ohio, USA
| | - Raymond L. Konger
- Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Department of Dermatology, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Department of Pathology, Richard L. Roudebush Veterans Administration Hospital, Indianapolis, Indiana, USA
| | - Craig A. Rohan
- Department of Pharmacology & Toxicology, Wright State University Boonshoft School of Medicine, Dayton, Ohio, USA
- Department of Dermatology, Wright State University Boonshoft School of Medicine, Dayton, Ohio, USA
- Department of Medicine, Dayton Veterans Affairs Medical Center, Dayton, Ohio, USA
| | - Jeffrey B. Travers
- Department of Pharmacology & Toxicology, Wright State University Boonshoft School of Medicine, Dayton, Ohio, USA
- Department of Dermatology, Wright State University Boonshoft School of Medicine, Dayton, Ohio, USA
- Department of Medicine, Dayton Veterans Affairs Medical Center, Dayton, Ohio, USA
| | - Young L. Kim
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA
- Purdue Institute for Cancer Research, Purdue University, West Lafayette, Indiana, USA
| |
Collapse
|
13
|
Myslicka M, Kawala-Sterniuk A, Bryniarska A, Sudol A, Podpora M, Gasz R, Martinek R, Kahankova Vilimkova R, Vilimek D, Pelc M, Mikolajewski D. Review of the application of the most current sophisticated image processing methods for the skin cancer diagnostics purposes. Arch Dermatol Res 2024; 316:99. [PMID: 38446274 DOI: 10.1007/s00403-024-02828-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 12/28/2023] [Accepted: 01/25/2024] [Indexed: 03/07/2024]
Abstract
This paper presents the most current and innovative solutions applying modern digital image processing methods for the purpose of skin cancer diagnostics. Skin cancer is one of the most common types of cancers. It is said that in the USA only, one in five people will develop skin cancer and this trend is constantly increasing. Implementation of new, non-invasive methods plays a crucial role in both identification and prevention of skin cancer occurrence. Early diagnosis and treatment are needed in order to decrease the number of deaths due to this disease. This paper also contains some information regarding the most common skin cancer types, mortality and epidemiological data for Poland, Europe, Canada and the USA. It also covers the most efficient and modern image recognition methods based on the artificial intelligence applied currently for diagnostics purposes. In this work, both professional, sophisticated as well as inexpensive solutions were presented. This paper is a review paper and covers the period of 2017 and 2022 when it comes to solutions and statistics. The authors decided to focus on the latest data, mostly due to the rapid technology development and increased number of new methods, which positively affects diagnosis and prognosis.
Collapse
Affiliation(s)
- Maria Myslicka
- Faculty of Medicine, Wroclaw Medical University, J. Mikulicza-Radeckiego 5, 50-345, Wroclaw, Poland.
| | - Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland.
| | - Anna Bryniarska
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Adam Sudol
- Faculty of Natural Sciences and Technology, University of Opole, Dmowskiego 7-9, 45-368, Opole, Poland
| | - Michal Podpora
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Rafal Gasz
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
| | - Radek Martinek
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Radana Kahankova Vilimkova
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Proszkowska 76, 45-758, Opole, Poland
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Dominik Vilimek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, Ostrava, 70800, Czech Republic
| | - Mariusz Pelc
- Institute of Computer Science, University of Opole, Oleska 48, 45-052, Opole, Poland
- School of Computing and Mathematical Sciences, University of Greenwich, Old Royal Naval College, Park Row, SE10 9LS, London, UK
| | - Dariusz Mikolajewski
- Institute of Computer Science, Kazimierz Wielki University in Bydgoszcz, ul. Kopernika 1, 85-074, Bydgoszcz, Poland
- Neuropsychological Research Unit, 2nd Clinic of the Psychiatry and Psychiatric Rehabilitation, Medical University in Lublin, Gluska 1, 20-439, Lublin, Poland
| |
Collapse
|
14
|
Schmidt VM, Zelger P, Wöss C, Fodor M, Hautz T, Schneeberger S, Huck CW, Arora R, Brunner A, Zelger B, Schirmer M, Pallua JD. Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains. Heliyon 2024; 10:e25844. [PMID: 38375262 PMCID: PMC10875450 DOI: 10.1016/j.heliyon.2024.e25844] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 01/30/2024] [Accepted: 02/02/2024] [Indexed: 02/21/2024] Open
Abstract
In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.
Collapse
Affiliation(s)
- Verena-Maria Schmidt
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - Philipp Zelger
- University Clinic for Hearing, Voice and Speech Disorders, Medical University of Innsbruck, Anichstrasse 35, 6020 Innsbruck, Austria
| | - Claudia Wöss
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - Margot Fodor
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Theresa Hautz
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Stefan Schneeberger
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Christian Wolfgang Huck
- Institute of Analytical Chemistry and Radiochemistry, University of Innsbruck, 6020 Innsbruck, Austria
| | - Rohit Arora
- Department of Orthopaedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| | - Andrea Brunner
- Institute of Pathology, Neuropathology, and Molecular Pathology, Medical University of Innsbruck, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - Bettina Zelger
- Institute of Pathology, Neuropathology, and Molecular Pathology, Medical University of Innsbruck, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - Michael Schirmer
- Department of Internal Medicine, Clinic II, Medical University of Innsbruck, Anichstrasse 35, 6020 Innsbruck, Austria
| | - Johannes Dominikus Pallua
- Department of Orthopaedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| |
Collapse
|
15
|
Mubarak HK, Zhou X, Palsgrove D, Sumer BD, Chen AY, Fei B. An Ensemble Learning Method for Detection of Head and Neck Squamous Cell Carcinoma Using Polarized Hyperspectral Microscopic Imaging. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12933:129330P. [PMID: 38711533 PMCID: PMC11073817 DOI: 10.1117/12.3007869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.
Collapse
Affiliation(s)
- Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University, Atlanta, GA
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, The University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
16
|
Zhou X, Ma L, Mubarak HK, Palsgrove D, Sumer BD, Chen AY, Fei B. Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:016005. [PMID: 38239390 PMCID: PMC10795499 DOI: 10.1117/1.jbo.29.1.016005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 12/04/2023] [Accepted: 12/20/2023] [Indexed: 01/22/2024]
Abstract
Significance Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications. Aim The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides. Approach We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: S 0 , S 1 , S 2 , and S 3 . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors. Results By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides. Conclusions With the spatial and spectral information from the Stokes vector data cubes (S 0 , S 1 , S 2 , and S 3 ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Hasan K. Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Doreen Palsgrove
- The University of Texas Southwestern Medical Center, Department of Pathology, Dallas, Texas, United States
| | - Baran D. Sumer
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
17
|
Leon R, Fabelo H, Ortega S, Cruz-Guerrero IA, Campos-Delgado DU, Szolna A, Piñeiro JF, Espino C, O'Shanahan AJ, Hernandez M, Carrera D, Bisshopp S, Sosa C, Balea-Fernandez FJ, Morera J, Clavo B, Callico GM. Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection. NPJ Precis Oncol 2023; 7:119. [PMID: 37964078 PMCID: PMC10646050 DOI: 10.1038/s41698-023-00475-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 10/24/2023] [Indexed: 11/16/2023] Open
Abstract
Brain surgery is one of the most common and effective treatments for brain tumour. However, neurosurgeons face the challenge of determining the boundaries of the tumour to achieve maximum resection, while avoiding damage to normal tissue that may cause neurological sequelae to patients. Hyperspectral (HS) imaging (HSI) has shown remarkable results as a diagnostic tool for tumour detection in different medical applications. In this work, we demonstrate, with a robust k-fold cross-validation approach, that HSI combined with the proposed processing framework is a promising intraoperative tool for in-vivo identification and delineation of brain tumours, including both primary (high-grade and low-grade) and secondary tumours. Analysis of the in-vivo brain database, consisting of 61 HS images from 34 different patients, achieve a highest median macro F1-Score result of 70.2 ± 7.9% on the test set using both spectral and spatial information. Here, we provide a benchmark based on machine learning for further developments in the field of in-vivo brain tumour detection and delineation using hyperspectral imaging to be used as a real-time decision support tool during neurosurgical workflows.
Collapse
Affiliation(s)
- Raquel Leon
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain.
| | - Himar Fabelo
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain.
- Fundación Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), Las Palmas de Gran Canaria, Spain.
| | - Samuel Ortega
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
- Nofima, Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway
| | - Ines A Cruz-Guerrero
- Facultad de Ciencias, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
- Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
- Department of Pediatric Plastic and Reconstructive Surgery, Children's Hospital Colorado, Aurora, Colorado, USA
| | - Daniel Ulises Campos-Delgado
- Facultad de Ciencias, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
- Instituto de Investigación en Comunicación Óptica, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
| | - Adam Szolna
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Juan F Piñeiro
- Instituto de Investigación en Comunicación Óptica, Universidad Autónoma de San Luis Potosí, San Luis Potosí, México
| | - Carlos Espino
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Aruma J O'Shanahan
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Maria Hernandez
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - David Carrera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Sara Bisshopp
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Coralia Sosa
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Francisco J Balea-Fernandez
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
- Department of Psychology, Sociology and Social Work, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Jesus Morera
- Department of Neurosurgery, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Bernardino Clavo
- Fundación Canaria Instituto de Investigación Sanitaria de Canarias (FIISC), Las Palmas de Gran Canaria, Spain
- Research Unit, University Hospital Doctor Negrin of Gran Canaria, Las Palmas de Gran Canaria, Spain
| | - Gustavo M Callico
- Research Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain
| |
Collapse
|
18
|
Chalopin C, Pfahl A, Köhler H, Knospe L, Maktabi M, Unger M, Jansen-Winkeln B, Thieme R, Moulla Y, Mehdorn M, Sucher R, Neumuth T, Gockel I, Melzer A. Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography. MINIM INVASIV THER 2023; 32:222-232. [PMID: 36622288 DOI: 10.1080/13645706.2022.2164469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 11/29/2022] [Indexed: 01/10/2023]
Abstract
INTRODUCTION Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA. MATERIAL AND METHODS The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works. RESULTS The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method. CONCLUSIONS Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Luise Knospe
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Department of Electrical, Mechanical and Industrial Engineering, Anhalt University of Applied Science, Köthen (Anhalt), Germany
| | - Michael Unger
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Matthias Mehdorn
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Robert Sucher
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic, and Vascular Surgery, University Hospital of Leipzig AöR, Leipzig, Germany
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery (ICCAS), Faculty of Medicine, Leipzig University, Leipzig, Germany
- Institute of Medical Science and Technology (IMSAT), University of Dundee, Dundee, UK
| |
Collapse
|
19
|
Stockmans TA, Snik F, Esposito M, van Dijk C, Keller CU. InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments. APPLIED OPTICS 2023; 62:7185-7198. [PMID: 37855574 DOI: 10.1364/ao.498021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 08/28/2023] [Indexed: 10/20/2023]
Abstract
Classic designs of hyperspectral instrumentation densely sample the spatial and spectral information of the scene of interest. Data may be compressed after the acquisition. In this paper, we introduce a framework for the design of an optimized, micropatterned snapshot hyperspectral imager that acquires an optimized subset of the spatial and spectral information in the scene. The data is thereby already compressed at the sensor level but can be restored to the full hyperspectral data cube by the jointly optimized reconstructor. This framework is implemented with TensorFlow and makes use of its automatic differentiation for the joint optimization of the layout of the micropatterned filter array as well as the reconstructor. We explore the achievable compression ratio for different numbers of filter passbands, number of scanning frames, and filter layouts using data collected by the Hyperscout instrument. We show resulting instrument designs that take snapshot measurements without losing significant information while reducing the data volume, acquisition time, or detector space by a factor of 40 as compared to classic, dense sampling. The joint optimization of a compressive hyperspectral imager design and the accompanying reconstructor provides an avenue to substantially reduce the data volume from hyperspectral imagers.
Collapse
|
20
|
Shinoda K. Unsupervised design for broadband multispectral and polarization filter array patterns. APPLIED OPTICS 2023; 62:7145-7155. [PMID: 37855568 DOI: 10.1364/ao.499545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 08/23/2023] [Indexed: 10/20/2023]
Abstract
Imaging multiple wavelength and polarization components is problematic due to the complexity of equipment and the increase in the number of imaging shots, so imaging using filter arrays with various patterns has been widely reported from elemental research to practical applications. Most of them use bandpass filters with different center wavelengths for each pixel. Recently, however, filter arrays with multimodal transmission characteristics have been proposed using photonic crystals or Fabry-Perot filters. In any of these methods, the design of the filter array arrangement pattern is important to improve the quality of the captured image, as well as the improvement of the demosaicking algorithm. One way to design a filter array pattern is to minimize the mean squared error (MSE) between the ideal image and the demosaicked image. However, the more multidimensional the imaging components, the more difficult it becomes to collect training data. In such cases, it is necessary to empirically determine candidate transmission characteristics and patterns of filter arrays. In this study, we propose a method for evaluating filter array patterns without using any training data in the design of filter arrays for multispectral and polarization imaging. The proposed method estimates the MSE by approximating the autocorrelation matrix without using image data by expressing the imaging model as a linear forward problem and the demosaicking as a linear inverse problem. Since this method can be applied not only to ideal bandpass filter arrangements, but also to multispectral filter arrays with multimodal spectral transmission characteristics and even multispectral polarization filter arrays with different extinction ratios at different wavelengths, we will show that image quality can be improved over empirical arrangements by evaluating these patterns and by testing examples of optimal designs using genetic algorithms.
Collapse
|
21
|
Ogura Y, Shimano M, Bise R, Yamashita T, Katagiri C, Sato I. Analysis of optical absorption of photoaged human skin using a high-frequency illumination microscopy analysis system. Exp Dermatol 2023; 32:1402-1411. [PMID: 37264684 DOI: 10.1111/exd.14840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2022] [Revised: 04/04/2023] [Accepted: 05/07/2023] [Indexed: 06/03/2023]
Abstract
Skin is composed of different layers, including the stratum corneum, epidermal living layer and papillary and reticular dermis. Each has specific optical properties due to differences in their biological components. Alterations in the skin's cutaneous biological components resulting from photoaging caused by chronic exposure to UV light affect the deterioration of appearance associated with the skin's optical properties. Various methods for analysing cutaneous optical properties have been previously proposed, including mathematical models and computer simulations. However, these were insufficient to elucidate changes in each skin layer and comprehensively understand the skin's integrated optical properties. We focused on UV-induced yellowing of the facial skin. We evaluated site-specific optical absorption of human skin tissue sections to investigate the yellowish discoloration, which is suggested to be related to the photodamage process. The method includes our original technique of separating the transmitted and scattered light using high-frequency illumination microscopy, leading to microscopic analysis of the tissue's optical absorption in the regions of interest. In analysing the sun-exposed facial skin tissue sections, we successfully showed that dermal regions of aged skin have increased absorption at 450 nm, where yellowish colours are complemented. Furthermore, we confirmed that elastic fibres with observable histological disorder resulting from photodamage are a prominent source of high optical absorption. We detected changes in the skin's optical absorption associated with dermal degeneration resulting from photodamage using a novel optical microscopy technique. The results provide a base for the evaluation of optical property changes for both yellowing discoloration and other tissue disorders.
Collapse
Affiliation(s)
- Yuki Ogura
- Shiseido Global Innovation Center, Yokohama, Japan
| | | | - Ryoma Bise
- Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka, Japan
| | | | | | - Imari Sato
- National Institute of Informatics, Tokyo, Japan
| |
Collapse
|
22
|
Sijilmassi O, López Alonso JM, Del Río Sevilla A, Barrio Asensio MDC. Multispectral Imaging Method for Rapid Identification and Analysis of Paraffin-Embedded Pathological Tissues. J Digit Imaging 2023; 36:1663-1674. [PMID: 37072579 PMCID: PMC10406798 DOI: 10.1007/s10278-023-00826-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 03/28/2023] [Accepted: 03/30/2023] [Indexed: 04/20/2023] Open
Abstract
The study of the interaction between light and biological tissue is of great help in the identification of diseases as well as structural alterations in tissues. In the present study, we have developed a tissue diagnostic technique by using multispectral imaging in the visible spectrum combined with principal component analysis (PCA). We used information from the propagation of light through paraffin-embedded tissues to assess differences in the eye tissues of control mouse embryos compared to mouse embryos whose mothers were deprived of folic acid (FA), a crucial vitamin necessary for the growth and development of the fetus. After acquiring the endmembers from the multispectral images, spectral unmixing was used to identify the abundances of those endmembers in each pixel. For each acquired image, the final analysis was performed by performing a pixel-by-pixel and wavelength-by-wavelength absorbance calculation. Non-negative least squares (NNLS) were used in this research. The abundance maps obtained for the first endmember revealed vascular alterations (vitreous and choroid) in the embryos with maternal FA deficiency. However, the abundance maps obtained for the third endmember showed alterations in the texture of some tissues such as the lens and retina. Results indicated that multispectral imaging applied to paraffin-embedded tissues enhanced tissue visualization. Using this method, first, it can be seen tissue damage location and then decide what kind of biological techniques to apply.
Collapse
Affiliation(s)
- Ouafa Sijilmassi
- Faculty of Optics and Optometry, Anatomy and Embryology Department, Universidad Complutense de Madrid, Madrid, Spain.
- Optics Department, Faculty of Optics and Optometry, Universidad Complutense De Madrid, Madrid, Spain.
| | - José-Manuel López Alonso
- Optics Department, Faculty of Optics and Optometry, Universidad Complutense De Madrid, Madrid, Spain
| | - Aurora Del Río Sevilla
- Faculty of Optics and Optometry, Anatomy and Embryology Department, Universidad Complutense de Madrid, Madrid, Spain
| | | |
Collapse
|
23
|
Moeini B, Avval TG, Brongersma HH, Průša S, Bábík P, Vaníčková E, Strohmeier BR, Bell DS, Eggett D, George SM, Linford MR. Area-Selective Atomic Layer Deposition of ZnO on Si\SiO 2 Modified with Tris(dimethylamino)methylsilane. MATERIALS (BASEL, SWITZERLAND) 2023; 16:4688. [PMID: 37445002 DOI: 10.3390/ma16134688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 06/19/2023] [Accepted: 06/25/2023] [Indexed: 07/15/2023]
Abstract
Delayed atomic layer deposition (ALD) of ZnO, i.e., area selective (AS)-ALD, was successfully achieved on silicon wafers (Si\SiO2) terminated with tris(dimethylamino)methylsilane (TDMAMS). This resist molecule was deposited in a home-built, near-atmospheric pressure, flow-through, gas-phase reactor. TDMAMS had previously been shown to react with Si\SiO2 in a single cycle/reaction and to drastically reduce the number of silanols that remain at the surface. ZnO was deposited in a commercial ALD system using dimethylzinc (DMZ) as the zinc precursor and H2O as the coreactant. Deposition of TDMAMS was confirmed by spectroscopic ellipsometry (SE), X-ray photoelectron spectroscopy (XPS), and wetting. ALD of ZnO, including its selectivity on TDMAMS-terminated Si\SiO2 (Si\SiO2\TDMAMS), was confirmed by in situ multi-wavelength ellipsometry, ex situ SE, XPS, and/or high-sensitivity/low-energy ion scattering (HS-LEIS). The thermal stability of the TDMAMS resist layer, which is an important parameter for AS-ALD, was investigated by heating Si\SiO2\TDMAMS in air and nitrogen at 330 °C. ALD of ZnO takes place more readily on Si\SiO2\TDMAMS heated in the air than in N2, suggesting greater damage to the surface heated in the air. To better understand the in situ ALD of ZnO on Si\SiO2\TDMAMS and modified (thermally stressed) forms of it, the ellipsometry results were plotted as the normalized growth per cycle. Even one short pulse of TDMAMS effectively passivates Si\SiO2. TDMAMS can be a useful, small-molecule inhibitor of ALD of ZnO on Si\SiO2 surfaces.
Collapse
Affiliation(s)
- Behnam Moeini
- Department of Chemistry and Biochemistry, Brigham Young University, Provo, UT 84602, USA
| | - Tahereh G Avval
- Department of Chemistry and Biochemistry, Brigham Young University, Provo, UT 84602, USA
| | - Hidde H Brongersma
- Department of Applied Physics, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands
| | - Stanislav Průša
- Institute of Physical Engineering, Brno University of Technology, Technická 2, 616 69 Brno, Czech Republic
- CEITEC BUT, Brno University of Technology, Purkyňova 123, 612 00 Brno, Czech Republic
| | - Pavel Bábík
- Institute of Physical Engineering, Brno University of Technology, Technická 2, 616 69 Brno, Czech Republic
- CEITEC BUT, Brno University of Technology, Purkyňova 123, 612 00 Brno, Czech Republic
| | - Elena Vaníčková
- Institute of Physical Engineering, Brno University of Technology, Technická 2, 616 69 Brno, Czech Republic
- CEITEC BUT, Brno University of Technology, Purkyňova 123, 612 00 Brno, Czech Republic
| | - Brian R Strohmeier
- Materials Group-NA, Avery Dennison Corporation, 8080 Norton Parkway, Mentor, OH 44060, USA
| | - David S Bell
- Restek Corporation, 110 Benner Circle, Bellefonte, PA 16823, USA
| | - Dennis Eggett
- Department of Statistics, Brigham Young University, Provo, UT 84602, USA
| | - Steven M George
- Department of Chemistry, University of Colorado, 215 UCB, Boulder, CO 80309, USA
| | - Matthew R Linford
- Department of Chemistry and Biochemistry, Brigham Young University, Provo, UT 84602, USA
| |
Collapse
|
24
|
Monvoisin N, Hemsley E, Laplanche L, Almuneau G, Calvez S, Monmayrant A. Spectrally-shaped illumination for improved optical inspection of lateral III-V-semiconductor oxidation. OPTICS EXPRESS 2023; 31:12955-12966. [PMID: 37157444 DOI: 10.1364/oe.480753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
We report an hyperspectral imaging microscopy system based on a spectrally-shaped illumination and its use to offer an enhanced in-situ inspection of a technological process that is critical in Vertical-Cavity Surface-Emitting Laser (VCSEL) manufacturing, the lateral III-V-semiconductor oxidation (AlOx). The implemented illumination source exploits a digital micromirror device (DMD) to arbitrarily tailor its emission spectrum. When combined to an imager, this source is shown to provide an additional ability to detect minute surface reflectance contrasts on any VCSEL or AlOx-based photonic structure and, in turn, to offer improved in-situ inspection of the oxide aperture shapes and dimensions down to the best-achievable optical resolution. The demonstrated technique is very versatile and could be readily extended to the real-time monitoring of oxidation or other semiconductor technological processes as soon as they rely on a real-time yet accurate measurement of spatio-spectral (reflectance) maps.
Collapse
|
25
|
Quintana-Quintana L, Ortega S, Fabelo H, Balea-Fernández FJ, Callico GM. Blur-specific image quality assessment of microscopic hyperspectral images. OPTICS EXPRESS 2023; 31:12261-12279. [PMID: 37157389 DOI: 10.1364/oe.476949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Hyperspectral (HS) imaging (HSI) expands the number of channels captured within the electromagnetic spectrum with respect to regular imaging. Thus, microscopic HSI can improve cancer diagnosis by automatic classification of cells. However, homogeneous focus is difficult to achieve in such images, being the aim of this work to automatically quantify their focus for further image correction. A HS image database for focus assessment was captured. Subjective scores of image focus were obtained from 24 subjects and then correlated to state-of-the-art methods. Maximum Local Variation, Fast Image Sharpness block-based Method and Local Phase Coherence algorithms provided the best correlation results. With respect to execution time, LPC was the fastest.
Collapse
|
26
|
Wawrzyk-Bochenek I, Rahnama M, Stachura M, Wilczyński S, Wawrzyk A. Evaluation of the Reduction of Skin Hyperpigmentation Changes under the Influence of a Preparation Containing Kojic Acid Using Hyperspectral Imaging—Preliminary Study. J Clin Med 2023; 12:jcm12072710. [PMID: 37048793 PMCID: PMC10094916 DOI: 10.3390/jcm12072710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 03/22/2023] [Accepted: 04/03/2023] [Indexed: 04/08/2023] Open
Abstract
Aim: The aim of this study was to demonstrate the effects of using a preparation containing kojic acid on skin hyperpigmentation using hyperspectral imaging, which enables a quantitative assessment of the effect of the preparation used on the reduction of skin discoloration. Materials and methods: Preliminary studies were carried out on 12 patients with post-acne skin. A hyperspectral camera with a spectral range of 400–1000 nm was used to image skin hyperpigmentation before and after the application of 3% kojic acid. Hyperspectral profiles were analyzed, and image analysis and processing methods were applied. Results: Studies performed using a hyperspectral camera have shown that kojic acid reduces skin discoloration by increasing skin brightness in 75% of patients tested, reducing skin contrast in approximately 83% and increasing skin homogeneity in approximately 67% of patients.
Collapse
Affiliation(s)
- Iga Wawrzyk-Bochenek
- Department of Basic Biomedical Science, Faculty of Pharmaceutical Sciences in Sosnowiec, Medical University of Silesia, Kasztanowa 3, 41-205 Sosnowiec, Poland
| | - Mansur Rahnama
- Chair and Department of Oral Surgery, Medical University of Lublin, Chodźki 6, 20-093 Lublin, Poland
| | - Martyna Stachura
- Department of Basic Biomedical Science, Faculty of Pharmaceutical Sciences in Sosnowiec, Medical University of Silesia, Kasztanowa 3, 41-205 Sosnowiec, Poland
| | - Sławomir Wilczyński
- Department of Basic Biomedical Science, Faculty of Pharmaceutical Sciences in Sosnowiec, Medical University of Silesia, Kasztanowa 3, 41-205 Sosnowiec, Poland
| | - Anna Wawrzyk
- Department of Basic Biomedical Science, Faculty of Pharmaceutical Sciences in Sosnowiec, Medical University of Silesia, Kasztanowa 3, 41-205 Sosnowiec, Poland
- Silesian Park of Medical Technology Kardio-Med Silesia in Zabrze, M. Curie Skłodowskiej 10C, 41-800 Zabrze, Poland
| |
Collapse
|
27
|
Tran MH, Fei B. Compact and ultracompact spectral imagers: technology and applications in biomedical imaging. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:040901. [PMID: 37035031 PMCID: PMC10075274 DOI: 10.1117/1.jbo.28.4.040901] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 02/27/2023] [Indexed: 05/18/2023]
Abstract
Significance Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging. Aim With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications. Approach We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition. Results There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings. Conclusions Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.
Collapse
Affiliation(s)
- Minh H. Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- Address all correspondence to Baowei Fei,
| |
Collapse
|
28
|
Muniz FB, Baffa MDFO, Garcia SB, Bachmann L, Felipe JC. Histopathological diagnosis of colon cancer using micro-FTIR hyperspectral imaging and deep learning. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 231:107388. [PMID: 36773592 DOI: 10.1016/j.cmpb.2023.107388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Revised: 01/26/2023] [Accepted: 01/30/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Current studies based on digital biopsy images have achieved satisfactory results in detecting colon cancer despite their limited visual spectral range. Such methods may be less accurate when applied to samples taken from the tumor margin region or to samples containing multiple diagnoses. In contrast with the traditional computer vision approach, micro-FTIR hyperspectral images quantify the tissue-light interaction on a histochemical level and characterize different tissue pathologies, as they present a unique spectral signature. Therefore, this paper investigates the possibility of using hyperspectral images acquired over micro-FTIR absorbance spectroscopy to characterize healthy, inflammatory, and tumor colon tissues. METHODS The proposed method consists of modeling hyperspectral data into a voxel format to detect the patterns of each voxel using fully connected deep neural network. A web-based computer-aided diagnosis tool for inference is also provided. RESULTS Our experiments were performed using the K-fold cross-validation protocol in an intrapatient approach and achieved an overall accuracy of 99% using a deep neural network and 96% using a linear support vector machine. Through the experiments, we noticed the high performance of the method in characterizing such tissues using deep learning and hyperspectral images, indicating that the infrared spectrum contains relevant information and can be used to assist pathologists during the diagnostic process.
Collapse
Affiliation(s)
- Frederico Barbosa Muniz
- Department of Computing and Mathematics, University of São Paulo, Bandeirantes Av. 3900, Monte Alegre, Ribeirão Preto, SP 14040-901, Brazil
| | - Matheus de Freitas Oliveira Baffa
- Department of Computing and Mathematics, University of São Paulo, Bandeirantes Av. 3900, Monte Alegre, Ribeirão Preto, SP 14040-901, Brazil
| | - Sergio Britto Garcia
- Department of Pathology and Legal Medicine, Medical School of Ribeirão Preto, University of São Paulo, Ribeirão Preto, SP, Brazil
| | - Luciano Bachmann
- Department of Physics, University of São Paulo, Ribeirão Preto, SP, Brazil
| | - Joaquim Cezar Felipe
- Department of Computing and Mathematics, University of São Paulo, Bandeirantes Av. 3900, Monte Alegre, Ribeirão Preto, SP 14040-901, Brazil.
| |
Collapse
|
29
|
Stergar J, Hren R, Milanič M. Design and Validation of a Custom-Made Hyperspectral Microscope Imaging System for Biomedical Applications. SENSORS (BASEL, SWITZERLAND) 2023; 23:2374. [PMID: 36904578 PMCID: PMC10007032 DOI: 10.3390/s23052374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 02/10/2023] [Accepted: 02/19/2023] [Indexed: 06/18/2023]
Abstract
Hyperspectral microscope imaging (HMI) is an emerging modality that integrates spatial information collected by standard laboratory microscopy and the spectral-based contrast obtained by hyperspectral imaging and may be instrumental in establishing novel quantitative diagnostic methodologies, particularly in histopathology. Further expansion of HMI capabilities hinges upon the modularity and versatility of systems and their proper standardization. In this report, we describe the design, calibration, characterization, and validation of the custom-made laboratory HMI system based on a Zeiss Axiotron fully motorized microscope and a custom-developed Czerny-Turner-type monochromator. For these important steps, we rely on a previously designed calibration protocol. Validation of the system demonstrates a performance comparable to classic spectrometry laboratory systems. We further demonstrate validation against a laboratory hyperspectral imaging system for macroscopic samples, enabling future comparison of spectral imaging results across length scales. An example of the utility of our custom-made HMI system on a standard hematoxylin and eosin-stained histology slide is also shown.
Collapse
Affiliation(s)
- Jošt Stergar
- Jožef Stefan Institute, Jamova Cesta 39, SI-1000 Ljubljana, Slovenia
- Faculty of Mathematics and Physics, University of Ljubljana, Jadranska ulica 19, SI-1000 Ljubljana, Slovenia
| | - Rok Hren
- Faculty of Mathematics and Physics, University of Ljubljana, Jadranska ulica 19, SI-1000 Ljubljana, Slovenia
| | - Matija Milanič
- Jožef Stefan Institute, Jamova Cesta 39, SI-1000 Ljubljana, Slovenia
- Faculty of Mathematics and Physics, University of Ljubljana, Jadranska ulica 19, SI-1000 Ljubljana, Slovenia
| |
Collapse
|
30
|
Karim K, Lamaoui A, Amine A. Paper-based optical sensors paired with smartphones for biomedical analysis. J Pharm Biomed Anal 2023; 225:115207. [PMID: 36584551 DOI: 10.1016/j.jpba.2022.115207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 12/07/2022] [Accepted: 12/16/2022] [Indexed: 12/24/2022]
Abstract
The traditional analytical methods used for biomedical analysis are expensive and not easy to handle and require sophisticated instruments, thus their application is limited in resource-limited settings. Due to their portability, low cost, and ability to be applied to different analytical techniques, paper-based analytical devices are becoming valuable tools for biomedical analysis. The integration of smartphones into analytical devices has provided the ability to build portable, cost-effective, straightforward analytical devices for biomedical analysis and mobile health. The key aim of this review is to emphasize the recent applications of PADs combined with a smartphone for the optical analysis of biomedical species. We started this review by highlighting the type of papers and their modifications with different materials to prepare the PADs. After that, this review presents various detection methods including colorimetry, fluorescence, and luminescence where the smartphone is used for read-out. In the end, we provided the recent applications of the analysis of different biomedical compounds such as cancer and cardiovascular biomarkers, metal ions, glucose, viruses, etc. We believe that the present review will attract a wide scientific community in the areas of analytical chemistry, sensors, and clinical testing.
Collapse
Affiliation(s)
- Khadija Karim
- Laboratoire Génie des Procedés & Environnement, Faculté des Sciences et Techniques, Hassan II University of Casablanca, B.P. 146, Mohammedia, Morocco
| | - Abderrahman Lamaoui
- Laboratoire Génie des Procedés & Environnement, Faculté des Sciences et Techniques, Hassan II University of Casablanca, B.P. 146, Mohammedia, Morocco
| | - Aziz Amine
- Laboratoire Génie des Procedés & Environnement, Faculté des Sciences et Techniques, Hassan II University of Casablanca, B.P. 146, Mohammedia, Morocco.
| |
Collapse
|
31
|
Urbina Ortega C, Quevedo Gutiérrez E, Quintana L, Ortega S, Fabelo H, Santos Falcón L, Marrero Callico G. Towards Real-Time Hyperspectral Multi-Image Super-Resolution Reconstruction Applied to Histological Samples. SENSORS (BASEL, SWITZERLAND) 2023; 23:1863. [PMID: 36850461 PMCID: PMC9963731 DOI: 10.3390/s23041863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 02/01/2023] [Accepted: 02/03/2023] [Indexed: 06/18/2023]
Abstract
Hyperspectral Imaging (HSI) is increasingly adopted in medical applications for the usefulness of understanding the spectral signature of specific organic and non-organic elements. The acquisition of such images is a complex task, and the commercial sensors that can measure such images is scarce down to the point that some of them have limited spatial resolution in the bands of interest. This work proposes an approach to enhance the spatial resolution of hyperspectral histology samples using super-resolution. As the data volume associated to HSI has always been an inconvenience for the image processing in practical terms, this work proposes a relatively low computationally intensive algorithm. Using multiple images of the same scene taken in a controlled environment (hyperspectral microscopic system) with sub-pixel shifts between them, the proposed algorithm can effectively enhance the spatial resolution of the sensor while maintaining the spectral signature of the pixels, competing in performance with other state-of-the-art super-resolution techniques, and paving the way towards its use in real-time applications.
Collapse
Affiliation(s)
- Carlos Urbina Ortega
- European Space Agency, TEC-ED, 2201 AZ Noordwijk, The Netherlands
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
- Telespazio Belgium SRL, Huygensstraat 34, 2201 DK Noordwijk, The Netherlands
| | - Eduardo Quevedo Gutiérrez
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
| | - Laura Quintana
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
| | - Samuel Ortega
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
- Nofima—Norwegian Institute of Food Fisheries and Aquaculture Research, NO-9291 Tromsø, Norway
| | - Himar Fabelo
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
- Fundación Instituto de Investigación Sanitaria de Canarias (FIISC), 35017 Gran Canaria, Spain
| | - Lucana Santos Falcón
- European Space Agency, TEC-ED, 2201 AZ Noordwijk, The Netherlands
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
| | - Gustavo Marrero Callico
- Research Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
| |
Collapse
|
32
|
Tran MH, Gomez O, Fei B. A Video Transformer Network for Thyroid Cancer Detection on Hyperspectral Histologic Images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12471:1247107. [PMID: 38577581 PMCID: PMC10993530 DOI: 10.1117/12.2654851] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/06/2024]
Abstract
Hyperspectral imaging is a label-free and non-invasive imaging modality that seeks to capture images in different wavelengths. In this study, we used a vision transformer that was pre-trained from video data to detect thyroid cancer on hyperspectral images. We built a dataset of 49 whole slide hyperspectral images (WS-HSI) of thyroid cancer. To improve training, we introduced 5 new data augmentation methods that transform spectra. We achieved an F-1 score of 88.1% and an accuracy of 89.64% on our test dataset. The transformer network and the whole slide hyperspectral imaging technique can have many applications in digital pathology.
Collapse
Affiliation(s)
- Minh Ha Tran
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
| | - Ofelia Gomez
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
33
|
Aloupogianni E, Ishikawa M, Ichimura T, Hamada M, Murakami T, Sasaki A, Nakamura K, Kobayashi N, Obi T. Effects of dimension reduction of hyperspectral images in skin gross pathology. Skin Res Technol 2023; 29:e13270. [PMID: 36823506 PMCID: PMC10155843 DOI: 10.1111/srt.13270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 12/17/2022] [Indexed: 02/05/2023]
Abstract
BACKGROUND Hyperspectral imaging (HSI) is an emerging modality for the gross pathology of the skin. Spectral signatures of HSI could discriminate malignant from benign tissue. Because of inherent redundancies in HSI and in order to facilitate the use of deep-learning models, dimension reduction is a common preprocessing step. The effects of dimension reduction choice, training scope, and number of retained dimensions have not been evaluated on skin HSI for segmentation tasks. MATERIALS AND METHODS An in-house dataset of HSI signatures from pigmented skin lesions was prepared and labeled with histology. Eleven different dimension reduction methods were used as preprocessing for tumor margin detection with support vector machines. Cluster-wise principal component analysis (ClusterPCA), a new variant of PCA, was proposed. The scope of application for dimension reduction was also investigated. RESULTS The components produced by ClusterPCA show good agreement with the expected optical properties of skin chromophores. Random forest importance performed best during classification. However, all methods suffered from low sensitivity and generalization. CONCLUSION Investigation of more complex reduction and segmentation schemes with emphasis on the nature of HSI and optical properties of the skin is necessary. Insights on dimension reduction for skin tissue could facilitate the development of HSI-based systems for cancer margin detection at gross level.
Collapse
Affiliation(s)
- Eleni Aloupogianni
- Department of Information and Communications EngineeringTokyo Institute of TechnologyYokohamaJapan
| | - Masahiro Ishikawa
- Faculty of Health and Medical CareSaitama Medical University Hidaka CampusHidakaJapan
| | - Takaya Ichimura
- Department of PathologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Mei Hamada
- Department of PathologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Takuo Murakami
- Department of DermatologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Atsushi Sasaki
- Department of PathologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Koichiro Nakamura
- Department of DermatologyFaculty of MedicineSaitama Medical University Moroyama CampusMoroyamaJapan
| | - Naoki Kobayashi
- Department of Information and Communications EngineeringTokyo Institute of TechnologyYokohamaJapan
| | - Takashi Obi
- Department of Information and Communications EngineeringTokyo Institute of TechnologyYokohamaJapan
- Institute of Innovative Research, Tokyo Institute of TechnologyTokyoJapan
| |
Collapse
|
34
|
Ma L, Sherey J, Palsgrove D, Fei B. Conditional Generative Adversarial Network (cGAN) for Synthesis of Digital Histologic Images from Hyperspectral Images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12471:124711D. [PMID: 38495871 PMCID: PMC10942653 DOI: 10.1117/12.2653715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Hyperspectral imaging (HSI) has been demonstrated in various digital pathology applications. However, the intrinsic high dimensionality of hyperspectral images makes it difficult for pathologists to visualize the information. The aim of this study is to develop a method to transform hyperspectral images of hemoxylin & eosin (H&E)-stained slides to natural-color RGB histologic images for easy visualization. Hyperspectral images were obtained at 40× magnification with an automated microscopic imaging system and downsampled by various factors to generate data equivalent to different magnifications. High-resolution digital histologic RGB images were cropped and registered to the corresponding hyperspectral images as the ground truth. A conditional generative adversarial network (cGAN) was trained to output natural color RGB images of the histological tissue samples. The generated synthetic RGBs have similar color and sharpness to real RGBs. Image classification was implemented using the real and synthetic RGBs, respectively, with a pretrained network. The classification of tumor and normal tissue using the HSI-synthesized RGBs yielded a comparable but slightly higher accuracy and AUC than the real RGBs. The proposed method can reduce the acquisition time of two imaging modalities while giving pathologists access to the high information density of HSI and the quality visualization of RGBs. This study demonstrated that HSI may provide a potentially better alternative to current RGB-based pathologic imaging and thus make HSI a viable tool for histopathological diagnosis.
Collapse
Affiliation(s)
- Ling Ma
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Jeremy Sherey
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
35
|
Brunner A, Willenbacher E, Willenbacher W, Zelger B, Zelger P, Huck CW, Pallua JD. Visible- and near-infrared hyperspectral imaging for the quantitative analysis of PD-L1+ cells in human lymphomas: Comparison with fluorescent multiplex immunohistochemistry. SPECTROCHIMICA ACTA. PART A, MOLECULAR AND BIOMOLECULAR SPECTROSCOPY 2023; 285:121940. [PMID: 36208576 DOI: 10.1016/j.saa.2022.121940] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 09/22/2022] [Accepted: 09/28/2022] [Indexed: 06/16/2023]
Abstract
INTRODUCTION We analyzed the expression of PD-L1 in human lymphomas using hyperspectral imaging (HSI) compared to visual assessment (VA) and conventional digital image analysis (DIA) to strengthen further the value of HSI as a tool for the evaluation of brightfield-based immunohistochemistry (IHC). In addition, fluorescent multiplex immunohistochemistry (mIHC) was used as a second detection method to analyze the impact of a different detection method. MATERIAL AND METHODS 18 cases (6 follicular lymphomas and 12 diffuse large B-cell lymphomas) were stained for PD-L1 by IHC and for PD-L1, CD3, and CD8 by fluorescent mIHC. The percentage of positively stained cells was evaluated with VA, HSI, and DIA for IHC and VA and DIA for mIHC. Results were compared between the different methods of detection and analysis. RESULTS An overall high concordance was found between VA, HSI, and DIA in IHC (Cohens Kappa = 0.810VA/HSI, 0.710 VA/DIA, and 0.516 HSI/DIA) and for VAmIHCversus DIAmIHC (Cohens Kappa = 0.894). Comparing IHC and mIHC general agreement differed depending on the methods compared but reached at most a moderate agreement (Coheńs Kappa between 0.250 and 0.483). This is reflected by the significantly higher percentage of PD-L1+ cells found with mIHC (pFriedman = 0.014). CONCLUSION Our study shows a good concordance for the different analysis methods. Compared to VA and DIA, HSI proved to be a reliable tool for assessing IHC. Understanding the regulation of PD-L1 expression will further enlighten the role of PD-L1 as a biomarker. Therefore it is necessary to develop an instrument, such as HSI, which can offer a reliable and objective evaluation of PD-L1 expression.
Collapse
Affiliation(s)
- A Brunner
- Innsbruck Medical University, Institute of Pathology, Neuropathology and Molecular Pathology, Innsbruck, Austria
| | - E Willenbacher
- Innsbruck Medical University, Internal Medicine. V, Hematology & Oncology, Innsbruck, Austria
| | - W Willenbacher
- Innsbruck Medical University, Internal Medicine. V, Hematology & Oncology, Innsbruck, Austria; Syndena GmbH, Connect to Cure, Karl-Kapferer-Straße 5, 6020 Innsbruck, Austria
| | - B Zelger
- Innsbruck Medical University, Institute of Pathology, Neuropathology and Molecular Pathology, Innsbruck, Austria
| | - P Zelger
- Innsbruck Medical University, University Clinic for Hearing, Voice and Speech Disorders, Anichstrasse 35, Innsbruck, Austria
| | - C W Huck
- University of Innsbruck, Institute of Analytical Chemistry and Radiochemistry, Innsbruck, Austria
| | - J D Pallua
- Innsbruck Medical University, Department of Traumatology and Orthopaedics, Innsbruck, Austria.
| |
Collapse
|
36
|
Tran MH, Gomez O, Fei B. An automatic whole-slide hyperspectral imaging microscope. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12391:1239106. [PMID: 38481979 PMCID: PMC10932749 DOI: 10.1117/12.2650815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
Whole slide imaging (WSI) is a common step used in histopathology to quickly digitize stained histological slides. Digital whole-slide images not only improve the efficiency of labeling but also open the door for computer-aided diagnosis, specifically machine learning-based methods. Hyperspectral imaging (HSI) is an imaging modality that captures data in various wavelengths, some beyond the range of visible lights. In this study, we developed and implemented an automated microscopy system that can acquire hyperspectral whole slide images (HWSI). The system is robust since it consists of parts that can be swapped and bought from different manufacturers. We used the automated system and built a database of 49 HWSI of thyroid cancer. The automatic whole-slide hyperspectral imaging microscope can have many potential applications in biological and medical areas.
Collapse
Affiliation(s)
- Minh Ha Tran
- The Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Ofelia Gomez
- The Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Baowei Fei
- The Center for Imaging and Surgical Innovation, University of Texas at Dallas, Richardson, TX
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
37
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Sumer BD, Fei B. Polarized hyperspectral microscopic imaging for collagen visualization on pathologic slides of head and neck squamous cell carcinoma. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238204. [PMID: 38481487 PMCID: PMC10932728 DOI: 10.1117/12.2655831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
38
|
Tomanic T, Rogelj L, Stergar J, Markelc B, Bozic T, Brezar SK, Sersa G, Milanic M. Estimating quantitative physiological and morphological tissue parameters of murine tumor models using hyperspectral imaging and optical profilometry. JOURNAL OF BIOPHOTONICS 2023; 16:e202200181. [PMID: 36054067 DOI: 10.1002/jbio.202200181] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 07/27/2022] [Accepted: 08/05/2022] [Indexed: 06/15/2023]
Abstract
Understanding tumors and their microenvironment are essential for successful and accurate disease diagnosis. Tissue physiology and morphology are altered in tumors compared to healthy tissues, and there is a need to monitor tumors and their surrounding tissues, including blood vessels, non-invasively. This preliminary study utilizes a multimodal optical imaging system combining hyperspectral imaging (HSI) and three-dimensional (3D) optical profilometry (OP) to capture hyperspectral images and surface shapes of subcutaneously grown murine tumor models. Hyperspectral images are corrected with 3D OP data and analyzed using the inverse-adding doubling (IAD) method to extract tissue properties such as melanin volume fraction and oxygenation. Blood vessels are segmented using the B-COSFIRE algorithm from oxygenation maps. From 3D OP data, tumor volumes are calculated and compared to manual measurements using a vernier caliper. Results show that tumors can be distinguished from healthy tissue based on most extracted tissue parameters ( p < 0.05 ). Furthermore, blood oxygenation is 50% higher within the blood vessels than in the surrounding tissue, and tumor volumes calculated using 3D OP agree within 26% with manual measurements using a vernier caliper. Results suggest that combining HSI and OP could provide relevant quantitative information about tumors and improve the disease diagnosis.
Collapse
Affiliation(s)
- Tadej Tomanic
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Luka Rogelj
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
| | - Jost Stergar
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| | - Bostjan Markelc
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Health Sciences, University of Ljubljana, Ljubljana, Slovenia
| | - Tim Bozic
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Simona Kranjc Brezar
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
| | - Gregor Sersa
- Department of Experimental Oncology, Institute of Oncology Ljubljana, Ljubljana, Slovenia
- Faculty of Health Sciences, University of Ljubljana, Ljubljana, Slovenia
| | - Matija Milanic
- Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
- Jozef Stefan Institute, Ljubljana, Slovenia
| |
Collapse
|
39
|
Liu GS, Shenson JA, Farrell JE, Blevins NH. Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:016004. [PMID: 36726664 PMCID: PMC9884103 DOI: 10.1117/1.jbo.28.1.016004] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 01/06/2023] [Indexed: 05/09/2023]
Abstract
SIGNIFICANCE Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems. AIM Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI. APPROACH MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods. RESULTS Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy. CONCLUSIONS The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.
Collapse
Affiliation(s)
- George S. Liu
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
| | - Jared A. Shenson
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
| | - Joyce E. Farrell
- Stanford University, Department of Electrical Engineering, Stanford, California, United States
| | - Nikolas H. Blevins
- Stanford University, Department of Otolaryngology — Head and Neck Surgery, Palo Alto, California, United States
- Address all correspondence to Nikolas H. Blevins,
| |
Collapse
|
40
|
Intraoperative Assessment of Tumor Margins in Tissue Sections with Hyperspectral Imaging and Machine Learning. Cancers (Basel) 2022; 15:cancers15010213. [PMID: 36612208 PMCID: PMC9818424 DOI: 10.3390/cancers15010213] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Revised: 12/16/2022] [Accepted: 12/27/2022] [Indexed: 12/31/2022] Open
Abstract
The intraoperative assessment of tumor margins of head and neck cancer is crucial for complete tumor resection and patient outcome. The current standard is to take tumor biopsies during surgery for frozen section analysis by a pathologist after H&E staining. This evaluation is time-consuming, subjective, methodologically limited and underlies a selection bias. Optical methods such as hyperspectral imaging (HSI) are therefore of high interest to overcome these limitations. We aimed to analyze the feasibility and accuracy of an intraoperative HSI assessment on unstained tissue sections taken from seven patients with oral squamous cell carcinoma. Afterwards, the tissue sections were subjected to standard histopathological processing and evaluation. We trained different machine learning models on the HSI data, including a supervised 3D convolutional neural network to perform tumor detection. The results were congruent with the histopathological annotations. Therefore, this approach enables the delineation of tumor margins with artificial HSI-based histopathological information during surgery with high speed and accuracy on par with traditional intraoperative tumor margin assessment (Accuracy: 0.76, Specificity: 0.89, Sensitivity: 0.48). With this, we introduce HSI in combination with ML hyperspectral imaging as a potential new tool for intraoperative tumor margin assessment.
Collapse
|
41
|
Chen Y, Xing F, Li W, Sheng Y, Di Y, Gan Z, Liu C. Snapshot multi-frame parallel spectral holographic microscopy based on a reconfigurable optical comb. OPTICS LETTERS 2022; 47:6468-6471. [PMID: 36538464 DOI: 10.1364/ol.479993] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 11/20/2022] [Indexed: 06/17/2023]
Abstract
We present a snapshot multi-frame parallel holographic microscopy system through a reconfigurable optical comb source, which consists of a digital micromirror device (DMD) based spectrum filter system and a spectroscopic Michelson interferometric system. The proposed system allows arbitrarily tuning comb spacing and comb number, and the capturing of multi-frame images without overlap in one exposure. As a result, high-quality spectral holograms can be obtained with less acquisition time. The performance of the system is detailed in the experiment and 45-wavelengths holographic imaging for perovskite micro-platelets is conducted, which proves the system has the ability to realize high-performance four-dimensional (4D) imaging.
Collapse
|
42
|
Park HW, Choi JW, Choi JY, Joo KK, Kim NR. Investigation of the Hue-Wavelength Response of a CMOS RGB-Based Image Sensor. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22239497. [PMID: 36502198 PMCID: PMC9739397 DOI: 10.3390/s22239497] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 11/30/2022] [Accepted: 12/02/2022] [Indexed: 05/27/2023]
Abstract
In this study, a non-linear hue-wavelength (H-W) curve was investigated from 400 to 650 nm. To date, no study has reported on H-W relationship measurements, especially down to the 400 nm region. A digital camera mounted with complementary metal oxide semiconductor (CMOS) image sensors was used. The obtained digital images of the sample were based on an RGB-based imaging analysis rather than multispectral imaging or hyperspectral imaging. In this study, we focused on the raw image to reconstruct the H-W curve. In addition, several factors affecting the digital image, such as exposure time or international organization for standardization (ISO), were investigated. In addition, cross check of the H-W response using laser was performed. We expect that our method will be useful as an auxiliary method in the future for obtaining the fluor emission wavelength information.
Collapse
Affiliation(s)
- Hyeon-Woo Park
- Center for Precision Neutrino Research, Department of Physics, Chonnam National University, Yongbong-ro 77, Puk-gu, Gwangju 61186, Republic of Korea
| | - Ji-Won Choi
- Center for Precision Neutrino Research, Department of Physics, Chonnam National University, Yongbong-ro 77, Puk-gu, Gwangju 61186, Republic of Korea
| | - Ji-Young Choi
- Department of Fire Safety, Seoyeong University, Seogang-ro 1, Puk-gu, Gwangju 61268, Republic of Korea
| | - Kyung-Kwang Joo
- Center for Precision Neutrino Research, Department of Physics, Chonnam National University, Yongbong-ro 77, Puk-gu, Gwangju 61186, Republic of Korea
| | - Na-Ri Kim
- Center for Precision Neutrino Research, Department of Physics, Chonnam National University, Yongbong-ro 77, Puk-gu, Gwangju 61186, Republic of Korea
| |
Collapse
|
43
|
Wen YC, Wen S, Hsu L, Chi S. Irradiance Independent Spectrum Reconstruction from Camera Signals Using the Interpolation Method. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22218498. [PMID: 36366197 PMCID: PMC9656597 DOI: 10.3390/s22218498] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 10/28/2022] [Accepted: 11/01/2022] [Indexed: 05/25/2023]
Abstract
The spectrum of light captured by a camera can be reconstructed using the interpolation method. The reconstructed spectrum is a linear combination of the reference spectra, where the weighting coefficients are calculated from the signals of the pixel and the reference samples by interpolation. This method is known as the look-up table (LUT) method. It is irradiance-dependent due to the dependence of the reconstructed spectrum shape on the sample irradiance. Since the irradiance can vary in field applications, an irradiance-independent LUT (II-LUT) method is required to recover spectral reflectance. This paper proposes an II-LUT method to interpolate the spectrum in the normalized signal space. Munsell color chips irradiated with D65 were used as samples. Example cameras are a tricolor camera and a quadcolor camera. Results show that the proposed method can achieve the irradiance independent spectrum reconstruction and computation time saving at the expense of the recovered spectral reflectance error. Considering that the irradiance variation will introduce additional errors, the actual mean error using the II-LUT method might be smaller than that of the ID-LUT method. It is also shown that the proposed method outperformed the weighted principal component analysis method in both accuracy and computation speed.
Collapse
Affiliation(s)
- Yu-Che Wen
- Department of Electrophysics, National Yang Ming Chiao Tung University, No. 1001 University Road, Hsinchu 30010, Taiwan
| | - Senfar Wen
- Department of Electrical Engineering, Yuan Ze University, No. 135 Yuan-Tung Road, Taoyuan 32003, Taiwan
| | - Long Hsu
- Department of Electrophysics, National Yang Ming Chiao Tung University, No. 1001 University Road, Hsinchu 30010, Taiwan
| | - Sien Chi
- Department of Photonics, National Yang Ming Chiao Tung University, No. 1001 University Road, Hsinchu 30010, Taiwan
| |
Collapse
|
44
|
Zhang L, Huang D, Chen X, Zhu L, Chen X, Xie Z, Huang G, Gao J, Shi W, Cui G. Visible near-infrared hyperspectral imaging and supervised classification for the detection of small intestinal necrosis tissue in vivo. BIOMEDICAL OPTICS EXPRESS 2022; 13:6061-6080. [PMID: 36733734 PMCID: PMC9872898 DOI: 10.1364/boe.470202] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 09/23/2022] [Accepted: 09/26/2022] [Indexed: 06/18/2023]
Abstract
Complete recognition of necrotic areas during small bowel tissue resection remains challenging due to the lack of optimal intraoperative aid identification techniques. This research utilizes hyperspectral imaging techniques to automatically distinguish normal and necrotic areas of small intestinal tissue. Sample data were obtained from the animal model of small intestinal tissue of eight Japanese large-eared white rabbits developed by experienced physicians. A spectral library of normal and necrotic regions of small intestinal tissue was created and processed using six different supervised classification algorithms. The results show that hyperspectral imaging combined with supervised classification algorithms can be a suitable technique to automatically distinguish between normal and necrotic areas of small intestinal tissue. This new technique could aid physicians in objectively identify normal and necrotic areas of small intestinal tissue.
Collapse
Affiliation(s)
- LeChao Zhang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - DanFei Huang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - XiaoJing Chen
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - LiBin Zhu
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - XiaoQing Chen
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - ZhongHao Xie
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuangZao Huang
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - JunZhao Gao
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - Wen Shi
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuiHua Cui
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| |
Collapse
|
45
|
Aloupogianni E, Ichimura T, Hamada M, Ishikawa M, Murakami T, Sasaki A, Nakamura K, Kobayashi N, Obi T. Hyperspectral imaging for tumor segmentation on pigmented skin lesions. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:106007. [PMID: 36316301 PMCID: PMC9619132 DOI: 10.1117/1.jbo.27.10.106007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 10/10/2022] [Indexed: 06/16/2023]
Abstract
SIGNIFICANCE Malignant skin tumors, which include melanoma and nonmelanoma skin cancers, are the most prevalent type of malignant tumor. Gross pathology of pigmented skin lesions (PSL) remains manual, time-consuming, and heavily dependent on the expertise of the medical personnel. Hyperspectral imaging (HSI) can assist in the detection of tumors and evaluate the status of tumor margins by their spectral signatures. AIM Tumor segmentation of medical HSI data is a research field. The goal of this study is to propose a framework for HSI-based tumor segmentation of PSL. APPROACH An HSI dataset of 28 PSL was prepared. Two frameworks for data preprocessing and tumor segmentation were proposed. Models based on machine learning and deep learning were used at the core of each framework. RESULTS Cross-validation performance showed that pixel-wise processing achieves higher segmentation performance, in terms of the Jaccard coefficient. Simultaneous use of spatio-spectral features produced more comprehensive tumor masks. A three-dimensional Xception-based network achieved performance similar to state-of-the-art networks while allowing for more detailed detection of the tumor border. CONCLUSIONS Good performance was achieved for melanocytic lesions, but margins were difficult to detect in some cases of basal cell carcinoma. The frameworks proposed in this study could be further improved for robustness against different pathologies and detailed delineation of tissue margins to facilitate computer-assisted diagnosis during gross pathology.
Collapse
Affiliation(s)
- Eleni Aloupogianni
- Tokyo Institute of Technology, Department of Information and Communications Engineering, Meguro, Japan
| | - Takaya Ichimura
- Saitama Medical University Moroyama Campus, Department of Pathology, Faculty of Medicine, Iruma, Japan
| | - Mei Hamada
- Saitama Medical University Moroyama Campus, Department of Pathology, Faculty of Medicine, Iruma, Japan
| | - Masahiro Ishikawa
- Saitama Medical University Hidaka Campus, Faculty of Health and Medical Care, Hidaka, Japan
| | - Takuo Murakami
- Saitama Medical University Moroyama Campus, Department of Dermatology, Faculty of Medicine, Iruma, Japan
| | - Atsushi Sasaki
- Saitama Medical University Moroyama Campus, Department of Pathology, Faculty of Medicine, Iruma, Japan
| | - Koichiro Nakamura
- Saitama Medical University Moroyama Campus, Department of Dermatology, Faculty of Medicine, Iruma, Japan
| | - Naoki Kobayashi
- Saitama Medical University Hidaka Campus, Faculty of Health and Medical Care, Hidaka, Japan
| | - Takashi Obi
- Tokyo Institute of Technology, Department of Information and Communications Engineering, Meguro, Japan
- Tokyo Institute of Technology, Institute of Innovative Research, Yokohama, Japan
| |
Collapse
|
46
|
Juntunen C, Abramczyk AR, Woller IM, Sung Y. Hyperspectral three-dimensional absorption imaging using snapshot optical tomography. PHYSICAL REVIEW APPLIED 2022; 18:034055. [PMID: 37274485 PMCID: PMC10237288 DOI: 10.1103/physrevapplied.18.034055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Hyperspectral imaging (HSI) records a series of two-dimensional (2D) images for different wavelengths to provide the chemical fingerprint at each pixel. Combining HSI with a tomographic data acquisition method, we can obtain the chemical fingerprint of a sample at each point in three-dimensional (3D) space. The so-called 3D HSI typically suffers from low imaging throughput due to the requirement of scanning the wavelength and rotating the beam or sample. In this paper we present an optical system which captures the entire four-dimensional (4D), i.e., 3D structure and 1D spectrum, dataset of a sample with the same throughput of conventional HSI systems. Our system works by combining snapshot projection optical tomography (SPOT) which collects multiple projection images with a single snapshot, and Fourier-transform spectroscopy (FTS) which results in superior spectral resolution by collecting and processing a series of interferogram images. Using this hyperspectral SPOT system we imaged the volumetric absorbance of dyed polystyrene microbeads, oxygenated red blood cells (RBCs), and deoxygenated RBCs. The 4D optical system demonstrated in this paper provides a tool for high-throughput chemical imaging of complex microscopic specimens.
Collapse
Affiliation(s)
- Cory Juntunen
- College of Engineering and Applied Science, University of Wisconsin, Milwaukee, Wisconsin 53211, USA
| | - Andrew R. Abramczyk
- College of Engineering and Applied Science, University of Wisconsin, Milwaukee, Wisconsin 53211, USA
| | - Isabel M. Woller
- College of Health Sciences, University of Wisconsin, Milwaukee, Wisconsin 53211, USA
| | - Yongjin Sung
- College of Engineering and Applied Science, University of Wisconsin, Milwaukee, Wisconsin 53211, USA
| |
Collapse
|
47
|
Wen YC, Wen S, Hsu L, Chi S. Spectral Reflectance Recovery from the Quadcolor Camera Signals Using the Interpolation and Weighted Principal Component Analysis Methods. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22166288. [PMID: 36016049 PMCID: PMC9416231 DOI: 10.3390/s22166288] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Revised: 08/14/2022] [Accepted: 08/19/2022] [Indexed: 05/25/2023]
Abstract
The recovery of surface spectral reflectance using the quadcolor camera was numerically studied. Assume that the RGB channels of the quadcolor camera are the same as the Nikon D5100 tricolor camera. The spectral sensitivity of the fourth signal channel was tailored using a color filter. Munsell color chips were used as reflective surfaces. When the interpolation method or the weighted principal component analysis (wPCA) method is used to reconstruct spectra, using the quadcolor camera can effectively reduce the mean spectral error of the test samples compared to using the tricolor camera. Except for computation time, the interpolation method outperforms the wPCA method in spectrum reconstruction. A long-pass optical filter can be applied to the fourth channel for reducing the mean spectral error. A short-pass optical filter can be applied to the fourth channel for reducing the mean color difference, but the mean spectral error will be larger. Due to the small color difference, the quadcolor camera using an optimized short-pass filter may be suitable as an imaging colorimeter. It was found that an empirical design rule to keep the color difference small is to reduce the error in fitting the color-matching functions using the camera spectral sensitivity functions.
Collapse
Affiliation(s)
- Yu-Che Wen
- Department of Electrophysics, National Yang Ming Chiao Tung University, No. 1001 University Road, Hsinchu 30010, Taiwan
| | - Senfar Wen
- Department of Electrical Engineering, Yuan Ze University, No. 135 Yuan-Tung Road, Taoyuan 320, Taiwan
| | - Long Hsu
- Department of Electrophysics, National Yang Ming Chiao Tung University, No. 1001 University Road, Hsinchu 30010, Taiwan
| | - Sien Chi
- Department of Photonics, National Yang Ming Chiao Tung University, No. 1001 University Road, Hsinchu 30010, Taiwan
| |
Collapse
|
48
|
Chalopin C, Nickel F, Pfahl A, Köhler H, Maktabi M, Thieme R, Sucher R, Jansen-Winkeln B, Studier-Fischer A, Seidlitz S, Maier-Hein L, Neumuth T, Melzer A, Müller-Stich BP, Gockel I. [Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2022; 93:940-947. [PMID: 35798904 DOI: 10.1007/s00104-022-01677-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/08/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article. OBJECTIVE What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery? MATERIAL AND METHODS Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors. RESULTS In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91. DISCUSSION This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.
Collapse
Affiliation(s)
- Claire Chalopin
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland.
| | - Felix Nickel
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - René Thieme
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Robert Sucher
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| | - Boris Jansen-Winkeln
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
- Abteilung für Allgemein‑, Viszeral- und Onkologische Chirurgie, Klinikum St. Georg Leipzig, Leipzig, Deutschland
| | - Alexander Studier-Fischer
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Silvia Seidlitz
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, Deutsches Krebsforschungszentrum, Heidelberg, Deutschland
| | - Thomas Neumuth
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Andreas Melzer
- Innovation Center Computer Assisted Surgery, Universität Leipzig, Semmelweisstr. 14, 04103, Leipzig, Deutschland
| | - Beat Peter Müller-Stich
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsklinikum Heidelberg, Heidelberg, Deutschland
| | - Ines Gockel
- Klinik und Poliklinik für Viszeral‑, Transplantations‑, Thorax- und Gefäßchirurgie, Universitätsklinikum Leipzig, Leipzig, Deutschland
| |
Collapse
|
49
|
Segmentation and Recognition of the Pathological Features of Squamous Cell Carcinoma of the Skin Based on Multispectral Imaging. J Clin Med 2022; 11:jcm11133815. [PMID: 35807100 PMCID: PMC9267474 DOI: 10.3390/jcm11133815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 06/23/2022] [Accepted: 06/29/2022] [Indexed: 02/04/2023] Open
Abstract
Cutaneous squamous cell carcinoma (cSCC) is one of the most common skin cancers, a definitive diagnosis of cSCC is crucial to prevent patients from missing out on treatment. The gold standard for the diagnosis of cSCC is still pathological biopsy. Currently, its diagnostic efficiency and accuracy largely depend on the experience of pathologists. Here, we present a simple, fast, and robust technique, a microscopic multispectral imaging system based on LED illumination, to diagnose cSCC qualitatively and quantitatively. The adaptive threshold segmentation method was used to segment the multispectral images into characteristic structures. There was a statistically significant difference between the average nucleocytoplasmic ratio of normal skin (4.239%) and cSCC tissues (15.607%) (p < 0.01), and the keratin pearls cSCC have well-defined qualitative features. These results show that the qualitative and quantitative features obtained from multispectral imaging can be used to comprehensively determine whether or not the tissue is cancerous. This work has significant implications for the development of a low-cost and easy-to-use device, which can not only reduce the complexity of pathological diagnosis but can also achieve the goal of convenient digital staining and access to critical histological information.
Collapse
|
50
|
Sun J, Wu Z, Wang L, Wu P, Li M, Yao Q, Yao G. Band selection for mapping chromophores of skin tissue. JOURNAL OF BIOPHOTONICS 2022; 15:e202200038. [PMID: 35293142 DOI: 10.1002/jbio.202200038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 02/26/2022] [Accepted: 03/14/2022] [Indexed: 06/14/2023]
Abstract
A numerical approach has been proposed to identify bands for optimally estimating the concentration of three types of viable chromophores within biological tissue. The bands are determined according to the condition number of absorption matrix associated with the attenuation coefficients of chromophores. The effectiveness of different sets of selected band combination was verified by using the spectral reflectance images of skin tissue acquired from standard forearm vascular occlusion tests via a spectroradiometer. Experimental results demonstrated that the concentration of chromophores within skin tissue could be estimated correctly and robustly only when the bands were deliberately selected.
Collapse
Affiliation(s)
- Jiuai Sun
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- School of Medical Imaging, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Zhonghang Wu
- School of Medical Imaging, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Le Wang
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Peiyu Wu
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Min Li
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Qi Yao
- Academy for Engineering and Technology, Fudan University, Shanghai, China
- Department of Research and Development, Zhongshan Fudan Joint Innovation Center, Guangdong, China
| | - Guangyu Yao
- Department of Thoracic Surgery, Zhongshan Hospital, Shanghai, China
| |
Collapse
|