1
|
Jong LJS, Post AL, Geldof F, Dashtbozorg B, Ruers TJM, Sterenborg HJCM. Separating Surface Reflectance from Volume Reflectance in Medical Hyperspectral Imaging. Diagnostics (Basel) 2024; 14:1812. [PMID: 39202300 PMCID: PMC11353750 DOI: 10.3390/diagnostics14161812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2024] [Revised: 08/13/2024] [Accepted: 08/16/2024] [Indexed: 09/03/2024] Open
Abstract
Hyperspectral imaging has shown great promise for diagnostic applications, particularly in cancer surgery. However, non-bulk tissue-related spectral variations complicate the data analysis. Common techniques, such as standard normal variate normalization, often lead to a loss of amplitude and scattering information. This study investigates a novel approach to address these spectral variations in hyperspectral images of optical phantoms and excised human breast tissue. Our method separates surface and volume reflectance, hypothesizing that spectral variability arises from significant variations in surface reflectance across pixels. An illumination setup was developed to measure samples with a hyperspectral camera from different axial positions but with identical zenith angles. This configuration, combined with a novel data analysis approach, allows for the estimation and separation of surface reflectance for each direction and volume reflectance across all directions. Validated with optical phantoms, our method achieved an 83% reduction in spectral variability. Its functionality was further demonstrated in excised human breast tissue. Our method effectively addresses variations caused by surface reflectance or glare while conserving surface reflectance information, which may enhance sample analysis and evaluation. It benefits samples with unknown refractive index spectra and can be easily adapted and applied across a wide range of fields where hyperspectral imaging is used.
Collapse
Affiliation(s)
- Lynn-Jade S. Jong
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Anouk L. Post
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | - Freija Geldof
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Behdad Dashtbozorg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| | - Theo J. M. Ruers
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | | |
Collapse
|
2
|
Schmidt VM, Zelger P, Wöss C, Fodor M, Hautz T, Schneeberger S, Huck CW, Arora R, Brunner A, Zelger B, Schirmer M, Pallua JD. Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains. Heliyon 2024; 10:e25844. [PMID: 38375262 PMCID: PMC10875450 DOI: 10.1016/j.heliyon.2024.e25844] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 01/30/2024] [Accepted: 02/02/2024] [Indexed: 02/21/2024] Open
Abstract
In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.
Collapse
Affiliation(s)
- Verena-Maria Schmidt
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - Philipp Zelger
- University Clinic for Hearing, Voice and Speech Disorders, Medical University of Innsbruck, Anichstrasse 35, 6020 Innsbruck, Austria
| | - Claudia Wöss
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - Margot Fodor
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Theresa Hautz
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Stefan Schneeberger
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Christian Wolfgang Huck
- Institute of Analytical Chemistry and Radiochemistry, University of Innsbruck, 6020 Innsbruck, Austria
| | - Rohit Arora
- Department of Orthopaedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| | - Andrea Brunner
- Institute of Pathology, Neuropathology, and Molecular Pathology, Medical University of Innsbruck, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - Bettina Zelger
- Institute of Pathology, Neuropathology, and Molecular Pathology, Medical University of Innsbruck, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - Michael Schirmer
- Department of Internal Medicine, Clinic II, Medical University of Innsbruck, Anichstrasse 35, 6020 Innsbruck, Austria
| | - Johannes Dominikus Pallua
- Department of Orthopaedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| |
Collapse
|
3
|
Witteveen M, Sterenborg HJCM, van Leeuwen TG, Aalders MCG, Ruers TJM, Post AL. Comparison of preprocessing techniques to reduce nontissue-related variations in hyperspectral reflectance imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:106003. [PMID: 36207772 PMCID: PMC9541333 DOI: 10.1117/1.jbo.27.10.106003] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Accepted: 08/23/2022] [Indexed: 06/16/2023]
Abstract
SIGNIFICANCE Hyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples. AIM To compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types. APPROACH We compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min-max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope-while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast. CONCLUSIONS Preprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min-max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable.
Collapse
Affiliation(s)
- Mark Witteveen
- the Netherlands Cancer Institute, Surgical Oncology, Amsterdam, The Netherlands
- University of Twente, Science and Technology, Nanobiophysics, Enschede, The Netherlands
| | - Henricus J. C. M. Sterenborg
- the Netherlands Cancer Institute, Surgical Oncology, Amsterdam, The Netherlands
- Amsterdam UMC, University of Amsterdam, Cancer Center Amsterdam, Amsterdam Cardiovascular Sciences, Department of Biomedical Engineering and Physics, Amsterdam, The Netherlands
| | - Ton G. van Leeuwen
- Amsterdam UMC, University of Amsterdam, Cancer Center Amsterdam, Amsterdam Cardiovascular Sciences, Department of Biomedical Engineering and Physics, Amsterdam, The Netherlands
| | - Maurice C. G. Aalders
- Amsterdam UMC, University of Amsterdam, Cancer Center Amsterdam, Amsterdam Cardiovascular Sciences, Department of Biomedical Engineering and Physics, Amsterdam, The Netherlands
- University of Amsterdam, Co van Ledden Hulsebosch Center, Amsterdam, The Netherlands
| | - Theo J. M. Ruers
- the Netherlands Cancer Institute, Surgical Oncology, Amsterdam, The Netherlands
- University of Twente, Science and Technology, Nanobiophysics, Enschede, The Netherlands
| | - Anouk L. Post
- the Netherlands Cancer Institute, Surgical Oncology, Amsterdam, The Netherlands
- Amsterdam UMC, University of Amsterdam, Cancer Center Amsterdam, Amsterdam Cardiovascular Sciences, Department of Biomedical Engineering and Physics, Amsterdam, The Netherlands
| |
Collapse
|
4
|
|
5
|
Kho E, Dashtbozorg B, de Boer LL, Van de Vijver KK, Sterenborg HJCM, Ruers TJM. Broadband hyperspectral imaging for breast tumor detection using spectral and spatial information. BIOMEDICAL OPTICS EXPRESS 2019; 10:4496-4515. [PMID: 31565506 PMCID: PMC6757478 DOI: 10.1364/boe.10.004496] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2019] [Revised: 07/26/2019] [Accepted: 07/29/2019] [Indexed: 05/20/2023]
Abstract
Complete tumor removal during breast-conserving surgery remains challenging due to the lack of optimal intraoperative margin assessment techniques. Here, we use hyperspectral imaging for tumor detection in fresh breast tissue. We evaluated different wavelength ranges and two classification algorithms; a pixel-wise classification algorithm and a convolutional neural network that combines spectral and spatial information. The highest classification performance was obtained using the full wavelength range (450-1650 nm). Adding spatial information mainly improved the differentiation of tissue classes within the malignant and healthy classes. High sensitivity and specificity were accomplished, which offers potential for hyperspectral imaging as a margin assessment technique to improve surgical outcome.
Collapse
Affiliation(s)
- Esther Kho
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, Netherlands
| | - Behdad Dashtbozorg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, Netherlands
- Department of Biomedical Engineering, Eindhoven University of Technology, 5600MB Eindhoven, Netherlands
| | - Lisanne L. de Boer
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, Netherlands
| | - Koen K. Van de Vijver
- Department of Pathology, Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, Netherlands
- Department of Pathology, Ghent University Hospital, Corneel Heymanslaan 10, 9000 Gent, Belgium
| | - Henricus J. C. M. Sterenborg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, Meibergdreef 9, 1105AZ Amsterdam, Netherlands
| | - Theo J. M. Ruers
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, Netherlands
- Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522NB Enschede, Netherlands
| |
Collapse
|
6
|
Halicek M, Little JV, Wang X, Chen AY, Fei B. Optical biopsy of head and neck cancer using hyperspectral imaging and convolutional neural networks. JOURNAL OF BIOMEDICAL OPTICS 2019; 24:1-9. [PMID: 30891966 PMCID: PMC6975184 DOI: 10.1117/1.jbo.24.3.036007] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2018] [Accepted: 01/14/2019] [Indexed: 05/21/2023]
Abstract
For patients undergoing surgical cancer resection of squamous cell carcinoma (SCCa), cancer-free surgical margins are essential for good prognosis. We developed a method to use hyperspectral imaging (HSI), a noncontact optical imaging modality, and convolutional neural networks (CNNs) to perform an optical biopsy of ex-vivo, surgical gross-tissue specimens, collected from 21 patients undergoing surgical cancer resection. Using a cross-validation paradigm with data from different patients, the CNN can distinguish SCCa from normal aerodigestive tract tissues with an area under the receiver operator curve (AUC) of 0.82. Additionally, normal tissue from the upper aerodigestive tract can be subclassified into squamous epithelium, muscle, and gland with an average AUC of 0.94. After separately training on thyroid tissue, the CNN can differentiate between thyroid carcinoma and normal thyroid with an AUC of 0.95, 92% accuracy, 92% sensitivity, and 92% specificity. Moreover, the CNN can discriminate medullary thyroid carcinoma from benign multinodular goiter (MNG) with an AUC of 0.93. Classical-type papillary thyroid carcinoma is differentiated from MNG with an AUC of 0.91. Our preliminary results demonstrate that an HSI-based optical biopsy method using CNNs can provide multicategory diagnostic information for normal and cancerous head-and-neck tissue, and more patient data are needed to fully investigate the potential and reliability of the proposed technique.
Collapse
Affiliation(s)
- Martin Halicek
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Emory University and Georgia Institute of Technology, Department of Biomedical Engineering, Atlanta, Georgia, United States
| | - James V. Little
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, Georgia, United States
| | - Xu Wang
- Emory University School of Medicine, Department of Hematology and Medical Oncology, Atlanta, Georgia, United States
| | - Amy Y. Chen
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Emory University School of Medicine, Department of Radiology and Imaging Sciences, Atlanta, Georgia, United States
- University of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
- Address all correspondence to Baowei Fei, E-mail:
| |
Collapse
|
7
|
Pardo A, Gutiérrez-Gutiérrez JA, Lihacova I, López-Higuera JM, Conde OM. On the spectral signature of melanoma: a non-parametric classification framework for cancer detection in hyperspectral imaging of melanocytic lesions. BIOMEDICAL OPTICS EXPRESS 2018; 9:6283-6301. [PMID: 31065429 PMCID: PMC6491016 DOI: 10.1364/boe.9.006283] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2018] [Revised: 10/16/2018] [Accepted: 10/17/2018] [Indexed: 05/20/2023]
Abstract
Early detection and diagnosis is a must in secondary prevention of melanoma and other cancerous lesions of the skin. In this work, we present an online, reservoir-based, non-parametric estimation and classification model that allows for this functionality on pigmented lesions, such that detection thresholding can be tuned to maximize accuracy and/or minimize overall false negative rates. This system has been tested in a dataset consisting of 116 patients and a total of 124 hyperspectral images of nevi, raised nevi and melanomas, detecting up to 100% of the suspicious lesions at the expense of some false positives.
Collapse
Affiliation(s)
- Arturo Pardo
- Grupo de Ingeniería Fotónica, TEISA, Universidad de Cantabria, Avenida Los Castros S/N, 39006, Cantabria,
Spain
| | - José A. Gutiérrez-Gutiérrez
- Grupo de Ingeniería Fotónica, TEISA, Universidad de Cantabria, Avenida Los Castros S/N, 39006, Cantabria,
Spain
| | - I. Lihacova
- Biophotonics Laboratory, Institute of Atomic Physics and Spectroscopy, Raina Blvd. 19, Riga, LV-1586,
Latvia
| | - José M. López-Higuera
- Grupo de Ingeniería Fotónica, TEISA, Universidad de Cantabria, Avenida Los Castros S/N, 39006, Cantabria,
Spain
- Centro de Investigación Biomédica en Red – Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Cantabria,
Spain
- Instituto de Investigación Sanitaria Valdecilla (IDIVAL), Calle Cardenal Herrera Oria S/N, 39011 Santander, Cantabria,
Spain
| | - Olga M. Conde
- Grupo de Ingeniería Fotónica, TEISA, Universidad de Cantabria, Avenida Los Castros S/N, 39006, Cantabria,
Spain
- Centro de Investigación Biomédica en Red – Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Cantabria,
Spain
- Instituto de Investigación Sanitaria Valdecilla (IDIVAL), Calle Cardenal Herrera Oria S/N, 39011 Santander, Cantabria,
Spain
| |
Collapse
|
8
|
McClatchy DM, Rizzo EJ, Meganck J, Kempner J, Vicory J, Wells WA, Paulsen KD, Pogue BW. Calibration and analysis of a multimodal micro-CT and structured light imaging system for the evaluation of excised breast tissue. Phys Med Biol 2017; 62:8983-9000. [PMID: 29048330 PMCID: PMC5729028 DOI: 10.1088/1361-6560/aa94b6] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
A multimodal micro-computed tomography (CT) and multi-spectral structured light imaging (SLI) system is introduced and systematically analyzed to test its feasibility to aid in margin delineation during breast conserving surgery (BCS). Phantom analysis of the micro-CT yielded a signal-to-noise ratio of 34, a contrast of 1.64, and a minimum detectable resolution of 240 μm for a 1.2 min scan. The SLI system, spanning wavelengths 490 nm to 800 nm and spatial frequencies up to 1.37 [Formula: see text], was evaluated with aqueous tissue simulating phantoms having variations in particle size distribution, scatter density, and blood volume fraction. The reduced scattering coefficient, [Formula: see text] and phase function parameter, γ, were accurately recovered over all wavelengths independent of blood volume fractions from 0% to 4%, assuming a flat sample geometry perpendicular to the imaging plane. The resolution of the optical system was tested with a step phantom, from which the modulation transfer function was calculated yielding a maximum resolution of 3.78 cycles per mm. The three dimensional spatial co-registration between the CT and optical imaging space was tested and shown to be accurate within 0.7 mm. A freshly resected breast specimen, with lobular carcinoma, fibrocystic disease, and adipose, was imaged with the system. The micro-CT provided visualization of the tumor mass and its spiculations, and SLI yielded superficial quantification of light scattering parameters for the malignant and benign tissue types. These results appear to be the first demonstration of SLI combined with standard medical tomography for imaging excised tumor specimens. While further investigations are needed to determine and test the spectral, spatial, and CT features required to classify tissue, this study demonstrates the ability of multimodal CT/SLI to quantify, visualize, and spatially navigate breast tumor specimens, which could potentially aid in the assessment of tumor margin status during BCS.
Collapse
Affiliation(s)
- David M McClatchy
- Thayer School of Engineering, Dartmouth College, 14 Engineering Dr., Hanover, NH 03755, United States of America
| | | | | | | | | | | | | | | |
Collapse
|