On the spectral signature of melanoma: a non-parametric classification framework for cancer detection in hyperspectral imaging of melanocytic lesions

Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains

In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.

Acceleration of Hyperspectral Skin Cancer Image Classification through Parallel Machine-Learning Methods

Hyperspectral imaging (HSI) has become a very compelling technique in different scientific areas; indeed, many researchers use it in the fields of remote sensing, agriculture, forensics, and medicine. In the latter, HSI plays a crucial role as a diagnostic support and for surgery guidance. However, the computational effort in elaborating hyperspectral data is not trivial. Furthermore, the demand for detecting diseases in a short time is undeniable. In this paper, we take up this challenge by parallelizing three machine-learning methods among those that are the most intensively used: Support Vector Machine (SVM), Random Forest (RF), and eXtreme Gradient Boosting (XGB) algorithms using the Compute Unified Device Architecture (CUDA) to accelerate the classification of hyperspectral skin cancer images. They all showed a good performance in HS image classification, in particular when the size of the dataset is limited, as demonstrated in the literature. We illustrate the parallelization techniques adopted for each approach, highlighting the suitability of Graphical Processing Units (GPUs) to this aim. Experimental results show that parallel SVM and XGB algorithms significantly improve the classification times in comparison with their serial counterparts.

Effects of dimension reduction of hyperspectral images in skin gross pathology

BACKGROUND

Hyperspectral imaging (HSI) is an emerging modality for the gross pathology of the skin. Spectral signatures of HSI could discriminate malignant from benign tissue. Because of inherent redundancies in HSI and in order to facilitate the use of deep-learning models, dimension reduction is a common preprocessing step. The effects of dimension reduction choice, training scope, and number of retained dimensions have not been evaluated on skin HSI for segmentation tasks.

MATERIALS AND METHODS

An in-house dataset of HSI signatures from pigmented skin lesions was prepared and labeled with histology. Eleven different dimension reduction methods were used as preprocessing for tumor margin detection with support vector machines. Cluster-wise principal component analysis (ClusterPCA), a new variant of PCA, was proposed. The scope of application for dimension reduction was also investigated.

RESULTS

The components produced by ClusterPCA show good agreement with the expected optical properties of skin chromophores. Random forest importance performed best during classification. However, all methods suffered from low sensitivity and generalization.

CONCLUSION

Investigation of more complex reduction and segmentation schemes with emphasis on the nature of HSI and optical properties of the skin is necessary. Insights on dimension reduction for skin tissue could facilitate the development of HSI-based systems for cancer margin detection at gross level.

Hyperspectral imaging for tumor segmentation on pigmented skin lesions

SIGNIFICANCE

Malignant skin tumors, which include melanoma and nonmelanoma skin cancers, are the most prevalent type of malignant tumor. Gross pathology of pigmented skin lesions (PSL) remains manual, time-consuming, and heavily dependent on the expertise of the medical personnel. Hyperspectral imaging (HSI) can assist in the detection of tumors and evaluate the status of tumor margins by their spectral signatures.

AIM

Tumor segmentation of medical HSI data is a research field. The goal of this study is to propose a framework for HSI-based tumor segmentation of PSL.

APPROACH

An HSI dataset of 28 PSL was prepared. Two frameworks for data preprocessing and tumor segmentation were proposed. Models based on machine learning and deep learning were used at the core of each framework.

RESULTS

Cross-validation performance showed that pixel-wise processing achieves higher segmentation performance, in terms of the Jaccard coefficient. Simultaneous use of spatio-spectral features produced more comprehensive tumor masks. A three-dimensional Xception-based network achieved performance similar to state-of-the-art networks while allowing for more detailed detection of the tumor border.

CONCLUSIONS

Good performance was achieved for melanocytic lesions, but margins were difficult to detect in some cases of basal cell carcinoma. The frameworks proposed in this study could be further improved for robustness against different pathologies and detailed delineation of tissue margins to facilitate computer-assisted diagnosis during gross pathology.

Comparison of preprocessing techniques to reduce nontissue-related variations in hyperspectral reflectance imaging

SIGNIFICANCE

Hyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples.

AIM

To compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types.

APPROACH

We compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min-max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope-while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast.

CONCLUSIONS

Preprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min-max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable.

Visible and Near-Infrared hyperspectral imaging (HSI) can reliably quantify CD3 and CD45 positive inflammatory cells in myocarditis: Pilot study on formalin-fixed paraffin-embedded specimens from myocard obtained during autopsy

INTRODUCTION

To implement Hyperspectral Imaging (HSI) as a tool for quantifying inflammatory cells in tissue specimens by the example of myocarditis in a collective of forensic patients.

MATERIAL AND METHODS

44 consecutive patients with suspected myocardial inflammation at autopsy, diagnosed between 2013 and 2018 at the Institute of ForensicMedicine, Medical University of Innsbruck, were selected for this study. Using the IMEC SNAPSCAN camera, visible and near infrared hyperspectral images were collected from slides stained with CD3 and CD45 to assess quantity and spatial distribution of positive cells. Results were compared with visual assessment (VA) and conventional digital image analysis (DIA).

RESULTS

Finally, specimens of 40 patients were evaluated, of whom 36 patients (90%) suffered from myocarditis, two patients (5%) had suspected healing/healed myocarditis, and two did no have myocarditis (5%). The amount of CD3 and CD45 positive cells did not differ significantly between VA, HSI, and DIA (p^VA/HSI/DIA = 0.46 for CD3 and 0.81 for CD45). Coheńs Kappa showed a very high correlation between VA versus HSI, VA versus DIA, and HSI versus DIA for CD3 (Coheńs Kappa = 0.91, 1.00, and 0.91, respectively). For CD45 an almost as high correlation was seen for VA versus HSI and HSI versus DIA (Coheńs Kappa = 0.75 and 0.70) and VA versus DIA (Coheńs Kappa = 0.89).

CONCLUSION

HSI is a reliable and objective method to count inflammatory cells in tissue slides of suspected myocarditis. Implementation of HSI in digital pathology might further expand the possibility of a sophisticated method.

Hyperspectral and multispectral image processing for gross-level tumor detection in skin lesions: a systematic review

SIGNIFICANCE

Skin cancer is one of the most prevalent cancers worldwide. In the advent of medical digitization and telepathology, hyper/multispectral imaging (HMSI) allows for noninvasive, nonionizing tissue evaluation at a macroscopic level.

AIM

We aim to summarize proposed frameworks and recent trends in HMSI-based classification and segmentation of gross-level skin tissue.

APPROACH

A systematic review was performed, targeting HMSI-based systems for the classification and segmentation of skin lesions during gross pathology, including melanoma, pigmented lesions, and bruises. The review adhered to the 2020 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. For eligible reports published from 2010 to 2020, trends in HMSI acquisition, preprocessing, and analysis were identified.

RESULTS

HMSI-based frameworks for skin tissue classification and segmentation vary greatly. Most reports implemented simple image processing or machine learning, due to small training datasets. Methodologies were evaluated on heavily curated datasets, with the majority targeting melanoma detection. The choice of preprocessing scheme influenced the performance of the system. Some form of dimension reduction is commonly applied to avoid redundancies that are inherent in HMSI systems.

CONCLUSIONS

To use HMSI for tumor margin detection in practice, the focus of system evaluation should shift toward the explainability and robustness of the decision-making process.

Differentiating Malignant from Benign Pigmented or Non-Pigmented Skin Tumours-A Pilot Study on 3D Hyperspectral Imaging of Complex Skin Surfaces and Convolutional Neural Networks

Several optical imaging techniques have been developed to ease the burden of skin cancer disease on our health care system. Hyperspectral images can be used to identify biological tissues by their diffuse reflected spectra. In this second part of a three-phase pilot study, we used a novel hand-held SICSURFIS Spectral Imager with an adaptable field of view and target-wise selectable wavelength channels to provide detailed spectral and spatial data for lesions on complex surfaces. The hyperspectral images (33 wavelengths, 477–891 nm) provided photometric data through individually controlled illumination modules, enabling convolutional networks to utilise spectral, spatial, and skin-surface models for the analyses. In total, 42 lesions were studied: 7 melanomas, 13 pigmented and 7 intradermal nevi, 10 basal cell carcinomas, and 5 squamous cell carcinomas. All lesions were excised for histological analyses. A pixel-wise analysis provided map-like images and classified pigmented lesions with a sensitivity of 87% and a specificity of 93%, and 79% and 91%, respectively, for non-pigmented lesions. A majority voting analysis, which provided the most probable lesion diagnosis, diagnosed 41 of 42 lesions correctly. This pilot study indicates that our non-invasive hyperspectral imaging system, which involves shape and depth data analysed by convolutional neural networks, is feasible for differentiating between malignant and benign pigmented and non-pigmented skin tumours, even on complex skin surfaces.

Hyperspectral imaging and robust statistics in non-melanoma skin cancer analysis

Non-Melanoma skin cancer is one of the most frequent types of cancer. Early detection is encouraged so as to ensure the best treatment, Hyperspectral imaging is a promising technique for non-invasive inspection of skin lesions, however, the optimal wavelengths for these purposes are yet to be conclusively determined. A visible-near infrared hyperspectral camera with an ad-hoc built platform was used for image acquisition in the present study. Robust statistical techniques were used to conclude an optimal range between 573.45 and 779.88 nm to distinguish between healthy and non-healthy skin. Wavelengths between 429.16 and 520.17 nm were additionally found to be optimal for the differentiation between cancer types.

Modeling and Synthesis of Breast Cancer Optical Property Signatures With Generative Models

Is it possible to find deterministic relationships between optical measurements and pathophysiology in an unsupervised manner and based on data alone? Optical property quantification is a rapidly growing biomedical imaging technique for characterizing biological tissues that shows promise in a range of clinical applications, such as intraoperative breast-conserving surgery margin assessment. However, translating tissue optical properties to clinical pathology information is still a cumbersome problem due to, amongst other things, inter- and intrapatient variability, calibration, and ultimately the nonlinear behavior of light in turbid media. These challenges limit the ability of standard statistical methods to generate a simple model of pathology, requiring more advanced algorithms. We present a data-driven, nonlinear model of breast cancer pathology for real-time margin assessment of resected samples using optical properties derived from spatial frequency domain imaging data. A series of deep neural network models are employed to obtain sets of latent embeddings that relate optical data signatures to the underlying tissue pathology in a tractable manner. These self-explanatory models can translate absorption and scattering properties measured from pathology, while also being able to synthesize new data. The method was tested on a total of 70 resected breast tissue samples containing 137 regions of interest, achieving rapid optical property modeling with errors only limited by current semi-empirical models, allowing for mass sample synthesis and providing a systematic understanding of dataset properties, paving the way for deep automated margin assessment algorithms using structured light imaging or, in principle, any other optical imaging technique seeking modeling. Code is available.

IDCube Lite: Free Interactive Discovery Cube software for multi- and hyperspectral applications

Multi- and hyperspectral imaging modalities encompass a growing number of spectral techniques that find many applications in geospatial, biomedical, machine vision and other fields. The rapidly increasing number of applications requires convenient easy-to-navigate software that can be used by new and experienced users to analyse data, and develop, apply and deploy novel algorithms. Herein, we present our platform, IDCube Lite, an Interactive Discovery Cube that performs essential operations in hyperspectral data analysis to realise the full potential of spectral imaging. The strength of the software lies in its interactive features that enable the users to optimise parameters and obtain visual input for the user in a way not previously accessible with other software packages. The entire software can be operated without any prior programming skills allowing interactive sessions of raw and processed data. IDCube Lite, a free version of the software described in the paper, has many benefits compared to existing packages and offers structural flexibility to discover new, hidden features that allow users to integrate novel computational methods.

Hyperspectral imaging as a diagnostic tool to differentiate between amalgam tattoos and other dark pigmented intraoral lesions

The goal of this project is to identify any in-depth benefits and drawbacks in the diagnosis of amalgam tattoos and other pigmented intraoral lesions using hyperspectral imagery collected from amalgam tattoos, benign, and malignant melanocytic neoplasms. Software solutions capable of classifying pigmented lesions of the skin already exist, but conventional red, green and blue images may be reaching an upper limit in their performance. Emerging technologies, such as hyperspectral imaging (HSI) utilize more than a hundred, continuous data channels, while also collecting data in the infrared. A total of 18 paraffin-embedded human tissue specimens of dark pigmented intraoral lesions (including the lip) were analyzed using visible and near-infrared (VIS-NIR) hyperspectral imagery obtained from HE-stained histopathological slides. Transmittance data were collected between 450 and 900 nm using a snapshot camera mounted to a microscope with a halogen light source. VIS-NIR spectra collected from different specimens, such as melanocytic cells and other tissues (eg, epithelium), produced distinct and diagnostic spectra that were used to identify these materials in several regions of interest, making it possible to distinguish between intraoral amalgam tattoos (intramucosal metallic foreign bodies) and melanocytic lesions of the intraoral mucosa and the lip (each with P < .01 using the independent t test). HSI is presented as a diagnostic tool for the rapidly growing field of digital pathology. In this preliminary study, amalgam tattoos were reliably differentiated from melanocytic lesions of the oral cavity and the lip.

Hyperspectral imaging and characterization of allergic contact dermatitis in the short-wave infrared

Short-wave infrared hyperspectral imaging is applied to diagnose and monitor a case of allergic contact dermatitis (ACD) due to poison ivy exposure in one subject. This approach directly demonstrates increased tissue fluid content in ACD lesional skin with a spectral signature that matches the spectral signature of intradermally injected normal saline. The best contrast between the affected and unaffected skin is achieved through a selection of specific wavelengths at 1070, 1340 and 1605 nm and combining them in a pseudo-red-green-blue color space. An image derived from these wavelengths normalized to unaffected skin defines a "tissue fluid index" that may aid in the quantitative diagnosis and monitoring of ACD. Further clinical testing of this promising approach towards disease detection and monitoring with tissue fluid content quantification is warranted.

Classification of Dermoscopy Skin Lesion Color-Images Using Fractal-Deep Learning Features

The detection of skin diseases is becoming one of the priority tasks worldwide due to the increasing amount of skin cancer. Computer-aided diagnosis is a helpful tool to help dermatologists in the detection of these kinds of illnesses. This work proposes a computer-aided diagnosis based on 1D fractal signatures of texture-based features combining with deep-learning features using transferred learning based in Densenet-201. This proposal works with three 1D fractal signatures built per color-image. The energy, variance, and entropy of the fractal signatures are used combined with 100 features extracted from Densenet-201 to construct the features vector. Because commonly, the classes in the dataset of skin lesion images are imbalanced, we use the technique of ensemble of classifiers: K-nearest neighbors and two types of support vector machines. The computer-aided diagnosis output was determined based on the linear plurality vote. In this work, we obtained an average accuracy of 97.35%, an average precision of 91.61%, an average sensitivity of 66.45%, and an average specificity of 97.85% in the eight classes’ classification in the International Skin Imaging Collaboration (ISIC) archive-2019.

Detection of Tumoral Epithelial Lesions Using Hyperspectral Imaging and Deep Learning

We propose a new method for the analysis and classification of HSI images. The method uses deep learning to interpret the molecular vibrational behaviour of healthy and tumoral human epithelial tissue, based on data gathered via SWIR (short-wave infrared) spectroscopy. We analyzed samples of Melanoma, Dysplastic Nevus and healthy skin. Preliminary results show that human epithelial tissue is sensitive to SWIR to the point of making possible the differentiation between healthy and tumor tissues. We conclude that HSI-SWIR can be used to build new methods for tumor classification.

Curvature and height corrections of hyperspectral images using built-in 3D laser profilometry

Optical imaging systems use a light source that illuminates a sample and a photodetector that detects light reflected from or transmitted through the sample. The sample surface curvature, surface-to-camera distance, and illumination-source-to-surface distance significantly affect the measured signal, resulting in image artifacts. To correct the images, a three-dimensional (3D) profilometry system was used to obtain 3D surface information. The 3D information enables image correction using Lambert cosine law and height correction. In this study, the feasibility of the correction method for push-broom hyperspectral imaging of three different objects is presented. Results show a significant reduction of image artifacts, making further image analysis more accurate and robust. The presented 3D profilometry method is applicable to all push-broom imaging systems and the described correction procedure can be applied to all spectral or monochromatic imaging systems.

Hyperspectral imaging of human skin aided by artificial neural networks

We developed a compact, hand-held hyperspectral imaging system for 2D neural network-based visualization of skin chromophores and blood oxygenation. State-of-the-art micro-optic multichannel matrix sensor combined with the tunable Fabry-Perot micro interferometer enables a portable diagnostic device sensitive to the changes of the oxygen saturation as well as the variations of blood volume fraction of human skin. Generalized object-oriented Monte Carlo model is used extensively for the training of an artificial neural network utilized for the hyperspectral image processing. In addition, the results are verified and validated via actual experiments with tissue phantoms and human skin in vivo. The proposed approach enables a tool combining both the speed of an artificial neural network processing and the accuracy and flexibility of advanced Monte Carlo modeling. Finally, the results of the feasibility studies and the experimental tests on biotissue phantoms and healthy volunteers are presented.

Photoacoustic/Ultrasound/Optical Coherence Tomography Evaluation of Melanoma Lesion and Healthy Skin in a Swine Model

The marked increase in the incidence of melanoma coupled with the rapid drop in the survival rate after metastasis has promoted the investigation into improved diagnostic methods for melanoma. High-frequency ultrasound (US), optical coherence tomography (OCT), and photoacoustic imaging (PAI) are three potential modalities that can assist a dermatologist by providing extra information beyond dermoscopic features. In this study, we imaged a swine model with spontaneous melanoma using these modalities and compared the images with images of nearby healthy skin. Histology images were used for validation.