In-Vivo and Ex-Vivo Tissue Analysis through Hyperspectral Imaging Techniques: Revealing the Invisible Features of Cancer

Hyperspectral Imaging for Tissue Classification after Advanced Stage Ovarian Cancer Surgery-A Pilot Study

The most important prognostic factor for the survival of advanced-stage epithelial ovarian cancer (EOC) is the completeness of cytoreductive surgery (CRS). Therefore, an intraoperative technique to detect microscopic tumors would be of great value. The aim of this pilot study is to assess the feasibility of near-infrared hyperspectral imaging (HSI) for EOC detection in ex vivo tissue samples. Images were collected during CRS in 11 patients in the wavelength range of 665−975 nm, and processed by calibration, normalization, and noise filtering. A linear support vector machine (SVM) was employed to classify healthy and tumorous tissue (defined as >50% tumor cells). Classifier performance was evaluated using leave-one-out cross-validation. Images of 26 tissue samples from 10 patients were included, containing 26,446 data points that were matched to histopathology. Tumorous tissue could be classified with an area under the curve of 0.83, a sensitivity of 0.81, a specificity of 0.70, and Matthew’s correlation coefficient of 0.41. This study paves the way to in vivo and intraoperative use of HSI during CRS. Hyperspectral imaging can scan a whole tissue surface in a fast and non-contact way. Our pilot study demonstrates that HSI and SVM learning can be used to discriminate EOC from surrounding tissue.

In vivo detection of head and neck tumors by hyperspectral imaging combined with deep learning methods

Currently, there are no fast and accurate screening methods available for head and neck cancer, the eighth most common tumor entity. For this study, we used hyperspectral imaging, an imaging technique for quantitative and objective surface analysis, combined with deep learning methods for automated tissue classification. As part of a prospective clinical observational study, hyperspectral datasets of laryngeal, hypopharyngeal and oropharyngeal mucosa were recorded in 98 patients before surgery in vivo. We established an automated data interpretation pathway that can classify the tissue into healthy and tumorous using convolutional neural networks with 2D spatial or 3D spatio-spectral convolutions combined with a state-of-the-art Densenet architecture. Using 24 patients for testing, our 3D spatio-spectral Densenet classification method achieves an average accuracy of 81%, a sensitivity of 83% and a specificity of 79%.

New Intraoperative Imaging Tools and Image-Guided Surgery in Gastric Cancer Surgery

Innovations and new advancements in intraoperative real-time imaging have gained significant importance in the field of gastric cancer surgery in the recent past. Currently, the most promising procedures include indocyanine green fluorescence imaging (ICG-FI) and hyperspectral imaging or multispectral imaging (HSI, MSI). ICG-FI is utilized in a broad range of clinical applications, e.g., assessment of perfusion or lymphatic drainage, and additional implementations are currently investigated. HSI is still in the experimental phase and its value and clinical relevance require further evaluation, but initial studies have shown a successful application in perfusion assessment, and prospects concerning non-invasive tissue and tumor classification are promising. The application of machine learning and artificial intelligence technologies might enable an automatic evaluation of the acquired image data in the future. Both methods facilitate the accurate visualization of tissue characteristics that are initially indistinguishable for the human eye. By aiding surgeons in optimizing the surgical procedure, image-guided surgery can contribute to the oncologic safety and reduction of complications in gastric cancer surgery and recent advances hold promise for the application of HSI in intraoperative tissue diagnostics.

Advances in surface plasmon resonance-based biosensor technologies for cancer biomarker detection

Surface plasmon resonance approach is a highly useful option to offer optical and label-free detection of target bioanalytes with numerous advantages (e.g., low-cost fabrication, appreciable sensitivity, label-free detection, and outstanding accuracy). As such, it allows early diagnosis of cancer biomarkers to monitor tumor progression and to prevent the recurrence of oncogenic tumors. This work highlights the recent progress in SPR biosensing technology for the diagnosis of various cancer types (e.g., lung, breast, prostate, and ovarian). Further, the performance of various SPR biosensors is also evaluated in terms of the basic quality assurance criteria (e.g., limit of detection (LOD), selectivity, sensor response time, and reusability). Finally, the limitations and future challenges associated with SPR biosensors are also discussed with respect to cancer biomarker detection.

A novel proposal to investigate the interplay between the spatial and spectral domains in near-infrared spectral imaging data by means of Image Decomposition, Encoding and Localization (IDEL)

The emergence of new spectral imaging applications in many science fields and in industry has not come to be a surprise, considering the immense potential this technique has to map spectral information. In the case of near-infrared spectral imaging, a rapid evolution of the technology has made it more and more appealing in non-destructive analysis of food and materials as well as in process monitoring applications. However, despite its great diffusion, some challenges remain open from the data analysis point of view, with the aim to fully uncover patterns and unveil the interplay between both the spatial and spectral domains. Here we propose a new approach, called Image Decomposition, Encoding and Localization (IDEL), where a spatial perspective is taken for the analysis of spectral images, while maintaining the significant information within the spectral domain. The methodology benefits from wavelet transform to exploit spatial features, encoding the outcoming images into a set of descriptors and utilizing multivariate analysis to isolate and extract the significant spatial-spectral information. A forensic case study of near-infrared images of biological stains on cotton fabrics is used as a benchmark. The stain and fabric have hardly distinguishable spectral signatures due to strong scattering effects that originate from the rough surface of the fabric and the high spectral absorbance of cotton in the near-infrared range. There is no selective information that can isolate signals related to these two components in the spectral images under study, and the complex spatial structure is highly interconnected to the spectral signatures. IDEL was capable of isolating the stains, (spatial) scattering effects, and a possible drying effect from the stains. It was possible to recover, at the same time, specific spectral regions that mostly highlight these isolated spatial structures, which was previously unobtainable.

LED-based Hyperspectral Endoscopic Imaging

Hyperspectral endoscopy can offer multiple advantages as compared to conventional endoscopy. Our goal is to design and develop a real-time hyperspectral endoscopic imaging system for the diagnosis of gastrointestinal (GI) tract cancers using a micro-LED array as an in-situ illumination source. The wavelengths of the system range from ultraviolet to visible and near infrared. To evaluate the use of the LED array for hyperspectral imaging, we designed a prototype system and conducted ex vivo experiments using normal and cancerous tissues of mice, chicken, and sheep. We compared the results of our LED-based approach with our reference hyperspectral camera system. The results confirm the similarity between the LED-based hyperspectral imaging system and the reference HSI camera. Our LED-based hyperspectral imaging system can be used not only as an endoscope but also as a laparoscopic or handheld devices for cancer detection and surgery.

Hyperspectral terahertz imaging and optical clearance for cancer classification in breast tumor surgical specimen

Purpose: We investigate the enhancement in terahertz (THz) images of freshly excised breast tumors upon treatment with an optical clearance agent. The hyperspectral imaging and spectral classifications are used to quantitatively demonstrate the image enhancement. Glycerol solution with 60% concentration is applied to excised breast tumor specimens for various time durations to investigate the effectiveness on image enhancement. Approach: THz reflection spectroscopy is utilized to obtain the absorption coefficient and the index of refraction of untreated and glycerol-treated tissues at each frequency up to 3 THz. Two classifiers, spectral angular mapping (SAM) based on several kernels and Euclidean minimum distance (EMD) are implemented to evaluate the effectiveness of the treatment. The testing raw data is obtained from five breast cancer specimens: two untreated specimens and three specimens treated with glycerol solution for 20, 40, or 60 min. All tumors used in the testing data have healthy tissues adjacent to cancerous ones consistent with the challenge faced in lumpectomy surgeries. Results: The glycerol-treated tissues showed a decrease in the absorption coefficients compared with untreated tissues, especially as the period of treatment increased. Although the sensitivity metric of the classifier presented higher values in the untreated tissues compared with the treated ones, the specificity and accuracy metrics demonstrated higher values for the treated tissues compared with the untreated ones. Conclusions: The biocompatible glycerol solution is a potential optical clearance agent in THz imaging while keeping the histopathology imaging intact. The SAM technique provided a good classification of cancerous tissues despite the small amount of cancer in the training data (only 7%). The SAM exponential kernel and EMD presented classification accuracy of ∼ 80 % to 85% compared with linear and polynomial kernels that provided accuracy ranging from 70% to 80%. Overall, glycerol treatment provides a potential improvement in cancer classification in freshly excised breast tumors.

A Semi-Supervised Reduced-Space Method for Hyperspectral Imaging Segmentation

The development of the hyperspectral remote sensor technology allows the acquisition of images with a very detailed spectral information for each pixel. Because of this, hyperspectral images (HSI) potentially possess larger capabilities in solving many scientific and practical problems in agriculture, biomedical, ecological, geological, hydrological studies. However, their analysis requires developing specialized and fast algorithms for data processing, due the high dimensionality of the data. In this work, we propose a new semi-supervised method for multilabel segmentation of HSI that combines a suitable linear discriminant analysis, a similarity index to compare different spectra, and a random walk based model with a direct label assignment. The user-marked regions are used for the projection of the original high-dimensional feature space to a lower dimensional space, such that the class separation is maximized. This allows to retain in an automatic way the most informative features, lightening the successive computational burden. The part of the random walk is related to a combinatorial Dirichlet problem involving a weighted graph, where the nodes are the projected pixel of the original HSI, and the positive weights depend on the distances between these nodes. We then assign to each pixel of the original image a probability quantifying the likelihood that the pixel (node) belongs to some subregion. The computation of the spectral distance involves both the coordinates in a features space of a pixel and of its neighbors. The final segmentation process is therefore reduced to a suitable optimization problem coupling the probabilities from the random walker computation, and the similarity with respect the initially labeled pixels. We discuss the properties of the new method with experimental results carried on benchmark images.

Comparison of Whiskbroom and Pushbroom darkfield elastic light scattering spectroscopic imaging for head and neck cancer identification in a mouse model

The early detection of head and neck cancer is a prolonged challenging task. It requires a precise and accurate identification of tissue alterations as well as a distinct discrimination of cancerous from healthy tissue areas. A novel approach for this purpose uses microspectroscopic techniques with special focus on hyperspectral imaging (HSI) methods. Our proof-of-principle study presents the implementation and application of darkfield elastic light scattering spectroscopy (DF ELSS) as a non-destructive, high-resolution, and fast imaging modality to distinguish lingual healthy from altered tissue regions in a mouse model. The main aspect of our study deals with the comparison of two varying HSI detection principles, which are a point-by-point and line scanning imaging, and whether one might be more appropriate in differentiating several tissue types. Statistical models are formed by deploying a principal component analysis (PCA) with the Bayesian discriminant analysis (DA) on the elastic light scattering (ELS) spectra. Overall accuracy, sensitivity, and precision values of 98% are achieved for both models whereas the overall specificity results in 99%. An additional classification of model-unknown ELS spectra is performed. The predictions are verified with histopathological evaluations of identical HE-stained tissue areas to prove the model's capability of tissue distinction. In the context of our proof-of-principle study, we assess the Pushbroom PCA-DA model to be more suitable for tissue type differentiations and thus tissue classification. In addition to the HE-examination in head and neck cancer diagnosis, the usage of HSI-based statistical models might be conceivable in a daily clinical routine.

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons

Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.

New Approach to the Old Challenge of Free Flap Monitoring-Hyperspectral Imaging Outperforms Clinical Assessment by Earlier Detection of Perfusion Failure

In reconstructive surgery, free flap failure, especially in complex osteocutaneous reconstructions, represents a significant clinical burden. Therefore, the aim of the presented study was to assess hyperspectral imaging (HSI) for monitoring of free flaps compared to clinical monitoring. In a prospective, non-randomized clinical study, patients with free flap reconstruction of the oro-maxillofacial-complex were included. Monitoring was assessed clinically and by using hyperspectral imaging (TIVITA™ Tissue-System, DiaspectiveVision GmbH, Pepelow, Germany) to determine tissue-oxygen-saturation [StO₂], near-infrared-perfusion-index [NPI], distribution of haemoglobin [THI] and water [TWI], and variance to an adjacent reference area (Δreference). A total of 54 primary and 11 secondary reconstructions were performed including fasciocutaneous and osteocutaneous flaps. Re-exploration was performed in 19 cases. A total of seven complete flap failures occurred, resulting in a 63% salvage rate. Mean time from flap inset to decision making for re-exploration based on clinical assessment was 23.1 ± 21.9 vs. 18.2 ± 19.4 h by the appearance of hyperspectral criteria indicating impaired perfusion (StO₂ ≤ 32% OR StO₂Δreference > −38% OR NPI ≤ 32.9 OR NPIΔreference ≥ −13.4%) resulting in a difference of 4.8 ± 5 h (p < 0.001). HSI seems able to detect perfusion compromise significantly earlier than clinical monitoring. These findings provide an interpretation aid for clinicians to simplify postoperative flap monitoring.

VNIR-NIR hyperspectral imaging fusion targeting intraoperative brain cancer detection

Currently, intraoperative guidance tools used for brain tumor resection assistance during surgery have several limitations. Hyperspectral (HS) imaging is arising as a novel imaging technique that could offer new capabilities to delineate brain tumor tissue in surgical-time. However, the HS acquisition systems have some limitations regarding spatial and spectral resolution depending on the spectral range to be captured. Image fusion techniques combine information from different sensors to obtain an HS cube with improved spatial and spectral resolution. This paper describes the contributions to HS image fusion using two push-broom HS cameras, covering the visual and near-infrared (VNIR) [400–1000 nm] and near-infrared (NIR) [900–1700 nm] spectral ranges, which are integrated into an intraoperative HS acquisition system developed to delineate brain tumor tissue during neurosurgical procedures. Both HS images were registered using intensity-based and feature-based techniques with different geometric transformations to perform the HS image fusion, obtaining an HS cube with wide spectral range [435–1638 nm]. Four HS datasets were captured to verify the image registration and the fusion process. Moreover, segmentation and classification methods were evaluated to compare the performance results between the use of the VNIR and NIR data, independently, with respect to the fused data. The results reveal that the proposed methodology for fusing VNIR–NIR data improves the classification results up to 21% of accuracy with respect to the use of each data modality independently, depending on the targeted classification problem.

Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.

Hyperspectral imaging and artificial intelligence to detect oral malignancy - part 1 - automated tissue classification of oral muscle, fat and mucosa using a light-weight 6-layer deep neural network

Background

Hyperspectral imaging (HSI) is a promising non-contact approach to tissue diagnostics, generating large amounts of raw data for whose processing computer vision (i.e. deep learning) is particularly suitable. Aim of this proof of principle study was the classification of hyperspectral (HS)-reflectance values into the human-oral tissue types fat, muscle and mucosa using deep learning methods. Furthermore, the tissue-specific hyperspectral signatures collected will serve as a representative reference for the future assessment of oral pathological changes in the sense of a HS-library.

Methods

A total of about 316 samples of healthy human-oral fat, muscle and oral mucosa was collected from 174 different patients and imaged using a HS-camera, covering the wavelength range from 500 nm to 1000 nm. HS-raw data were further labelled and processed for tissue classification using a light-weight 6-layer deep neural network (DNN).

Results

The reflectance values differed significantly (p < .001) for fat, muscle and oral mucosa at almost all wavelengths, with the signature of muscle differing the most. The deep neural network distinguished tissue types with an accuracy of > 80% each.

Conclusion

Oral fat, muscle and mucosa can be classified sufficiently and automatically by their specific HS-signature using a deep learning approach. Early detection of premalignant-mucosal-lesions using hyperspectral imaging and deep learning is so far represented rarely in in medical and computer vision research domain but has a high potential and is part of subsequent studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13005-021-00292-0.

Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection

Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.

Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation

Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.

Deep learning-based framework for the distinction of membranous nephropathy: a new approach through hyperspectral imagery

Background

Common subtypes seen in Chinese patients with membranous nephropathy (MN) include idiopathic membranous nephropathy (IMN) and hepatitis B virus-related membranous nephropathy (HBV-MN). However, the morphologic differences are not visible under the light microscope in certain renal biopsy tissues.

Methods

We propose here a deep learning-based framework for processing hyperspectral images of renal biopsy tissue to define the difference between IMN and HBV-MN based on the component of their immune complex deposition.

Results

The proposed framework can achieve an overall accuracy of 95.04% in classification, which also leads to better performance than support vector machine (SVM)-based algorithms.

Conclusion

IMN and HBV-MN can be correctly separated via the deep learning framework using hyperspectral imagery. Our results suggest the potential of the deep learning algorithm as a new method to aid in the diagnosis of MN.

Optoacoustic imaging in endocrinology and metabolism

Imaging is an essential tool in research, diagnostics and the management of endocrine disorders. Ultrasonography, nuclear medicine techniques, MRI, CT and optical methods are already used for applications in endocrinology. Optoacoustic imaging, also termed photoacoustic imaging, is emerging as a method for visualizing endocrine physiology and disease at different scales of detail: microscopic, mesoscopic and macroscopic. Optoacoustic contrast arises from endogenous light absorbers, such as oxygenated and deoxygenated haemoglobin, lipids and water, or exogenous contrast agents, and reveals tissue vasculature, perfusion, oxygenation, metabolic activity and inflammation. The development of high-performance optoacoustic scanners for use in humans has given rise to a variety of clinical investigations, which complement the use of the technology in preclinical research. Here, we review key progress with optoacoustic imaging technology as it relates to applications in endocrinology; for example, to visualize thyroid morphology and function, and the microvasculature in diabetes mellitus or adipose tissue metabolism, with particular focus on multispectral optoacoustic tomography and raster-scan optoacoustic mesoscopy. We explain the merits of optoacoustic microscopy and focus on mid-infrared optoacoustic microscopy, which enables label-free imaging of metabolites in cells and tissues. We showcase current optoacoustic applications within endocrinology and discuss the potential of these technologies to advance research and clinical practice.

Computed tomography imaging spectrometry based on superiorization and guided image filtering

Computed tomography imaging spectrometry (CTIS) is a snapshot hyperspectral imaging technique that can obtain a three-dimensional (${2D +}\lambda$) data cube of the target scene within a single exposure. Previous studies of CTIS suggest that reconstructions usually suffer from severe artifacts due to the limited number of projections available. To overcome this limitation, an iterative algorithm combining superiorization and guided image filtering is proposed to explore the intrinsic properties of the hyperspectral data cube as well as the characteristics of zero-order diffraction for the first time, to the best of our knowledge. Results from both simulative studies and proof-of-concept experiments demonstrate its superiority in suppressing artifacts and improving precision over the frequently used expectation maximization algorithm.

Diagnosis of cholangiocarcinoma from microscopic hyperspectral pathological dataset by deep convolution neural networks

This paper focuses on automatic Cholangiocarcinoma (CC) diagnosis from microscopic hyperspectral (HSI) pathological dataset with deep learning method. The first benchmark based on the microscopic hyperspectral pathological images is set up. Particularly, 880 scenes of multidimensional hyperspectral Cholangiocarcinoma images are collected and manually labeled each pixel as either tumor or non-tumor for supervised learning. Moreover, each scene from the slide is given a binary label indicating whether it is from a patient or a normal person. Different from traditional RGB images, the HSI acquires pixels in multiple spectral intervals, which is added as an extension on the channel dimension of 3-channel RGB image. This work aims at fully exploiting the spatial-spectral HSI data through a deep Convolution Neural Network (CNN). The whole scene is first divided into several patches. Then they are fed into CNN for the tumor/non-tumor binary prediction and the tumor area regression. The further diagnosis on the scene is made by random forest based on the features from patch prediction. Experiments show that HSI provides a more accurate result than RGB image. Moreover, a spectral interval convolution and normalization scheme are proposed for further mining the spectral information in HSI, which demonstrates the effectiveness of the spatial-spectral data for CC diagnosis.

Modulation by Ozone Therapy of Oxidative Stress in Chemotherapy-Induced Peripheral Neuropathy: The Background for a Randomized Clinical Trial

(1) Background: Chemotherapy-induced peripheral neuropathy (CIPN) decreases the quality of life of patients and can lead to a dose reduction and/or the interruption of chemotherapy treatment, limiting its effectiveness. Potential pathophysiological mechanisms involved in the pathogenesis of CIPN include chronic oxidative stress and subsequent increase in free radicals and proinflammatory cytokines. Approaches for the treatment of CIPN are highly limited in their number and efficacy, although several antioxidant-based therapies have been tried. On the other hand, ozone therapy can induce an adaptive antioxidant and anti-inflammatory response, which could be potentially useful in the management of CIPN. (2) Methods: The aims of this works are: (a) to summarize the potential mechanisms that could induce CIPN by the most relevant drugs (platinum, taxanes, vinca alkaloids, and bortezomib), with particular focus on the role of oxidative stress; (b) to summarize the current situation of prophylactic and treatment approaches; (c) to describe the action mechanisms of ozone therapy to modify oxidative stress and inflammation with its potential repercussions for CIPN; (d) to describe related experimental and clinical reports with ozone therapy in chemo-induced neurologic symptoms and CIPN; and (e) to show the main details about an ongoing focused clinical trial. (3) Results: A wide background relating to the mechanisms of action and a small number of experimental and clinical reports suggest that ozone therapy could be useful to prevent or improve CIPN. (4) Conclusions: Currently, there are no clinically relevant approaches for the prevention and treatment of stablished CIPN. The potential role of ozone therapy in this syndrome merits further research. Randomized controlled trials are ongoing.

Visualization of quantitative lipid distribution in mouse liver through near-infrared hyperspectral imaging

Lipid distribution in the liver provides crucial information for diagnosing the severity of fatty liver and fatty liver-associated liver cancer. Therefore, a noninvasive, label-free, and quantitative modality is eagerly anticipated. We report near-infrared hyperspectral imaging for the quantitative visualization of lipid content in mouse liver based on partial least square regression (PLSR) and support vector regression (SVR). Analysis results indicate that SVR with standard normal variate pretreatment outperforms PLSR by achieving better root mean square error (15.3 mg/g) and higher determination coefficient (0.97). The quantitative mapping of lipid content in the mouse liver is realized using SVR.

Multiparametric Radiomics for Predicting the Aggressiveness of Papillary Thyroid Carcinoma Using Hyperspectral Images

Papillary thyroid carcinoma (PTC) is primarily treated by surgical resection. During surgery, surgeons often need intraoperative frozen analysis and pathologic consultation in order to detect PTC. In some cases pathologists cannot determine if the tumor is aggressive until the operation has been completed. In this work, we have taken tumor classification a step further by determining the tumor aggressiveness of fresh surgical specimens. We employed hyperspectral imaging (HSI) in combination with multiparametric radiomic features to complete this task. The study cohort includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. A total of 67 features were extracted from this data. Using machine learning classification methods, we were able to achieve an AUC of 0.85. Our study shows that hyperspectral imaging and multiparametric radiomic features could aid in the pathological detection of tumor aggressiveness using fresh surgical spemens obtained during surgery.

[Integrity of swallowing apparatus-past, present, and future]

BACKGROUND

Swallowing is one of the most complex movement patterns. The integrity of the epithelial lining is essential.

OBJECTIVE

Which surgical techniques were developed at the beginnings of modern surgery and what methods are now available to maintain/reconstitute the integrity of the swallowing apparatus?

MATERIALS AND METHODS

This study comprises a selective literature search in early operation manuals and online archives, with incorporation of the authors' own experience.

RESULTS

Up until the 1950s, only very limited techniques were available to reconstruct the epithelial lining. Microvascular reanastomosed grafts were the game changer for reconstructive surgery, enabling reconstitution of the swallowing apparatus in primary surgery but also in challenging secondary interventions after insufficient or complicated primary therapy.

CONCLUSION

The need for anatomical and functional rehabilitation by reconstructive surgery is as pertinent as ever. Particularly in the oncological context, improved early detection and novel local treatment modalities could minimize treatment-associated damage to swallowing.

Comprehensive review of surgical microscopes: technology development and medical applications

SIGNIFICANCE

Surgical microscopes provide adjustable magnification, bright illumination, and clear visualization of the surgical field and have been increasingly used in operating rooms. State-of-the-art surgical microscopes are integrated with various imaging modalities, such as optical coherence tomography (OCT), fluorescence imaging, and augmented reality (AR) for image-guided surgery.

AIM

This comprehensive review is based on the literature of over 500 papers that cover the technology development and applications of surgical microscopy over the past century. The aim of this review is threefold: (i) providing a comprehensive technical overview of surgical microscopes, (ii) providing critical references for microscope selection and system development, and (iii) providing an overview of various medical applications.

APPROACH

More than 500 references were collected and reviewed. A timeline of important milestones during the evolution of surgical microscope is provided in this study. An in-depth technical overview of the optical system, mechanical system, illumination, visualization, and integration with advanced imaging modalities is provided. Various medical applications of surgical microscopes in neurosurgery and spine surgery, ophthalmic surgery, ear-nose-throat (ENT) surgery, endodontics, and plastic and reconstructive surgery are described.

RESULTS

Surgical microscopy has been significantly advanced in the technical aspects of high-end optics, bright and shadow-free illumination, stable and flexible mechanical design, and versatile visualization. New imaging modalities, such as hyperspectral imaging, OCT, fluorescence imaging, photoacoustic microscopy, and laser speckle contrast imaging, are being integrated with surgical microscopes. Advanced visualization and AR are being added to surgical microscopes as new features that are changing clinical practices in the operating room.

CONCLUSIONS

The combination of new imaging technologies and surgical microscopy will enable surgeons to perform challenging procedures and improve surgical outcomes. With advanced visualization and improved ergonomics, the surgical microscope has become a powerful tool in neurosurgery, spinal, ENT, ophthalmic, plastic and reconstructive surgeries.

Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach

The primary treatment for malignant brain tumors is surgical resection. While gross total resection improves the prognosis, a supratotal resection may result in neurological deficits. On the other hand, accurate intraoperative identification of the tumor boundaries may be very difficult, resulting in subtotal resections. Histological examination of biopsies can be used repeatedly to help achieve gross total resection but this is not practically feasible due to the turn-around time of the tissue analysis. Therefore, intraoperative techniques to recognize tissue types are investigated to expedite the clinical workflow for tumor resection and improve outcome by aiding in the identification and removal of the malignant lesion. Hyperspectral imaging (HSI) is an optical imaging technique with the power of extracting additional information from the imaged tissue. Because HSI images cannot be visually assessed by human observers, we instead exploit artificial intelligence techniques and leverage a Convolutional Neural Network (CNN) to investigate the potential of HSI in twelve in vivo specimens. The proposed framework consists of a 3D-2D hybrid CNN-based approach to create a joint extraction of spectral and spatial information from hyperspectral images. A comparison study was conducted exploiting a 2D CNN, a 1D DNN and two conventional classification methods (SVM, and the SVM classifier combined with the 3D-2D hybrid CNN) to validate the proposed network. An overall accuracy of 80% was found when tumor, healthy tissue and blood vessels were classified, clearly outperforming the state-of-the-art approaches. These results can serve as a basis for brain tumor classification using HSI, and may open future avenues for image-guided neurosurgical applications.

Custom optical imaging system for ex-vivo breast cancer detection based on spectral signature

BACKGROUND AND PURPOSE

Breast cancer is a popular well-known tumor in women globally and the subsequent driving reason for malignancy death. The purpose of the present study is to develop Low cost, commercial, and affordable system that discriminates malignant from normal breast tissues by exploiting the unique properties of Hyperspectral (HS) Imaging.

MATERIALS AND METHODS

The difference in the optical properties of the investigated breast tissues gives various reactions to light transmission, absorption, and especially the reflection over the spectral range. A custom optical imaging system (COIS) was designed to assess variable responses to monochromatic LEDs (415, 565, 660 nm) to highlight the differences in the reflectance properties of malignant/normal tissue. Statistical analysis was computed for determining the ideal wavelength to differentiate between normal and malignant regions. The experiment was repeated using the same LEDs, and low-cost CCD camera to examine the capability of such a system to discriminate between normal and malignant tissue.

RESULTS

Spectral images obtained by Hyperspectral camera, have been analyzed to reveal the difference of reflectance malignant and normal breast tissue. Superficial spectral reflection image with blue LED (415 nm) showed high variance (10.11). However, a more-depth reflection image with red LED (660 nm) showed low variance (4.44). So the optimum contrast image was produced by combining the three spectral information images from blue, green, and red LED. The COIS using a commercial CCD camera was in agreement with the HS camera.

CONCLUSIONS

The novel COIS of the commercial Low-cost CCD Camera is reliable and can be used with endoscopy technique as an assistant tool for surgical doctor to make decision and assess the resection edges in real time during surgery.

Hyperspectral evaluation of hepatic oxygenation in a model of total vs. arterial liver ischaemia

Liver ischaemia reperfusion injury (IRI) is a dreaded pathophysiological complication which may lead to an impaired liver function. The level of oxygen hypoperfusion affects the level of cellular damage during the reperfusion phase. Consequently, intraoperative localisation and quantification of oxygen impairment would help in the early detection of liver ischaemia. To date, there is no real-time, non-invasive, and intraoperative tool which can compute an organ oxygenation map, quantify and discriminate different types of vascular occlusions intraoperatively. Hyperspectral imaging (HSI) is a non-invasive optical methodology which can quantify tissue oxygenation and which has recently been applied to the medical field. A hyperspectral camera detects the relative reflectance of a tissue in the range of 500 to 1000 nm, allowing the quantification of organic compounds such as oxygenated and deoxygenated haemoglobin at different depths. Here, we show the first comparative study of liver oxygenation by means of HSI quantification in a model of total vascular inflow occlusion (VIO) vs. hepatic artery occlusion (HAO), correlating optical properties with capillary lactate and histopathological evaluation. We found that liver HSI could discriminate between VIO and HAO. These results were confirmed via cross-validation of HSI which detected and quantified intestinal congestion in VIO. A significant correlation between the near-infrared spectra and capillary lactate was found (r = − 0.8645, p = 0.0003 VIO, r = − 0.7113, p = 0.0120 HAO). Finally, a statistically significant negative correlation was found between the histology score and the near-infrared parameter index (NIR) (r = − 0.88, p = 0.004). We infer that HSI, by predicting capillary lactates and the histopathological score, would be a suitable non-invasive tool for intraoperative liver perfusion assessment.

Parallel Classification Pipelines for Skin Cancer Detection Exploiting Hyperspectral Imaging on Hybrid Systems

The early detection of skin cancer is of crucial importance to plan an effective therapy to treat the lesion. In routine medical practice, the diagnosis is based on the visual inspection of the lesion and it relies on the dermatologists’ expertise. After a first examination, the dermatologist may require a biopsy to confirm if the lesion is malignant or not. This methodology suffers from false positives and negatives issues, leading to unnecessary surgical procedures. Hyperspectral imaging is gaining relevance in this medical field since it is a non-invasive and non-ionizing technique, capable of providing higher accuracy than traditional imaging methods. Therefore, the development of an automatic classification system based on hyperspectral images could improve the medical practice to distinguish pigmented skin lesions from malignant, benign, and atypical lesions. Additionally, the system can assist general practitioners in first aid care to prevent noncritical lesions from reaching dermatologists, thereby alleviating the workload of medical specialists. In this paper is presented a parallel pipeline for skin cancer detection that exploits hyperspectral imaging. The computational times of the serial processing have been reduced by adopting multicore and many-core technologies, such as OpenMP and CUDA paradigms. Different parallel approaches have been combined, leading to the development of fifteen classification pipeline versions. Experimental results using in-vivo hyperspectral images show that a hybrid parallel approach is capable of classifying an image of 50 × 50 pixels with 125 bands in less than 1 s.

Assessment of hyperspectral indicators related to the content of phenolic compounds and multispectral fluorescence records in chicory leaves exposed to various light environments

Hyperspectral analysis represents a powerful technique for diagnostics of morphological and chemical information from aboveground parts of the plants, but the real potential of the method in pre-screening of phenolics in leaves is still insufficiently explored. In this study, assessment of the sensitivity and reliability of non-invasive methods of various phenolic compounds, also analyzed by HPLC in chicory plants (Cichorium intybus L.) exposed to various color light pretreatments was done. The hyperspectral records in visible and near infrared (VNIR) spectra were recorded using a handheld spectrometer and relationships between the specific hyperspectral parameters and the contents of tested phenolic compounds in chicory leaves were analyzed. Moreover, the correlations between the hyperspectral parameters and related parameters derived from the multispectral fluorescence records were assessed to compare the sensitivity of both techniques. The results indicated a relatively high correlation of anthocyanin-related parameters (ARI, mARI, mACI indices) with the content of some of tested phenolic compounds (quercetin-3-gluconuride, isorhamnetine-3-gluconuride, etc.), as well as with fluorescence ANTH index. Similar trends were observed in flavonoid parameter based on the near infra-red spectral bands (700, 760 nm), which expressed a high correlation with chlorogenic acid. On the other hand, the most frequently used flavonoid (FLAVI) indices based on UV-to-blue band reflectance showed very weak correlations with phenolic compounds, as well as with fluorescence FLAV index. The detailed analysis of the correlation between reflectance and fluorescence flavonoid parameters has shown that the parameters based on spectral reflectance are sensitive to increase of UV-absorbing compounds from low to moderate values, but, unlike the fluorescence parameter, they are not useful to recognize a further increase from middle to high or very high contents. Thus, our results outlined the possibilities, but also the limits of the use of hyperspectral analysis for rapid screening phenolic content, providing a practical evidence towards more efficient production of bioactive compounds for pharmaceutical or nutraceutical use.

Polarizer-Free AOTF-Based SWIR Hyperspectral Imaging for Biomedical Applications

Optical biomedical imaging in short wave infrared (SWIR) range within 0.9–1.7 μm is a rapidly developing technique. For this reason, there is an increasing interest in cost-effective and robust hardware for hyperspectral imaging data acquisition in this range. Tunable-filter-based solutions are of particular interest as they provide image processing flexibility and effectiveness in terms of collected data volume. Acousto-optical tunable filters (AOTFs) provide a unique set of features necessary for high-quality SWIR hyperspectral imaging. In this paper, we discuss a polarizer-free configuration of an imaging AOTF that provides a compact and easy-to-integrate design of the whole imager. We have carried out image quality analysis of this system, assembled it and validated its efficiency through multiple experiments. The developed system can be helpful in many hyperspectral applications including biomedical analyses.

Dark-field hyperspectral imaging for label free detection of nano-bio-materials

Nanomaterials are playing an increasingly important role in cancer diagnosis and treatment. Nanoparticle (NP)-based technologies have been utilized for targeted drug delivery during chemotherapies, photodynamic therapy, and immunotherapy. Another active area of research is the toxicity studies of these nanomaterials to understand the cellular uptake and transport of these materials in cells, tissues, and environment. Traditional techniques such as transmission electron microscopy, and mass spectrometry to analyze NP-based cellular transport or toxicity effect are expensive, require extensive sample preparation, and are low-throughput. Dark-field hyperspectral imaging (DF-HSI), an integration of spectroscopy and microscopy/imaging, provides the ability to investigate cellular transport of these NPs and to quantify the distribution of them within bio-materials. DF-HSI also offers versatility in non-invasively monitoring microorganisms, single cell, and proteins. DF-HSI is a low-cost, label-free technique that is minimally invasive and is a viable choice for obtaining high-throughput quantitative molecular analyses. Multimodal imaging modalities such as Fourier transform infrared and Raman spectroscopy are also being integrated with HSI systems to enable chemical imaging of the samples. HSI technology is being applied in surgeries to obtain molecular information about the tissues in real-time. This article provides brief overview of fundamental principles of DF-HSI and its application for nanomaterials, protein-detection, single-cell analysis, microbiology, surgical procedures along with technical challenges and future integrative approach with other imaging and measurement modalities. This article is categorized under: Diagnostic Tools > in vitro Nanoparticle-Based Sensing Diagnostic Tools > in vivo Nanodiagnostics and Imaging Implantable Materials and Surgical Technologies > Nanoscale Tools and Techniques in Surgery.

Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range

SIGNIFICANCE

Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups.

AIM

An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging.

APPROACH

Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources.

RESULTS

The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm).

CONCLUSIONS

Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.

Hyperspectral imaging for diagnosis and detection of ex-vivo breast cancer

BACKGROUND AND PURPOSE

Breast cancer is one of the most widely recognized tumors. .Diagnosis made in the early stage of disease may imporve outcomes. The discovery of malignant growth utilizing noninvasive light intrusive methods in lieu of conventional excisional biopsy may assist in achieving this goal.

MATERIALS AND METHODS

The change of the optical properties of ex-vivo breast tissues provides different responses to light transmission, absorption, and particularly the reflection over the spectrum range. We offer the use of Hyperspectral imaging (HSI) with advanced image processing and pattern recognition in order to analyze HSI data for breast cancer detection. The spectral signatures were mined and evaluated in both malignant and normal tissue. K-mean clustering was designed for classifying hyperspectral data in order to evaluate and detection of cancer tissue. This method was used to detect ex-vivo breast cancer. Spatial spectral images were created to high spot the differences in the reflectance properties of malignant versus normal tissue.

RESULTS

Trials showed that the superficial spectral reflection images within 500 nm wavelength showed high variance (214.65) between cancerous and normal breast tissues. On the other hand, image within 620 nm wavelength showed low variance (0.0020).However, the superimposed of spectral region 420-620 nm was proposed as the optimum bandwidth. Finally, the proposed HS imaging system was capable to discriminate the tumor region from normal tissue of the ex-vivo breast sample with sensitivity and a specificity of 95 % and 96 %.

CONCLUSIONS

High sensitivity and specificity were achieved, which proposes potential for HSI as an edge evaluation method to enhance the surgical outcome compared to the presently available techniques in the clinics.

Hyperspectral Superpixel-Wise Glioblastoma Tumor Detection in Histological Samples

The combination of hyperspectral imaging (HSI) and digital pathology may yield more accurate diagnosis. In this work, we propose the use of superpixels in HS images for combining regions of pixels that can be classified according to their spectral information to classify glioblastoma (GB) brain tumors in histologic slides. The superpixels are generated by a modified simple linear iterative clustering (SLIC) method to accommodate HS images. This work employs a dataset of H&E (Hematoxylin and Eosin) stained histology slides from 13 patients with GB and over 426,000 superpixels. A linear support vector machine (SVM) classifier was performed on independent training, validation, and testing datasets. The results of this investigation show that the proposed method can detect GB brain tumors from non-tumor samples with average sensitivity and specificity of 87% and 81%, respectively. The overall accuracy of this method is 83%. The study demonstrates that hyperspectral digital pathology can be useful for detecting GB brain tumors by exploiting spectral information alone on a superpixel level.

Hyperspectral Microscopic Imaging for Automatic Detection of Head and Neck Squamous Cell Carcinoma Using Histologic Image and Machine Learning

The purpose of this study is to develop hyperspectral imaging (HSI) for automatic detection of head and neck cancer cells on histologic slides. A compact hyperspectral microscopic system is developed in this study. Histologic slides from 15 patients with squamous cell carcinoma (SCC) of the larynx and hypopharynx are imaged with the system. The proposed nuclei segmentation method based on principle component analysis (PCA) can extract most nuclei in the hyperspectral image without extracting other sub-cellular components. Both spectra-based support vector machine (SVM) and patch-based convolutional neural network (CNN) are used for nuclei classification. CNNs were trained with both hyperspectral images and pseudo RGB images of extracted nuclei, in order to evaluate the usefulness of extra information provided by hyperspectral imaging. The average accuracy of spectra-based SVM classification is 68%. The average AUC and average accuracy of the HSI patch-based CNN classification is 0.94 and 82.4%, respectively. The hyperspectral microscopic imaging and classification methods provide an automatic tool to aid pathologists in detecting SCC on histologic slides.

Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited]

Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.

Gastric cancer diagnosis using hyperspectral imaging with principal component analysis and spectral angle mapper

SIGNIFICANCE

Hyperspectral imaging (HSI) is an emerging optical technique that has a double function of spectroscopy and imaging.

AIM

Near-infrared hyperspectral imaging (NIR-HSI) (900 to 1700 nm) with the help of chemometrics was investigated for gastric cancer diagnosis.

APPROACH

Mean spectra and standard deviation of normal and cancerous pixels were extracted. Principal component analysis (PCA) was used to compress the dimension of hypercube data and select the optimal wavelengths. Moreover, spectral angle mapper (SAM) was utilized as chemometrics to discriminate gastric cancer from normal.

RESULTS

Major spectral difference of cancerous and normal gastric tissue was observed around 975, 1215, and 1450 nm by comparison. A total of six wavelengths (i.e., 975, 1075, 1215, 1275, 1390, and 1450 nm) were then selected as optimal wavelengths by PCA. The accuracy using SAM is up to 90% according to hematoxylin-eosin results.

CONCLUSIONS

These results suggest that NIR-HSI has the potential as a cutting-edge optical diagnostic technique for gastric cancer diagnosis with suitable chemometrics.

Callico G. Non-Invasive Skin Cancer Diagnosis Using Hyperspectral Imaging for In-Situ Clinical Support

Skin cancer is one of the most common forms of cancer worldwide and its early detection its key to achieve an effective treatment of the lesion. Commonly, skin cancer diagnosis is based on dermatologist expertise and pathological assessment of biopsies. Although there are diagnosis aid systems based on morphological processing algorithms using conventional imaging, currently, these systems have reached their limit and are not able to outperform dermatologists. In this sense, hyperspectral (HS) imaging (HSI) arises as a new non-invasive technology able to facilitate the detection and classification of pigmented skin lesions (PSLs), employing the spectral properties of the captured sample within and beyond the human eye capabilities. This paper presents a research carried out to develop a dermatological acquisition system based on HSI, employing 125 spectral bands captured between 450 and 950 nm. A database composed of 76 HS PSL images from 61 patients was obtained and labeled and classified into benign and malignant classes. A processing framework is proposed for the automatic identification and classification of the PSL based on a combination of unsupervised and supervised algorithms. Sensitivity and specificity results of 87.5% and 100%, respectively, were obtained in the discrimination of malignant and benign PSLs. This preliminary study demonstrates, as a proof-of-concept, the potential of HSI technology to assist dermatologists in the discrimination of benign and malignant PSLs during clinical routine practice using a real-time and non-invasive hand-held device.

Classification of hyperspectral endocrine tissue images using support vector machines

BACKGROUND

Thyroidectomy is one of the most commonly performed surgical procedures. The region of the neck has a very complex structural organization. It would be beneficial to introduce a tool that can assist the surgeon in tissue discrimination during the procedure. One such solution is the noninvasive and contactless technique, called hyperspectral imaging (HSI).

METHODS

To interpret the HSI data, we implemented a supervised classification method to automatically discriminate the parathyroid, the thyroid, and the recurrent laryngeal nerve from surrounding tissue(muscle, skin) and materials (instruments, gauze). A leave-one-patient-out cross-validation was performed.

RESULTS

The best performance was obtained using support vector machine (SVM) with a classification and visualization in less than 1.4 seconds. A mean patient accuracy of 68% ± 23% was obtained for all tissues and material types.

CONCLUSIONS

The proposed method showed promising results and have to be confirmed on a larger cohort of patient data.

Callicó G, Fei B. Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks

Hyperspectral imaging (HSI) technology has demonstrated potential to provide useful information about the chemical composition of tissue and its morphological features in a single image modality. Deep learning (DL) techniques have demonstrated the ability of automatic feature extraction from data for a successful classification. In this study, we exploit HSI and DL for the automatic differentiation of glioblastoma (GB) and non-tumor tissue on hematoxylin and eosin (H&E) stained histological slides of human brain tissue. GB detection is a challenging application, showing high heterogeneity in the cellular morphology across different patients. We employed an HSI microscope, with a spectral range from 400 to 1000 nm, to collect 517 HS cubes from 13 GB patients using 20× magnification. Using a convolutional neural network (CNN), we were able to automatically detect GB within the pathological slides, achieving average sensitivity and specificity values of 88% and 77%, respectively, representing an improvement of 7% and 8% respectively, as compared to the results obtained using RGB (red, green, and blue) images. This study demonstrates that the combination of hyperspectral microscopic imaging and deep learning is a promising tool for future computational pathologies.

Conditional generative adversarial network for synthesizing hyperspectral images of breast cancer cells from digitized histology

Hyperspectral imaging (HSI), which acquires up to hundreds of bands, has been proposed as a promising imaging modality for digitized histology beyond RGB imaging to provide more quantitative information to assist pathologists with disease detection in samples. While digitized RGB histology is quite standardized and easy to acquire, histological HSI often requires custom-made equipment and longer imaging times compared to RGB. In this work, we present a dataset of corresponding RGB digitized histology and histological HSI of breast cancer, and we develop a conditional generative adversarial network (GAN) to artificially synthesize HSI from standard RGB images of normal and cancer cells. The results of the GAN synthesized HSI are promising, showing structural similarity (SSIM) of approximately 80% and mean absolute error (MAE) of 6 to 11%. Further work is needed to establish the ability of generating HSI from RGB images on larger datasets.

Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning

The performance of hyperspectral imaging (HSI) for tumor detection is investigated in ex-vivo specimens from the thyroid (N = 200) and salivary glands (N = 16) from 82 patients. Tissues were imaged with HSI in broadband reflectance and autofluorescence modes. For comparison, the tissues were imaged with two fluorescent dyes. Additionally, HSI was used to synthesize three-band RGB multiplex images to represent the human-eye response and Gaussian RGBs, which are referred to as HSI-synthesized RGB images. Using histological ground truths, deep learning algorithms were developed for tumor detection. For the classification of thyroid tumors, HSI-synthesized RGB images achieved the best performance with an AUC score of 0.90. In salivary glands, HSI had the best performance with 0.92 AUC score. This study demonstrates that HSI could aid surgeons and pathologists in detecting tumors of the thyroid and salivary glands.

Augmented reality visualization of hyperspectral imaging classifications for image-guided brain tumor phantom resection

Wearable augmented reality (AR) is an emerging technology with enormous potential for use in the medical field, from training and procedure simulations to image-guided surgery. Medical AR seeks to enable surgeons to see tissue segmentations in real time. With the objective of achieving real-time guidance, the emphasis on speed produces the need for a fast method for imaging and classification. Hyperspectral imaging (HSI) is a non-contact, optical imaging modality that rapidly acquires hundreds of images of tissue at different wavelengths, which can be used to generate spectral data of the tissue. Combining HSI information and machine-learning algorithms allows for effective tissue classification. In this paper, we constructed a brain tissue phantom with porcine blood, yellow-dyed gelatin, and colorless gelatin to represent blood vessels, tumor, and normal brain tissue, respectively. Using a segmentation algorithm, hundreds of hyperspectral images were compiled to classify each of the pixels. Three segmentation labels were generated from the data, each with a different type of tissue. Our system virtually superimposes the HSI channels and segmentation labels of a brain tumor phantom onto the real scene using the HoloLens AR headset. The user can manipulate and interact with the segmentation labels and HSI channels by repositioning, rotating, changing visibility, and switching between them. All actions can be performed through either hand or voice controls. This creates a convenient and multifaceted visualization of brain tissue in real time with minimal user restrictions. We demonstrate the feasibility of a fast and practical HIS-AR technique for potential use of image-guided brain surgery.

Applications of hyperspectral imaging in the detection and diagnosis of solid tumors

Hyperspectral imaging (HSI) is an emerging new technology in solid tumor diagnosis and detection. It incorporates traditional imaging and spectroscopy together to obtain both spatial and spectral information from tissues simultaneously in a non-invasive manner. This imaging modality is based on the principle that different tissues inherit different spectral reflectance responses that present as unique spectral fingerprints. HSI captures those composition-specific fingerprints to identify cancerous and normal tissues. It becomes a promising tool for performing tumor diagnosis and detection from the label-free histopathological examination to real-time intraoperative assistance. This review introduces the basic principles of HSI and summarizes its methodology and recent advances in solid tumor detection. In particular, the advantages of HSI applied to solid tumors are highlighted to show its potential for clinical use.

O’Shanahan A, Carrera D, Bisshopp S, Sosa C, Marquez M, Camacho R, Plaza MDLL, Morera J, M. Callico G. Most Relevant Spectral Bands Identification for Brain Cancer Detection Using Hyperspectral Imaging

Hyperspectral imaging (HSI) is a non-ionizing and non-contact imaging technique capable of obtaining more information than conventional RGB (red green blue) imaging. In the medical field, HSI has commonly been investigated due to its great potential for diagnostic and surgical guidance purposes. However, the large amount of information provided by HSI normally contains redundant or non-relevant information, and it is extremely important to identify the most relevant wavelengths for a certain application in order to improve the accuracy of the predictions and reduce the execution time of the classification algorithm. Additionally, some wavelengths can contain noise and removing such bands can improve the classification stage. The work presented in this paper aims to identify such relevant spectral ranges in the visual-and-near-infrared (VNIR) region for an accurate detection of brain cancer using in vivo hyperspectral images. A methodology based on optimization algorithms has been proposed for this task, identifying the relevant wavelengths to achieve the best accuracy in the classification results obtained by a supervised classifier (support vector machines), and employing the lowest possible number of spectral bands. The results demonstrate that the proposed methodology based on the genetic algorithm optimization slightly improves the accuracy of the tumor identification in ~5%, using only 48 bands, with respect to the reference results obtained with 128 bands, offering the possibility of developing customized acquisition sensors that could provide real-time HS imaging. The most relevant spectral ranges found comprise between 440.5-465.96 nm, 498.71-509.62 nm, 556.91-575.1 nm, 593.29-615.12 nm, 636.94-666.05 nm, 698.79-731.53 nm and 884.32-902.51 nm.

High-Level Synthesis of Multiclass SVM Using Code Refactoring to Classify Brain Cancer from Hyperspectral Images

Currently, high-level synthesis (HLS) methods and tools are a highly relevant area in the strategy of several leading companies in the field of system-on-chips (SoCs) and field programmable gate arrays (FPGAs). HLS facilitates the work of system developers, who benefit from integrated and automated design workflows, considerably reducing the design time. Although many advances have been made in this research field, there are still some uncertainties about the quality and performance of the designs generated with the use of HLS methodologies. In this paper, we propose an optimization of the HLS methodology by code refactoring using Xilinx SDSoCTM (Software-Defined System-On-Chip). Several options were analyzed for each alternative through code refactoring of a multiclass support vector machine (SVM) classifier written in C, using two different Zynq®-7000 SoC devices from Xilinx, the ZC7020 (ZedBoard) and the ZC7045 (ZC706). The classifier was evaluated using a brain cancer database of hyperspectral images. The proposed methodology not only reduces the required resources using less than 20% of the FPGA, but also reduces the power consumption −23% compared to the full implementation. The speedup obtained of 2.86× (ZC7045) is the highest found in the literature for SVM hardware implementations.

Hyperspectral Imaging of Head and Neck Squamous Cell Carcinoma for Cancer Margin Detection in Surgical Specimens from 102 Patients Using Deep Learning

Surgical resection of head and neck (H and N) squamous cell carcinoma (SCC) may yield inadequate surgical cancer margins in 10 to 20% of cases. This study investigates the performance of label-free, reflectance-based hyperspectral imaging (HSI) and autofluorescence imaging for SCC detection at the cancer margin in excised tissue specimens from 102 patients and uses fluorescent dyes for comparison. Fresh surgical specimens (n = 293) were collected during H and N SCC resections (n = 102). The tissue specimens were imaged with reflectance-based HSI and autofluorescence imaging and afterwards with two fluorescent dyes for comparison. A histopathological ground truth was made. Deep learning tools were developed to detect SCC with new patient samples (inter-patient) and machine learning for intra-patient tissue samples. Area under the curve (AUC) of the receiver-operator characteristic was used as the main evaluation metric. Additionally, the performance was estimated in mm increments circumferentially from the tumor-normal margin. In intra-patient experiments, HSI classified conventional SCC with an AUC of 0.82 up to 3 mm from the cancer margin, which was more accurate than proflavin dye and autofluorescence (both p < 0.05). Intra-patient autofluorescence imaging detected human papilloma virus positive (HPV+) SCC with an AUC of 0.99 at 3 mm and greater accuracy than proflavin dye (p < 0.05). The inter-patient results showed that reflectance-based HSI and autofluorescence imaging outperformed proflavin dye and standard red, green, and blue (RGB) images (p < 0.05). In new patients, HSI detected conventional SCC in the larynx, oropharynx, and nasal cavity with 0.85-0.95 AUC score, and autofluorescence imaging detected HPV+ SCC in tonsillar tissue with 0.91 AUC score. This study demonstrates that label-free, reflectance-based HSI and autofluorescence imaging methods can accurately detect the cancer margin in ex-vivo specimens within minutes. This non-ionizing optical imaging modality could aid surgeons and reduce inadequate surgical margins during SCC resections.