Quantitative Wavelength Analysis and Image Classification for Intraoperative Cancer Diagnosis with Hyperspectral Imaging

A hyperspectral surgical microscope with super-resolution reconstruction for intraoperative image guidance

Hyperspectral imaging (HSI) is an emerging imaging modality in medical applications, especially for intraoperative image guidance. A surgical microscope improves surgeons' visualization with fine details during surgery. The combination of HSI and surgical microscope can provide a powerful tool for surgical guidance. However, to acquire high-resolution hyperspectral images, the long integration time and large image file size can be a burden for intraoperative applications. Super-resolution reconstruction allows acquisition of low-resolution hyperspectral images and generates high-resolution HSI. In this work, we developed a hyperspectral surgical microscope and employed our unsupervised super-resolution neural network, which generated high-resolution hyperspectral images with fine textures and spectral characteristics of tissues. The proposed method can reduce the acquisition time and save storage space taken up by hyperspectral images without compromising image quality, which will facilitate the adaptation of hyperspectral imaging technology in intraoperative image guidance.

Dual-camera laparoscopic imaging with super-resolution reconstruction for intraoperative hyperspectral image guidance

Laparoscopic and robotic surgery, as one type of minimally invasive surgery (MIS), has gained popularity due to the improved surgeon ergonomics, instrument precision, operative time, and postoperative recovery. Hyperspectral imaging (HSI) is an emerging medical imaging modality, which has proved useful for intraoperative image guidance. Snapshot hyperspectral cameras are ideal for intraoperative laparoscopic imaging because of their compact size and light weight, but low spatial resolution can be a limitation. In this work, we developed a dual-camera laparoscopic imaging system that consists of a high-resolution color camera and a snapshot hyperspectral camera, and we employed super-resolution reconstruction to fuse the images from both cameras to generate high-resolution hyperspectral images. The experimental results show that our method can significantly improve the resolution of hyperspectral images without compromising the image quality or spectral signatures. The proposed super-resolution reconstruction method is promising to promote the employment of high-speed hyperspectral imaging in laparoscopic surgery.

Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging

One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.

Comparison of preprocessing techniques to reduce nontissue-related variations in hyperspectral reflectance imaging

SIGNIFICANCE

Hyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples.

AIM

To compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types.

APPROACH

We compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min-max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope-while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast.

CONCLUSIONS

Preprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min-max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable.

Hyperspectral autofluorescence characterization of drusen and sub-RPE deposits in age-related macular degeneration

Background:

Soft drusen and basal linear deposit (BLinD) are two forms of the same extracellular lipid rich material that together make up an Oil Spill on Bruch’s membrane (BrM). Drusen are focal and can be recognized clinically. In contrast BLinD is thin and diffusely distributed, and invisible clinically, even on highest resolution OCT, but has been detected on en face hyperspectral autofluorescence (AF) imaging ex vivo. We sought to optimize histologic hyperspectral AF imaging and image analysis for recognition of drusen and sub-RPE deposits (including BLinD and basal laminar deposit), for potential clinical application.

Methods:

Twenty locations specifically with drusen and 12 additional locations specifically from fovea, perifovea and mid-periphery from RPE/BrM flatmounts from 4 AMD donors underwent hyperspectral AF imaging with 4 excitation wavelengths (λ_ex 436, 450, 480 and 505 nm), and the resulting image cubes were simultaneously decomposed with our published non-negative matrix factorization (NMF). Rank 4 recovery of 4 emission spectra was chosen for each excitation wavelength.

Results:

A composite emission spectrum, sensitive and specific for drusen and presumed sub-RPE deposits (the SDr spectrum) was recovered with peak at 510–520 nm in all tissues with drusen, with greatest amplitudes at excitations λ_ex 436, 450 and 480 nm. The RPE spectra of combined sources Lipofuscin (LF)/Melanolipofuscin (MLF) were of comparable amplitude and consistently recapitulated the spectra S1, S2 and S3 previously reported from all tissues: tissues with drusen, foveal and extra-foveal locations.

Conclusions:

A clinical hyperspectral AF camera, with properly chosen excitation wavelengths in the blue range and a hyperspectral AF detector, should be capable of detecting and quantifying drusen and sub-RPE deposits, the earliest known lesions of AMD, before any other currently available imaging modality.

Comprehensive review of surgical microscopes: technology development and medical applications

SIGNIFICANCE

Surgical microscopes provide adjustable magnification, bright illumination, and clear visualization of the surgical field and have been increasingly used in operating rooms. State-of-the-art surgical microscopes are integrated with various imaging modalities, such as optical coherence tomography (OCT), fluorescence imaging, and augmented reality (AR) for image-guided surgery.

AIM

This comprehensive review is based on the literature of over 500 papers that cover the technology development and applications of surgical microscopy over the past century. The aim of this review is threefold: (i) providing a comprehensive technical overview of surgical microscopes, (ii) providing critical references for microscope selection and system development, and (iii) providing an overview of various medical applications.

APPROACH

More than 500 references were collected and reviewed. A timeline of important milestones during the evolution of surgical microscope is provided in this study. An in-depth technical overview of the optical system, mechanical system, illumination, visualization, and integration with advanced imaging modalities is provided. Various medical applications of surgical microscopes in neurosurgery and spine surgery, ophthalmic surgery, ear-nose-throat (ENT) surgery, endodontics, and plastic and reconstructive surgery are described.

RESULTS

Surgical microscopy has been significantly advanced in the technical aspects of high-end optics, bright and shadow-free illumination, stable and flexible mechanical design, and versatile visualization. New imaging modalities, such as hyperspectral imaging, OCT, fluorescence imaging, photoacoustic microscopy, and laser speckle contrast imaging, are being integrated with surgical microscopes. Advanced visualization and AR are being added to surgical microscopes as new features that are changing clinical practices in the operating room.

CONCLUSIONS

The combination of new imaging technologies and surgical microscopy will enable surgeons to perform challenging procedures and improve surgical outcomes. With advanced visualization and improved ergonomics, the surgical microscope has become a powerful tool in neurosurgery, spinal, ENT, ophthalmic, plastic and reconstructive surgeries.

Hyperspectral imaging in wound care: A systematic review

Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.

Hyperspectral imaging in wound care: A systematic review

Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.

O’Shanahan A, Carrera D, Bisshopp S, Sosa C, Marquez M, Camacho R, Plaza MDLL, Morera J, M. Callico G. Most Relevant Spectral Bands Identification for Brain Cancer Detection Using Hyperspectral Imaging

Hyperspectral imaging (HSI) is a non-ionizing and non-contact imaging technique capable of obtaining more information than conventional RGB (red green blue) imaging. In the medical field, HSI has commonly been investigated due to its great potential for diagnostic and surgical guidance purposes. However, the large amount of information provided by HSI normally contains redundant or non-relevant information, and it is extremely important to identify the most relevant wavelengths for a certain application in order to improve the accuracy of the predictions and reduce the execution time of the classification algorithm. Additionally, some wavelengths can contain noise and removing such bands can improve the classification stage. The work presented in this paper aims to identify such relevant spectral ranges in the visual-and-near-infrared (VNIR) region for an accurate detection of brain cancer using in vivo hyperspectral images. A methodology based on optimization algorithms has been proposed for this task, identifying the relevant wavelengths to achieve the best accuracy in the classification results obtained by a supervised classifier (support vector machines), and employing the lowest possible number of spectral bands. The results demonstrate that the proposed methodology based on the genetic algorithm optimization slightly improves the accuracy of the tumor identification in ~5%, using only 48 bands, with respect to the reference results obtained with 128 bands, offering the possibility of developing customized acquisition sensors that could provide real-time HS imaging. The most relevant spectral ranges found comprise between 440.5-465.96 nm, 498.71-509.62 nm, 556.91-575.1 nm, 593.29-615.12 nm, 636.94-666.05 nm, 698.79-731.53 nm and 884.32-902.51 nm.

In-Vivo and Ex-Vivo Tissue Analysis through Hyperspectral Imaging Techniques: Revealing the Invisible Features of Cancer

In contrast to conventional optical imaging modalities, hyperspectral imaging (HSI) is able to capture much more information from a certain scene, both within and beyond the visual spectral range (from 400 to 700 nm). This imaging modality is based on the principle that each material provides different responses to light reflection, absorption, and scattering across the electromagnetic spectrum. Due to these properties, it is possible to differentiate and identify the different materials/substances presented in a certain scene by their spectral signature. Over the last two decades, HSI has demonstrated potential to become a powerful tool to study and identify several diseases in the medical field, being a non-contact, non-ionizing, and a label-free imaging modality. In this review, the use of HSI as an imaging tool for the analysis and detection of cancer is presented. The basic concepts related to this technology are detailed. The most relevant, state-of-the-art studies that can be found in the literature using HSI for cancer analysis are presented and summarized, both in-vivo and ex-vivo. Lastly, we discuss the current limitations of this technology in the field of cancer detection, together with some insights into possible future steps in the improvement of this technology.

Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some⁻Different⁻Light into the Dark

Hyperspectral/Multispectral imaging (HSI/MSI) technologies are able to sample from tens to hundreds of spectral channels within the electromagnetic spectrum, exceeding the capabilities of human vision. These spectral techniques are based on the principle that every material has a different response (reflection and absorption) to different wavelengths. Thereby, this technology facilitates the discrimination between different materials. HSI has demonstrated good discrimination capabilities for materials in fields, for instance, remote sensing, pollution monitoring, field surveillance, food quality, agriculture, astronomy, geological mapping, and currently, also in medicine. HSI technology allows tissue observation beyond the limitations of the human eye. Moreover, many researchers are using HSI as a new diagnosis tool to analyze optical properties of tissue. Recently, HSI has shown good performance in identifying human diseases in a non-invasive manner. In this paper, we show the potential use of these technologies in the medical domain, with emphasis in the current advances in gastroenterology. The main aim of this review is to provide an overview of contemporary concepts regarding HSI technology together with state-of-art systems and applications in gastroenterology. Finally, we discuss the current limitations and upcoming trends of HSI in gastroenterology.

Spatio-spectral classification of hyperspectral images for brain cancer detection during surgical operations

Surgery for brain cancer is a major problem in neurosurgery. The diffuse infiltration into the surrounding normal brain by these tumors makes their accurate identification by the naked eye difficult. Since surgery is the common treatment for brain cancer, an accurate radical resection of the tumor leads to improved survival rates for patients. However, the identification of the tumor boundaries during surgery is challenging. Hyperspectral imaging is a non-contact, non-ionizing and non-invasive technique suitable for medical diagnosis. This study presents the development of a novel classification method taking into account the spatial and spectral characteristics of the hyperspectral images to help neurosurgeons to accurately determine the tumor boundaries in surgical-time during the resection, avoiding excessive excision of normal tissue or unintentionally leaving residual tumor. The algorithm proposed in this study to approach an efficient solution consists of a hybrid framework that combines both supervised and unsupervised machine learning methods. Firstly, a supervised pixel-wise classification using a Support Vector Machine classifier is performed. The generated classification map is spatially homogenized using a one-band representation of the HS cube, employing the Fixed Reference t-Stochastic Neighbors Embedding dimensional reduction algorithm, and performing a K-Nearest Neighbors filtering. The information generated by the supervised stage is combined with a segmentation map obtained via unsupervised clustering employing a Hierarchical K-Means algorithm. The fusion is performed using a majority voting approach that associates each cluster with a certain class. To evaluate the proposed approach, five hyperspectral images of surface of the brain affected by glioblastoma tumor in vivo from five different patients have been used. The final classification maps obtained have been analyzed and validated by specialists. These preliminary results are promising, obtaining an accurate delineation of the tumor area.

Tumor Margin Classification of Head and Neck Cancer Using Hyperspectral Imaging and Convolutional Neural Networks

One of the largest factors affecting disease recurrence after surgical cancer resection is negative surgical margins. Hyperspectral imaging (HSI) is an optical imaging technique with potential to serve as a computer aided diagnostic tool for identifying cancer in gross ex-vivo specimens. We developed a tissue classifier using three distinct convolutional neural network (CNN) architectures on HSI data to investigate the ability to classify the cancer margins from ex-vivo human surgical specimens, collected from 20 patients undergoing surgical cancer resection as a preliminary validation group. A new approach for generating the HSI ground truth using a registered histological cancer margin is applied in order to create a validation dataset. The CNN-based method classifies the tumor-normal margin of squamous cell carcinoma (SCCa) versus normal oral tissue with an area under the curve (AUC) of 0.86 for inter-patient validation, performing with 81% accuracy, 84% sensitivity, and 77% specificity. Thyroid carcinoma cancer-normal margins are classified with an AUC of 0.94 for inter-patient validation, performing with 90% accuracy, 91% sensitivity, and 88% specificity. Our preliminary results on a limited patient dataset demonstrate the predictive ability of HSI-based cancer margin detection, which warrants further investigation with more patient data and additional processing techniques to optimize the proposed deep learning method.

Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging

Surgical cancer resection requires an accurate and timely diagnosis of the cancer margins in order to achieve successful patient remission. Hyperspectral imaging (HSI) has emerged as a useful, noncontact technique for acquiring spectral and optical properties of tissue. A convolutional neural network (CNN) classifier is developed to classify excised, squamous-cell carcinoma, thyroid cancer, and normal head and neck tissue samples using HSI. The CNN classification was validated by the manual annotation of a pathologist specialized in head and neck cancer. The preliminary results of 50 patients indicate the potential of HSI and deep learning for automatic tissue-labeling of surgical specimens of head and neck patients.

Hyperspectral Imaging of Neoplastic Progression in a Mouse Model of Oral Carcinogenesis

Hyperspectral imaging (HSI) is an emerging modality for medical applications and holds great potential for noninvasive early detection of cancer. It has been reported that early cancer detection can improve the survival and quality of life of head and neck cancer patients. In this paper, we explored the possibility of differentiating between premalignant lesions and healthy tongue tissue using hyperspectral imaging in a chemical induced oral cancer animal model. We proposed a novel classification algorithm for cancer detection using hyperspectral images. The method detected the dysplastic tissue with an average area under the curve (AUC) of 0.89. The hyperspectral imaging and classification technique may provide a new tool for oral cancer detection.

A Minimum Spanning Forest-Based Method for Noninvasive Cancer Detection With Hyperspectral Imaging

GOAL

The purpose of this paper is to develop a classification method that combines both spectral and spatial information for distinguishing cancer from healthy tissue on hyperspectral images in an animal model.

METHODS

An automated algorithm based on a minimum spanning forest (MSF) and optimal band selection has been proposed to classify healthy and cancerous tissue on hyperspectral images. A support vector machine classifier is trained to create a pixel-wise classification probability map of cancerous and healthy tissue. This map is then used to identify markers that are used to compute mutual information for a range of bands in the hyperspectral image and thus select the optimal bands. An MSF is finally grown to segment the image using spatial and spectral information.

CONCLUSION

The MSF based method with automatically selected bands proved to be accurate in determining the tumor boundary on hyperspectral images.

SIGNIFICANCE

Hyperspectral imaging combined with the proposed classification technique has the potential to provide a noninvasive tool for cancer detection.

Superpixel-based spectral classification for the detection of head and neck cancer with hyperspectral imaging

Hyperspectral imaging (HSI) is an emerging imaging modality for medical applications. HSI acquires two dimensional images at various wavelengths. The combination of both spectral and spatial information provides quantitative information for cancer detection and diagnosis. This paper proposes using superpixels, principal component analysis (PCA), and support vector machine (SVM) to distinguish regions of tumor from healthy tissue. The classification method uses 2 principal components decomposed from hyperspectral images and obtains an average sensitivity of 93% and an average specificity of 85% for 11 mice. The hyperspectral imaging technology and classification method can have various applications in cancer research and management.

Framework for hyperspectral image processing and quantification for cancer detection during animal tumor surgery

Hyperspectral imaging (HSI) is an imaging modality that holds strong potential for rapid cancer detection during image-guided surgery. But the data from HSI often needs to be processed appropriately in order to extract the maximum useful information that differentiates cancer from normal tissue. We proposed a framework for hyperspectral image processing and quantification, which includes a set of steps including image preprocessing, glare removal, feature extraction, and ultimately image classification. The framework has been tested on images from mice with head and neck cancer, using spectra from 450- to 900-nm wavelength. The image analysis computed Fourier coefficients, normalized reflectance, mean, and spectral derivatives for improved accuracy. The experimental results demonstrated the feasibility of the hyperspectral image processing and quantification framework for cancer detection during animal tumor surgery, in a challenging setting where sensitivity can be low due to a modest number of features present, but potential for fast image classification can be high. This HSI approach may have potential application in tumor margin assessment during image-guided surgery, where speed of assessment may be the dominant factor.