A background correction method to compensate illumination variation in hyperspectral imaging

Deep learning-based hyperspectral image correction and unmixing for brain tumor surgery

Hyperspectral imaging for fluorescence-guided brain tumor resection improves visualization of tissue differences, which can ameliorate patient outcomes. However, current methods do not effectively correct for heterogeneous optical and geometric tissue properties, leading to less accurate results. We propose two deep learning models for correction and unmixing that can capture these effects. While one is trained with protoporphyrin IX (PpIX) concentration labels, the other is semi-supervised. The models were evaluated on phantom and pig brain data with known PpIX concentration; the supervised and semi-supervised models achieved Pearson correlation coefficients (phantom, pig brain) between known and computed PpIX concentrations of (0.997, 0.990) and (0.98, 0.91), respectively. The classical approach achieved (0.93, 0.82). The semi-supervised approach also generalizes better to human data, achieving a 36% lower false-positive rate for PpIX detection and giving qualitatively more realistic results than existing methods. These results show promise for using deep learning to improve hyperspectral fluorescence-guided neurosurgery.

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral trade-off. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from a red-green-blue (RGB) image without complete hyperspectral measurements. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral trade-off, offering simple hardware requirements and potential applications of various machine learning techniques.

Spectral Similarity Measures for In Vivo Human Tissue Discrimination Based on Hyperspectral Imaging

PROBLEM

Similarity measures are widely used as an approved method for spectral discrimination or identification with their applications in different areas of scientific research. Even though a range of works have been presented, only a few showed slightly promising results for human tissue, and these were mostly focused on pathological and non-pathological tissue classification.

METHODS

In this work, several spectral similarity measures on hyperspectral (HS) images of in vivo human tissue were evaluated for tissue discrimination purposes. Moreover, we introduced two new hybrid spectral measures, called SID-JM-TAN(SAM) and SID-JM-TAN(SCA). We analyzed spectral signatures obtained from 13 different human tissue types and two different materials (gauze, instruments), collected from HS images of 100 patients during surgeries.

RESULTS

The quantitative results showed the reliable performance of the different similarity measures and the proposed hybrid measures for tissue discrimination purposes. The latter produced higher discrimination values, up to 6.7 times more than the classical spectral similarity measures. Moreover, an application of the similarity measures was presented to support the annotations of the HS images. We showed that the automatic checking of tissue-annotated thyroid and colon tissues was successful in 73% and 60% of the total spectra, respectively. The hybrid measures showed the highest performance. Furthermore, the automatic labeling of wrongly annotated tissues was similar for all measures, with an accuracy of up to 90%.

CONCLUSION

In future work, the proposed spectral similarity measures will be integrated with tools to support physicians in annotations and tissue labeling of HS images.

Multispectral imaging of nailfold capillaries using light-emitting diode illumination

SIGNIFICANCE

The capillaries are the smallest blood vessels in the body, typically imaged using video capillaroscopy to aid diagnosis of connective tissue diseases, such as systemic sclerosis. Video capillaroscopy allows visualization of morphological changes in the nailfold capillaries but does not provide any physiological information about the blood contained within the capillary network. Extracting parameters such as hemoglobin oxygenation could increase sensitivity for diagnosis and measurement of microvascular disease progression.

AIM

To design, construct, and test a low-cost multispectral imaging (MSI) system using light-emitting diode (LED) illumination to assess relative hemoglobin oxygenation in the nailfold capillaries.

APPROACH

An LED ring light was first designed and modeled. The ring light was fabricated using four commercially available LED colors and a custom-designed printed circuit board. The experimental system was characterized and results compared with the illumination model. A blood phantom with variable oxygenation was used to determine the feasibility of using the illumination-based MSI system for oximetry. Nailfold capillaries were then imaged in a healthy subject.

RESULTS

The illumination modeling results were in close agreement with the constructed system. Imaging of the blood phantom demonstrated sensitivity to changing hemoglobin oxygenation, which was in line with the spectral modeling of reflection. The morphological properties of the volunteer capillaries were comparable to those measured in current gold standard systems.

CONCLUSIONS

LED-based illumination could be used as a low-cost approach to enable MSI of the nailfold capillaries to provide insight into the oxygenation of the blood contained within the capillary network.

Comparison of preprocessing techniques to reduce nontissue-related variations in hyperspectral reflectance imaging

SIGNIFICANCE

Hyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples.

AIM

To compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types.

APPROACH

We compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min-max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope-while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast.

CONCLUSIONS

Preprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min-max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable.

Assessment of angle-dependent spectral distortion to develop accurate hyperspectral endoscopy

Hyperspectral endoscopy has shown its potential to improve disease diagnosis in gastrointestinal tracts. Recent approaches in developing hyperspectral endoscopy are mainly focusing on enhancing image speed and quality of spectral information under a clinical environment, but there are many issues in obtaining consistent spectral information due to complicated imaging conditions, including imaging angle, non-uniform illumination, working distance, and low reflected signal. We quantitatively investigated the effect of imaging angle on the distortion of spectral information by exploiting a bifurcated fiber, spectrometer, and tissue-mimicking phantom. Spectral distortion becomes severe as increasing the angle of the imaging fiber or shortening camera exposure time for fast image acquisition. Moreover, spectral ranges from 450 to 550 nm are more susceptible to the angle-dependent spectral distortion than longer spectral ranges. Therefore, imaging angles close to normal and longer target spectral ranges with enough detector exposure time could minimize spectral distortion in hyperspectral endoscopy. These findings will help implement clinical HSI endoscopy for the robust and accurate measurement of spectral information from patients in vivo.