Medical hyperspectral imaging: a review

In-situ detection of microplastics in the aquatic environment: A systematic literature review

Microplastics are ubiquitous in the aquatic environment and have emerged as a significant environmental issue due to their potential impacts on human health and the ecosystem. Current laboratory-based microplastic detection methods suffer from various drawbacks, including a lack of standardisation, limited spatial and temporal coverage, high costs, and time-consuming procedures. Consequently, there is a need for the development of in-situ techniques to detect and monitor microplastics to effectively identify and understand their sources, pathways, and behaviours. Herein, we adopt a systematic literature review method to assess the development and application of experimental and field technologies designed for the in-situ detection and monitoring of aquatic microplastics, without the need for sample preparation. Four scientific databases were searched in March 2023, resulting in a review of 62 relevant studies. These studies were classified into seven sensor categories and their working principles were discussed. The sensor classes include optical devices, digital holography, Raman spectroscopy, other spectroscopy, hyperspectral imaging, remote sensing, and other methods. We also looked at how data from these technologies are integrated with machine learning models to develop classifiers capable of accurately characterising the physical and chemical properties of microplastics and discriminating them from other particles. This review concluded that in-situ detection of microplastics in aquatic environments is feasible and can be achieved with high accuracy, even though the methods are still in the early stages of development. Nonetheless, further research is still needed to enhance the in-situ detection of microplastics. This includes exploring the possibility of combining various detection methods and developing robust machine-learning classifiers. Additionally, there is a recommendation for in-situ implementation of the reviewed methods to assess their effectiveness in detecting microplastics and identify their limitations.

Retinal hyperspectral imaging in mouse models of Parkinson's disease and healthy aging

Retinal hyperspectral imaging (HSI) is a non-invasive in vivo approach that has shown promise in Alzheimer's disease. Parkinson's disease is another neurodegenerative disease where brain pathobiology such as alpha-synuclein and iron overaccumulation have been implicated in the retina. However, it remains unknown whether HSI is altered in in vivo models of Parkinson's disease, whether it differs from healthy aging, and the mechanisms which drive these changes. To address this, we conducted HSI in two mouse models of Parkinson's disease across different ages; an alpha-synuclein overaccumulation model (hA53T transgenic line M83, A53T) and an iron deposition model (Tau knock out, TauKO). In comparison to wild-type littermates the A53T and TauKO mice both demonstrated increased reflectivity at short wavelengths ~ 450 to 600 nm. In contrast, healthy aging in three background strains exhibited the opposite effect, a decreased reflectance in the short wavelength spectrum. We also demonstrate that the Parkinson's hyperspectral signature is similar to that from an Alzheimer's disease model, 5xFAD mice. Multivariate analyses of HSI were significant when plotted against age. Moreover, when alpha-synuclein, iron or retinal nerve fibre layer thickness were added as a cofactor this improved the R2 values of the correlations in certain groups. This study demonstrates an in vivo hyperspectral signature in Parkinson's disease that is consistent in two mouse models and is distinct from healthy aging. There is also a suggestion that factors including retinal deposition of alpha-synuclein and iron may play a role in driving the Parkinson's disease hyperspectral profile and retinal nerve fibre layer thickness in advanced aging. These findings suggest that HSI may be a promising translation tool in Parkinson's disease.

A proof of concept for microcirculation monitoring using machine learning based hyperspectral imaging in critically ill patients: a monocentric observational study

BACKGROUND

Impaired microcirculation is a cornerstone of sepsis development and leads to reduced tissue oxygenation, influenced by fluid and catecholamine administration during treatment. Hyperspectral imaging (HSI) is a non-invasive bedside technology for visualizing physicochemical tissue characteristics. Machine learning (ML) for skin HSI might offer an automated approach for bedside microcirculation assessment, providing an individualized tissue fingerprint of critically ill patients in intensive care. The study aimed to determine if machine learning could be utilized to automatically identify regions of interest (ROIs) in the hand, thereby distinguishing between healthy individuals and critically ill patients with sepsis using HSI.

METHODS

HSI raw data from 75 critically ill sepsis patients and from 30 healthy controls were recorded using TIVITA® Tissue System and analyzed using an automated ML approach. Additionally, patients were divided into two groups based on their SOFA scores for further subanalysis: less severely ill (SOFA ≤ 5) and severely ill (SOFA > 5). The analysis of the HSI raw data was fully-automated using MediaPipe for ROI detection (palm and fingertips) and feature extraction. HSI Features were statistically analyzed to highlight relevant wavelength combinations using Mann-Whitney-U test and Benjamini, Krieger, and Yekutieli (BKY) correction. In addition, Random Forest models were trained using bootstrapping, and feature importances were determined to gain insights regarding the wavelength importance for a model decision.

RESULTS

An automated pipeline for generating ROIs and HSI feature extraction was successfully established. HSI raw data analysis accurately distinguished healthy controls from sepsis patients. Wavelengths at the fingertips differed in the ranges of 575-695 nm and 840-1000 nm. For the palm, significant differences were observed in the range of 925-1000 nm. Feature importance plots indicated relevant information in the same wavelength ranges. Combining palm and fingertip analysis provided the highest reliability, with an AUC of 0.92 to distinguish between sepsis patients and healthy controls.

CONCLUSION

Based on this proof of concept, the integration of automated and standardized ROIs along with automated skin HSI analyzes, was able to differentiate between healthy individuals and patients with sepsis. This approach offers a reliable and objective assessment of skin microcirculation, facilitating the rapid identification of critically ill patients.

Fluorescence and tracers in surgery: the coming future

The revolution that we are seeing in the world of surgery will determine the way we understand surgical approaches in coming years. Since the implementation of minimally invasive surgery, innovations have constantly been developed to allow the laparoscopic approach to go further and be applied to more and more procedures. In recent years, we have been in the middle of another revolutionary era, with robotic surgery, the application of artificial intelligence and image-guided surgery. The latter includes 3D reconstructions for surgical planning, virtual reality, holograms or tracer-guided surgery, where ICG-guided fluorescence has provided a different perspective on surgery. ICG has been used to identify anatomical structures, assess tissue perfusion, and identify tumors or tumor lymphatic drainage. But the most important thing is that this technology has come hand in hand with the potential to develop other types of tracers that will facilitate the identification of tumor cells and ureters, as well as different light beams to identify anatomical structures. These will lead to other types of systems to assess tissue perfusion without the use of tracers, such as hyperspectral imaging. Combined with the upcoming introduction of ICG quantification, these developments represent a real revolution in the surgical world. With the imminent implementation of these technological advances, a review of their clinical application in general surgery is timely, and this review serves that aim.

Label-Free Optical Technologies to Enhance Noninvasive Endoscopic Imaging of Early-Stage Cancers

White light endoscopic imaging allows for the examination of internal human organs and is essential in the detection and treatment of early-stage cancers. To facilitate diagnosis of precancerous changes and early-stage cancers, label-free optical technologies that provide enhanced malignancy-specific contrast and depth information have been extensively researched. The rapid development of technology in the past two decades has enabled integration of these optical technologies into clinical endoscopy. In recent years, the significant advantages of using these adjunct optical devices have been shown, suggesting readiness for clinical translation. In this review, we provide an overview of the working principles and miniaturization considerations and summarize the clinical and preclinical demonstrations of several such techniques for early-stage cancer detection. We also offer an outlook for the integration of multiple technologies and the use of computer-aided diagnosis in clinical endoscopy.

Unique Hyperspectral Response Design Enabled by Periodic Surface Textures in Photodiodes

The applications of hyperspectral imaging across disciplines such as healthcare, automobiles, forensics, and astronomy are constrained by the requirement for intricate filters and dispersion lenses. By utilization of devices with engineered spectral responses and advanced signal processing techniques, the spectral imaging process can be made more approachable across various fields. We propose a spectral response design method employing photon-trapping surface textures (PTSTs), which eliminates the necessity for external diffraction optics and facilitates system miniaturization. We have developed an analytical model to calculate electromagnetic wave coupling using the effective refractive index of silicon in the presence of PTST. We have extensively validated the model against simulations and experimental data, ensuring the accuracy of our predictions. We observe a strong linear relationship between the peak coupling wavelength and the PTST period along with a moderate proportional relation to the PTST diameters. Additionally, we identify a significant correlation between inter-PTST spacing and wave propagation modes. The experimental validation of the model is conducted using PTST-equipped photodiodes fabricated through complementary metal-oxide-semiconductor-compatible processes. Further, we demonstrate the electrical and optical performance of these PTST-equipped photodiodes to show high speed (response time: 27 ps), high gain (multiplication gain, M: 90), and a low operating voltage (breakdown voltage: ∼ 8.0 V). Last, we utilize the distinctive response of the fabricated PTST-equipped photodiode to simulate hyperspectral imaging, providing a proof of principle. These findings are crucial for the progression of on-chip integration of high-performance spectrometers, guaranteeing real-time data manipulation, and cost-effective production of hyperspectral imaging systems.

Investigation on the influence of the skin tone on hyperspectral imaging for free flap surgery

Hyperspectral imaging (HSI) is a new emerging modality useful for the noncontact assessment of free flap perfusion. This measurement technique relies on the optical properties within the tissue. Since the optical properties of hemoglobin (Hb) and melanin overlap, the results of the perfusion assessment and other tissue-specific parameters are likely to be distorted by the melanin, especially at higher melanin concentrations. Many spectroscopic devices have been shown to struggle with a melanin related bias, which results in a clinical need to improve non-invasive perfusion assessment, especially for a more pigmented population. This study investigated the influence of skin tones on tissue indices measurements using HSI. In addition, other factors that might affect HSI, such as age, body mass index (BMI), sex or smoking habits, were also considered. Therefore, a prospective feasibility study was conducted, including 101 volunteers from whom tissue indices measurements were performed on 16 different body sites. Skin tone classification was performed using the Fitzpatrick skin type classification questionnaire, and the individual typology angle (ITA) acquired from the RGB images was calculated simultaneously with the measurements. Tissue indices provided by the used HSI-device were correlated to the possible influencing factors. The results show that a dark skin tone and, therefore, higher levels of pigmentation influence the HSI-derived tissue indices. In addition, possible physiological factors influencing the HSI-measurements were found. In conclusion, the HSI-based tissue indices can be used for perfusion assessment for people with lighter skin tone levels but show limitations in people with darker skin tones. Furthermore, it could be used for a more individual perfusion assessment if different physiological influencing factors are respected.

Design of a metasurface deflector for guided absorption enhancement in a Si PIN photodiode

We numerically demonstrated a surface-illuminated Si PIN photodiode (PD) structure with a metasurface composed of etched isosceles triangle pillars that can enhance sensitivity in the near-infrared wavelength range (NIR) by enabling directional scattering (DS) of photons. The metasurface is designed to act as a deflector to increase the absorption efficiency by extending the photon dwell time. This is particularly effective in thin intrinsic layers (i-layers) of silicon, surpassing the capabilities of conventional omnidirectional scattering gratings. Our results show a 3.5-fold increase in internal quantum efficiency over wavelengths above 0.9 µm compared to the structure without metasurface. The absorption enhancement brought about by directional scattering is not limited to thin i-layers; it can potentially improve a wide range of photodiode geometries and structures. Furthermore, the proposed structure, consisting of an all-Si layer and a simple geometric etching process, makes it compatible with foundry fabrication methods and opens up new possibilities for expanding applications of Si PDs.

SSTU: Swin-Spectral Transformer U-Net for hyperspectral whole slide image reconstruction

Whole Slide Imaging and Hyperspectral Microscopic Imaging provide great quality data with high spatial and spectral resolution for histopathology. Existing Hyperspectral Whole Slide Imaging systems combine the advantages of the techniques above, thus providing rich information for pathological diagnosis. However, it cannot avoid the problems of slow acquisition speed and mass data storage demand. Inspired by the spectral reconstruction task in computer vision and remote sensing, the Swin-Spectral Transformer U-Net (SSTU) has been developed to reconstruct Hyperspectral Whole Slide images (HWSis) from multiple Hyperspectral Microscopic images (HMis) of small Field of View and Whole Slide images (WSis). The Swin-Spectral Transformer (SST) module in SSTU takes full advantage of Transformer in extracting global attention. Firstly, Swin Transformer is exploited in space domain, which overcomes the high computation cost in Vision Transformer structures, while it maintains the spatial features extracted from WSis. Furthermore, Spectral Transformer is exploited to collect the long-range spectral features in HMis. Combined with the multi-scale encoder-bottleneck-decoder structure of U-Net, SSTU network is formed by sequential and symmetric residual connections of SSTs, which reconstructs a selected area of HWSi from coarse to fine. Qualitative and quantitative experiments prove the performance of SSTU in HWSi reconstruction task superior to other state-of-the-art spectral reconstruction methods.

Naevi Characterization Using Hyperspectral Imaging: A Pilot Study

PURPOSE

The prevalence of choroidal naevi is common and has been found to be up to 10%. Little is known regarding the optical properties of choroidal naevi. A novel hyperspectral eye fundus camera was used to investigate choroidal naevi's optical density spectra in the retina.

METHODS

In an ophthalmology clinic setting, patients with choroidal naevi were included in the study. Visual acuity and pressure were tested. Following mydriatics, optical coherence tomography and fundus photography were taken as a reference, after which a hyperspectral image with 12 nm spectral resolution at 450-700 nm was taken. The optical density spectra was measured across the area of the naevus.

RESULTS

Nine patients with 11 naevi were examined. The visual acuity was not affected by any of the naevi. All the naevi were flat as measured either with the optical coherence tomography and/or on inspection, and only one naevi had a risk factor (orange pigmentation). The Wasserstein distance between the background and the naevi was higher at 695 nm compared to 555 nm (p = .002). The naevi could be grouped into three clusters based on the extracted optical density spectra.

CONCLUSION

Choroidal naevi are better visible in longer wavelengths compared to shorter wavelengths. This finding can be used to contour and follow choroidal naevi. Choroidal naevi expose different optical density spectra that can be grouped into three different clusters. One of these clusters has an optical density spectra resembling the absorption spectra of lipofuscin, which may indicate the content of this pigment.

Test-time augmentation with synthetic data addresses distribution shifts in spectral imaging

PURPOSE

Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically.

METHODS

Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation.

RESULTS

The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times.

CONCLUSION

Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging.

Artificial neural networks trained on simulated multispectral data for real-time imaging of skin microcirculatory blood oxygen saturation

Significance

Imaging blood oxygen saturation (SO 2 ) in the skin can be of clinical value when studying ischemic tissue. Emerging multispectral snapshot cameras enable real-time imaging but are limited by slow analysis when using inverse Monte Carlo (MC), the gold standard for analyzing multispectral data. Using artificial neural networks (ANNs) facilitates a significantly faster analysis but requires a large amount of high-quality training data from a wide range of tissue types for a precise estimation ofSO 2 .

Aim

We aim to develop a framework for training ANNs that estimatesSO 2 in real time from multispectral data with a precision comparable to inverse MC.

Approach

ANNs are trained using synthetic data from a model that includes MC simulations of light propagation in tissue and hardware characteristics. The model includes physiologically relevant variations in optical properties, unique sensor characteristics, variations in illumination spectrum, and detector noise. This approach enables a rapid way of generating high-quality training data that covers different tissue types and skin pigmentation.

Results

The ANN implementation analyzes an image in 0.11 s, which is at least 10,000 times faster than inverse MC. The hardware modeling is significantly improved by an in-house calibration of the sensor spectral response. An in-vivo example shows that inverse MC and ANN give almost identicalSO 2 values with a mean absolute deviation of 1.3%-units.

Conclusions

ANN can replace inverse MC and enable real-time imaging of microcirculatorySO 2 in the skin if detailed and precise modeling of both tissue and hardware is used when generating training data.

Advancements in non-invasive hyperspectral imaging: Mapping blood oxygen levels and vascular health for clinical and research applications

Oxygen content is crucial for the functioning of human body organs, as it plays a vital role in cellular respiration, which generates energy necessary for life-sustaining functions. The absence of adequate oxygen leads to cellular dysfunction and eventual organismal death due to energy deprivation. In this study, we designed a rapid, non-invasive, and non-contact custom hyperspectral imaging system to assess blood perfusion in arteries, capillaries, and veins across various human organs, including the arm, eye, and leg. The system recorded cube images consisting of multispectral image ranges, capturing spectral information in both the visible and infrared spectra. Segmentation of the visible spectrum (400 to 700 nm) and the infrared spectrum (700 to 1000 nm) facilitated the mapping of blood oxygen levels in the investigated samples. The estimated oxygen levels were calculated using the custom hyperspectral imaging system and associated algorithm, with validation and calibration performed against the gold standard pulse oximeter. Our results demonstrate that the custom hyperspectral imaging system accurately mapped blood perfusion and oxygen levels in organs, showing strong agreement with pulse oximeter measurements. This study underscores the utility of custom hyperspectral imaging in non-invasively assessing blood oxygenation and perfusion in human organs, offering a promising avenue for clinical diagnostics and monitoring of vascular health.

Hyperspectral retinal imaging biomarkers of ocular and systemic diseases

Hyperspectral imaging is a frontier in the field of medical imaging technology. It enables the simultaneous collection of spectroscopic and spatial data. Structural and physiological information encoded in these data can be used to identify and localise typically elusive biomarkers. Studies of retinal hyperspectral imaging have provided novel insights into disease pathophysiology and new ways of non-invasive diagnosis and monitoring of retinal and systemic diseases. This review provides a concise overview of recent advances in retinal hyperspectral imaging.

Ultra-high-speed four-dimensional hyperspectral imaging

We propose, to the best of our knowledge, a novel deep learning-enabled four-dimensional spectral imaging system composed of a reflective coded aperture snapshot spectral imaging system and a panchromatic camera. The system simultaneously captures a compressively coded hyperspectral measurement and a panchromatic measurement. The hyperspectral data cube is recovered by the U-net-3D network. The depth information of the scene is then acquired by estimating a disparity map between the hyperspectral data cube and the panchromatic measurement through stereo matching. This disparity map is used to align the hyperspectral data cube and the panchromatic measurement. A designed fusion network is used to improve the spatial reconstruction of the hyperspectral data cube by fusing aligned panchromatic measurements. The hardware prototype of the proposed system demonstrates high-speed four-dimensional spectral imaging that allows for simultaneously acquiring depth and spectral images with an 8 nm spectral resolution between 450 and 700 nm, 2.5 mm depth accuracy, and a 1.83 s reconstruction time.

Facilitating clinically relevant skin tumor diagnostics with spectroscopy-driven machine learning

In the dawning era of artificial intelligence (AI), health care stands to undergo a significant transformation with the increasing digitalization of patient data. Digital imaging, in particular, will serve as an important platform for AI to aid decision making and diagnostics. A growing number of studies demonstrate the potential of automatic pre-surgical skin tumor delineation, which could have tremendous impact on clinical practice. However, current methods rely on having ground truth images in which tumor borders are already identified, which is not clinically possible. We report a novel approach where hyperspectral images provide spectra from small regions representing healthy tissue and tumor, which are used to generate prediction maps using artificial neural networks (ANNs), after which a segmentation algorithm automatically identifies the tumor borders. This circumvents the need for ground truth images, since an ANN model is trained with data from each individual patient, representing a more clinically relevant approach.

Feasibility Study of Scanning Spectral Imaging Based on a Birefringence Flat Plate

Hyper-spectral imaging (HSI) systems can be divided into two main types as follows: a group of systems that includes a dedicated dispersion/filtering component whose role is to physically separate the different wavelengths and a group of systems that sample all wavelengths in parallel, so that the separation into wavelengths is performed by signal processing (interferometric method). There is a significant advantage to systems of the second type in terms of the integration time required to obtain a signal with a high signal-to-noise ratio since the signal-to-noise ratio of methods based on scanning interferometry (Windowing method) is better compared to methods based on dispersion. The current research deals with the feasibility study of a new concept for an HSI system that is based on scanning interferometry using the "push-broom" method. In this study, we investigated the viability of incorporating a simple birefringent plate into a scanning optical system. By exploiting the motion of the platform on which the system is mounted, we extracted the spectral information of the scanned region. This approach combines the benefits of scanning interferometry with the simplicity of the setup. According to the theory, a chirped cosine-shaped interferogram is obtained for each wavelength due to the nonlinear behavior of the optical path difference of light in the birefringent plate as a function of the angle. An algorithm converts the signal from a superposition of chirped cosine signals to a scaled interferogram such that Fourier transforming (FT) the interferogram retrieves the spectral information. This innovative idea can turn a simple monochrome camera into a hyperspectral camera by adding a relief lens and a birefringent plate.

AI-Driven Spectral Decomposition: Predicting the Most Probable Protein Compositions from Surface Enhanced Raman Spectroscopy Spectra of Amino Acids

Surface-enhanced Raman spectroscopy (SERS) is a powerful tool for elucidating the molecular makeup of materials. It possesses the unique characteristics of single-molecule sensitivity and extremely high specificity. However, the true potential of SERS, particularly in capturing the biochemical content of particles, remains underexplored. In this study, we harnessed transformer neural networks to interpret SERS spectra, aiming to discern the amino acid profiles within proteins. By training the network on the SERS profiles of 20 amino acids of human proteins, we explore the feasibility of predicting the predominant proteins within the µL-scale detection volume of SERS. Our results highlight a consistent alignment between the model's predictions and the protein's known amino acid compositions, deepening our understanding of the inherent information contained within SERS spectra. For instance, the model achieved low root mean square error (RMSE) scores and minimal deviation in the prediction of amino acid compositions for proteins such as Bovine Serum Albumin (BSA), ACE2 protein, and CD63 antigen. This novel methodology offers a robust avenue not only for protein analytics but also sets a precedent for the broader realm of spectral analyses across diverse material categories. It represents a solid step forward to establishing SERS-based proteomics.

Optical Methods for Brain Tumor Detection: A Systematic Review

Background: In brain tumor surgery, maximal tumor resection is typically desired. This is complicated by infiltrative tumor cells which cannot be visually distinguished from healthy brain tissue. Optical methods are an emerging field that can potentially revolutionize brain tumor surgery through intraoperative differentiation between healthy and tumor tissues. Methods: This study aimed to systematically explore and summarize the existing literature on the use of Raman Spectroscopy (RS), Hyperspectral Imaging (HSI), Optical Coherence Tomography (OCT), and Diffuse Reflectance Spectroscopy (DRS) for brain tumor detection. MEDLINE, Embase, and Web of Science were searched for studies evaluating the accuracy of these systems for brain tumor detection. Outcome measures included accuracy, sensitivity, and specificity. Results: In total, 44 studies were included, covering a range of tumor types and technologies. Accuracy metrics in the studies ranged between 54 and 100% for RS, 69 and 99% for HSI, 82 and 99% for OCT, and 42 and 100% for DRS. Conclusions: This review provides insightful evidence on the use of optical methods in distinguishing tumor from healthy brain tissue.

Semi-Supervised Multiscale Dynamic Graph Convolution Network for Hyperspectral Image Classification

In recent years, convolutional neural networks (CNNs)-based methods achieve cracking performance on hyperspectral image (HSI) classification tasks, due to its hierarchical structure and strong nonlinear fitting capacity. Most of them, however, are supervised approaches that need a large number of labeled data to train them. Conventional convolution kernels are fixed shape of rectangular with fixed sizes, which are good at capturing short-range relations between pixels within HSIs but ignore the long-range context within HSIs, limiting their performance. To overcome the limitations mentioned above, we present a dynamic multiscale graph convolutional network (GCN) classifier (DMSGer). DMSGer first constructs a relatively small graph at region-level based on a superpixel segmentation algorithm and metric-learning. A dynamic pixel-level feature update strategy is then applied to the region-level adjacency matrix, which can help DMSGer learn the pixel representation dynamically. Finally, to deeply understand the complex contents within HSIs, our model is expanded into a multiscale version. On the one hand, by introducing graph learning theory, DMSGer accomplishes HSI classification tasks in a semi-supervised manner, relieving the pressure of collecting abundant labeled samples. Superpixels are generally in irregular shapes and sizes which can group only similar pixels in a neighborhood. On the other hand, based on the proposed dynamic-GCN, the pixel-level and region-level information can be captured simultaneously in one graph convolution layer such that the classification results can be improved. Also, due to the proper multiscale expansion, more helpful information can be captured from HSIs. Extensive experiments were conducted on four public HSIs, and the promising results illustrate that our DMSGer is robust in classifying HSIs. Our source codes are available at https://github.com/TangXu-Group/DMSGer.

Increased susceptibility to ischemia causes exacerbated response to microinjuries in the cirrhotic liver

Fractional laser ablation is a technique developed in dermatology to induce remodeling of skin scars by creating a dense pattern of microinjuries. Despite remarkable clinical results, this technique has yet to be tested for scars in other tissues. As a first step toward determining the suitability of this technique, we aimed to (1) characterize the response to microinjuries in the healthy and cirrhotic liver, and (2) determine the underlying cause for any differences in response. Healthy and cirrhotic rats were treated with a fractional laser then euthanized from 0 h up to 14 days after treatment. Differential expression was assessed using RNAseq with a difference-in-differences model. Spatial maps of tissue oxygenation were acquired with hyperspectral imaging and disruptions in blood supply were assessed with tomato lectin perfusion. Healthy rats showed little damage beyond the initial microinjury and healed completely by 7 days without scarring. In cirrhotic rats, hepatocytes surrounding microinjury sites died 4-6 h after ablation, resulting in enlarged and heterogeneous zones of cell death. Hepatocytes near blood vessels were spared, particularly near the highly vascularized septa. Gene sets related to ischemia and angiogenesis were enriched at 4 h. Laser-treated regions had reduced oxygen saturation and broadly disrupted perfusion of nodule microvasculature, which matched the zones of cell death. Our results demonstrate that the cirrhotic liver has an exacerbated response to microinjuries and increased susceptibility to ischemia from microvascular damage, likely related to the vascular derangements that occur during cirrhosis development. Modifications to the fractional laser tool, such as using a femtosecond laser or reducing the spot size, may be able to prevent large disruptions of perfusion and enable further development of a laser-induced microinjury treatment for cirrhosis.

Development of a large volume line scanning, high spectral range and resolution 3D hyperspectral photoluminescence imaging microscope for diamond and other high refractive index materials

Hyperspectral photoluminescence (PL) imaging is a powerful technique that can be used to understand the spatial distribution of emitting species in many materials. Volumetric hyperspectral imaging of weakly emitting color centers often necessitates considerable data collection times when using commercial systems. We report the development of a line-scanning hyperspectral imaging microscope capable of measuring the luminescence emission spectra for diamond volumes up to 2.20 × 30.00 × 6.30 mm with a high lateral spatial resolution of 1-3 µm. In an single X-λ measurement, spectra covering a 711 nm range, in a band from 400-1100 nm, with a spectral resolution up to 0.25 nm can be acquired. Data sets can be acquired with 723 (X) × 643 (Y) × 1172 (λ) pixels at a rate of 6 minutes/planar image slice, allowing for volumetric hyperspectral imaging with high sampling. This instrument demonstrates the ability to detect emission from several different color centers in diamond both at the surface and internally, providing a non-destructive method to probe their 3D spatial distribution, and is currently not achievable with any other commonly used system or technique.

Direct object detection with snapshot multispectral compressed imaging in a short-wave infrared band

Snapshot multispectral imaging (SMSI) has attracted much attention in recent years for its compact structure and superior performance. High-level image analysis based on SMSI, such as object classification and recognition, usually takes the image reconstruction as the first step, which hinders its application in many important real-time scenarios. Here we demonstrate the first, to our knowledge, reconstruction-free strategy for object detection with SMSI in the short-wave infrared (SWIR) band. The implementation of our SMSI is based on a modified 4f system which modulates the light with a random phase mask, and the distinctive point spread function in each narrowband endows the system with spectrum resolving ability. A deep learning network with a CenterNet structure is trained to detect a small object by constructing a dataset with the PSF of our SMSI system and the sky images as background. Our results indicate that a small object with a spectral feature can be detected directly with the compressed image output by our SMSI system. This work paves the way toward the use of SMSI to detect a multispectral object in practical applications.

Hyperspectral and multispectral imaging in neurosurgery: a systematic literature review and meta-analysis

INTRODUCTION

The neuro-surgical community is witnessing a rising interest for surgical application of multispectral/hyperspectral imaging. Several potential technical applications of this optical imaging are reported, but the set-up is variable and so are the processing methods. We present a systematic review of the relevant literature on the topic.

MATERIALS AND METHODS

A literature search based on the PRISMA principles was performed on PubMed, SCOPUS, and Web of Science, using MESH terms and Boolean operators. Papers regarding intra-operative in-vivo application of multispectral and/or hyperspectral imaging in humans during neurosurgical procedures were included. Papers reporting technologies related to radiological applications were excluded. A meta-analysis on the performance metrics was also conducted.

RESULTS

Our search string retrieved 20 papers. The main applications of optical imaging during neurosurgery concern tumour detection and improvement of the extent of resection (15 papers) or visualization of perfusion changes during neuro-oncology or neuro-vascular surgery (5 papers). All the retrieved articles were pilot studies, proof of concepts, or case reports, with limited number of patients recruited. Sensitivity, specificity, and accuracy were promising in most of the reports, but the metanalysis showed heterogeneous approaches and results among studies.

CONCLUSIONS

The present review shows that several approaches are currently being tested to integrate hyperspectral imaging in neurosurgery, but most of the studies reported a limited pool of patients, with different approaches to data collection and analysis. Further studies on larger cohorts of patients are therefore desirable to fully explore the potential of this imaging technique.

Deep neural networks can differentiate thyroid pathologies on infrared hyperspectral images

BACKGROUND AND OBJECTIVE

The thyroid is a gland responsible for producing important body hormones. Several pathologies can affect this gland, such as thyroiditis, hypothyroidism, and thyroid cancer. The visual histological analysis of thyroid specimens is a valuable process that enables pathologists to detect diseases with high efficiency, providing the patient with a better prognosis. Existing computer vision systems developed to aid in the analysis of histological samples have limitations in distinguishing pathologies with similar characteristics or samples containing multiple diseases. To overcome this challenge, hyperspectral images are being studied to represent biological samples based on their molecular interaction with light.

METHODS

In this study, we address the acquisition of infrared absorbance spectra from each voxel of histological specimens. This data is then used for the development of a multiclass fully-connected neural network model that discriminates spectral patterns, enabling the classification of voxels as healthy, cancerous, or goiter.

RESULTS

Through experiments using the k-fold cross-validation protocol, we obtained an average accuracy of 93.66 %, a sensitivity of 93.47 %, and a specificity of 96.93 %. Our results demonstrate the feasibility of using infrared hyperspectral imaging to characterize healthy tissue and thyroid pathologies using absorbance measurements. The proposed deep learning model has the potential to improve diagnostic efficiency and enhance patient outcomes.

Hyperspectral imaging and evaluation of surgical margins: where do we stand?

PURPOSE OF REVIEW

To highlight the recent literature on the use of hyperspectral imaging (HSI) for cancer margin evaluation ex vivo, for head and neck cancer pathology and in vivo during head and neck cancer surgery.

RECENT FINDINGS

HSI can be used ex vivo on unstained and stained tissue sections to analyze head and neck tissue and tumor cells in combination with machine learning approaches to analyze head and neck cancer cell characteristics and to discriminate the tumor border from normal tissue. Data on in vivo applications during head and neck cancer surgery are preliminary and limited. Even now an accuracy of 80% for tumor versus nonneoplastic tissue classification can be achieved for certain tasks, within the current in vivo settings.

SUMMARY

Significant progress has been made to introduce HSI for ex vivo head and neck cancer pathology evaluation and for an intraoperative use to define the tumor margins. To optimize the accuracy for in vivo use, larger HSI databases with annotations for head and neck cancer are needed.

Mapping cutaneous field carcinogenesis of nonmelanoma skin cancer using mesoscopic imaging of pro-inflammation cues

Nonmelanoma skin cancers remain the most widely diagnosed types of cancers globally. Thus, for optimal patient management, it has become imperative that we focus our efforts on the detection and monitoring of cutaneous field carcinogenesis. The concept of field cancerization (or field carcinogenesis), introduced by Slaughter in 1953 in the context of oral cancer, suggests that invasive cancer may emerge from a molecularly and genetically altered field affecting a substantial area of underlying tissue including the skin. A carcinogenic field alteration, present in precancerous tissue over a relatively large area, is not easily detected by routine visualization. Conventional dermoscopy and microscopy imaging are often limited in assessing the entire carcinogenic landscape. Recent efforts have suggested the use of noninvasive mesoscopic (between microscopic and macroscopic) optical imaging methods that can detect chronic inflammatory features to identify pre-cancerous and cancerous angiogenic changes in tissue microenvironments. This concise review covers major types of mesoscopic optical imaging modalities capable of assessing pro-inflammatory cues by quantifying blood haemoglobin parameters and hemodynamics. Importantly, these imaging modalities demonstrate the ability to detect angiogenesis and inflammation associated with actinically damaged skin. Representative experimental preclinical and human clinical studies using these imaging methods provide biological and clinical relevance to cutaneous field carcinogenesis in altered tissue microenvironments in the apparently normal epidermis and dermis. Overall, mesoscopic optical imaging modalities assessing chronic inflammatory hyperemia can enhance the understanding of cutaneous field carcinogenesis, offer a window of intervention and monitoring for actinic keratoses and nonmelanoma skin cancers and maximise currently available treatment options.

Targeted multispectral filter array design for the optimization of endoscopic cancer detection in the gastrointestinal tract

Significance

Color differences between healthy and diseased tissue in the gastrointestinal (GI) tract are detected visually by clinicians during white light endoscopy; however, the earliest signs of cancer are often just a slightly different shade of pink compared to healthy tissue making it hard to detect. Improving contrast in endoscopy is important for early detection of disease in the GI tract during routine screening and surveillance.

Aim

We aim to target alternative colors for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic "chip-on-tip" configuration.

Approach

Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the GI tract. The toolbox was first extended to use additional classification models (k -nearest neighbor, support vector machine, and spectral angle mapper). Using input spectral data from published clinical trials examining the esophagus and colon, we optimized the design of MSFAs with three to nine different bands.

Results

We examined the variation of the spectral and spatial classification accuracies as a function of the number of bands. The MSFA configurations tested showed good classification accuracies when compared to the full hyperspectral data available from the clinical spectra used in these studies.

Conclusion

The ability to retain good classification accuracies with a reduced number of spectral bands could enable the future deployment of multispectral imaging in an endoscopic chip-on-tip configuration using simplified MSFA hardware. Further studies using an expanded clinical dataset are needed to confirm these findings.

CrossU-Net: Dual-modality cross-attention U-Net for segmentation of precancerous lesions in gastric cancer

Gastric precancerous lesions (GPL) significantly elevate the risk of gastric cancer, and precise diagnosis and timely intervention are critical for patient survival. Due to the elusive pathological features of precancerous lesions, the early detection rate is less than 10%, which hinders lesion localization and diagnosis. In this paper, we provide a GPL pathological dataset and propose a novel method for improving the segmentation accuracy on a limited-scale dataset, namely RGB and Hyperspectral dual-modal pathological image Cross-attention U-Net (CrossU-Net). Specifically, we present a self-supervised pre-training model for hyperspectral images to serve downstream segmentation tasks. Secondly, we design a dual-stream U-Net-based network to extract features from different modal images. To promote information exchange between spatial information in RGB images and spectral information in hyperspectral images, we customize the cross-attention mechanism between the two networks. Furthermore, we use an intermediate agent in this mechanism to improve computational efficiency. Finally, we add a distillation loss to align predicted results for both branches, improving network generalization. Experimental results show that our CrossU-Net achieves accuracy and Dice of 96.53% and 91.62%, respectively, for GPL lesion segmentation, providing a promising spectral research approach for the localization and subsequent quantitative analysis of pathological features in early diagnosis.

Computer vision for plant pathology: A review with examples from cocoa agriculture

Plant pathogens can decimate crops and render the local cultivation of a species unprofitable. In extreme cases this has caused famine and economic collapse. Timing is vital in treating crop diseases, and the use of computer vision for precise disease detection and timing of pesticide application is gaining popularity. Computer vision can reduce labour costs, prevent misdiagnosis of disease, and prevent misapplication of pesticides. Pesticide misapplication is both financially costly and can exacerbate pesticide resistance and pollution. Here, we review the application and development of computer vision and machine learning methods for the detection of plant disease. This review goes beyond the scope of previous works to discuss important technical concepts and considerations when applying computer vision to plant pathology. We present new case studies on adapting standard computer vision methods and review techniques for acquiring training data, the use of diagnostic tools from biology, and the inspection of informative features. In addition to an in-depth discussion of convolutional neural networks (CNNs) and transformers, we also highlight the strengths of methods such as support vector machines and evolved neural networks. We discuss the benefits of carefully curating training data and consider situations where less computationally expensive techniques are advantageous. This includes a comparison of popular model architectures and a guide to their implementation.

Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains

In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.

Planar 16-band metasurface-enhanced spectral filter for integrated image sensing

We study theoretically and demonstrate experimentally a 16-band narrow band wavelength selective filter in the near-infrared range. The combination of a pair of distributed Bragg reflectors with a sub-wavelength grating metasurface embedded in the intra-cavity provides a narrow response which can be tuned by adjusting the geometry of the sub-wavelength grating metasurface. The key advantage of this approach is its ease of fabrication, where the spectral response is tuned by merely changing the grating period, resulting in a perfectly planar geometry that can be easily integrated with a broad variety of photodetectors, thus enabling attractive applications such as bio-imaging, time-of-flight sensors and LiDAR. The experimental results are supported by numerical simulations and effective medium theory that unveil the mechanisms that lead to the optical response of the device. It is also shown how the polarization dependence of the structure can be used to determine very accurately the polarization of incoming light.

Machine Learning Method and Hyperspectral Imaging for Precise Determination of Glucose and Silicon Levels

This article introduces an algorithm for detecting glucose and silicon levels in solution. The research focuses on addressing the critical need for accurate and efficient glucose monitoring, particularly in the context of diabetic management. Understanding and monitoring silicon levels in the body is crucial due to its significant role in various physiological processes. Silicon, while often overshadowed by other minerals, plays a vital role in bone health, collagen formation, and connective tissue integrity. Moreover, recent research suggests its potential involvement in neurological health and the prevention of certain degenerative diseases. Investigating silicon levels becomes essential for a comprehensive understanding of its impact on overall health and well-being and paves the way for targeted interventions and personalized healthcare strategies. The approach presented in this paper is based on the integration of hyperspectral data and artificial intelligence techniques. The algorithm investigates the effectiveness of two distinct models utilizing SVMR and a perceptron independently. SVMR is employed to establish a robust regression model that maps input features to continuous glucose and silicon values. The study outlines the methodology, including feature selection, model training, and evaluation metrics. Experimental results demonstrate the algorithm's effectiveness at accurately predicting glucose and silicon concentrations and showcases its potential for real-world application in continuous glucose and silicon monitoring systems.

HyperVein: A Hyperspectral Image Dataset for Human Vein Detection

HyperSpectral Imaging (HSI) plays a pivotal role in various fields, including medical diagnostics, where precise human vein detection is crucial. HyperSpectral (HS) image data are very large and can cause computational complexities. Dimensionality reduction techniques are often employed to streamline HS image data processing. This paper presents a HS image dataset encompassing left- and right-hand images captured from 100 subjects with varying skin tones. The dataset was annotated using anatomical data to represent vein and non-vein areas within the images. This dataset is utilised to explore the effectiveness of dimensionality reduction techniques, namely: Principal Component Analysis (PCA), Folded PCA (FPCA), and Ward's Linkage Strategy using Mutual Information (WaLuMI) for vein detection. To generate experimental results, the HS image dataset was divided into train and test datasets. Optimum performing parameters for each of the dimensionality reduction techniques in conjunction with the Support Vector Machine (SVM) binary classification were determined using the Training dataset. The performance of the three dimensionality reduction-based vein detection methods was then assessed and compared using the test image dataset. Results show that the FPCA-based method outperforms the other two methods in terms of accuracy. For visualization purposes, the classification prediction image for each technique is post-processed using morphological operators, and results show the significant potential of HS imaging in vein detection.

Dual-camera laparoscopic imaging with super-resolution reconstruction for intraoperative hyperspectral image guidance

Laparoscopic and robotic surgery, as one type of minimally invasive surgery (MIS), has gained popularity due to the improved surgeon ergonomics, instrument precision, operative time, and postoperative recovery. Hyperspectral imaging (HSI) is an emerging medical imaging modality, which has proved useful for intraoperative image guidance. Snapshot hyperspectral cameras are ideal for intraoperative laparoscopic imaging because of their compact size and light weight, but low spatial resolution can be a limitation. In this work, we developed a dual-camera laparoscopic imaging system that consists of a high-resolution color camera and a snapshot hyperspectral camera, and we employed super-resolution reconstruction to fuse the images from both cameras to generate high-resolution hyperspectral images. The experimental results show that our method can significantly improve the resolution of hyperspectral images without compromising the image quality or spectral signatures. The proposed super-resolution reconstruction method is promising to promote the employment of high-speed hyperspectral imaging in laparoscopic surgery.

An Ensemble Learning Method for Detection of Head and Neck Squamous Cell Carcinoma Using Polarized Hyperspectral Microscopic Imaging

Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.

Nerve Detection and Visualization Using Hyperspectral Imaging for Surgical Guidance

During surgery of delicate regions, differentiation between nerve and surrounding tissue is crucial. Hyperspectral imaging (HSI) techniques can enhance the contrast between types of tissue beyond what the human eye can differentiate. Whereas an RGB image captures 3 bands within the visible light range (e.g., 400 nm to 700 nm), HSI can acquire many bands in wavelength increments that highlight regions of an image across a wavelength spectrum. We developed a workflow to identify nerve tissues from other similar tissues such as fat, bone, and muscle. Our workflow uses spectral angle mapper (SAM) and endmember selection. The method is robust for different types of environment and lighting conditions. We validated our workflow on two samples of human tissues. We used a compact HSI system that can image from 400 to 1700 nm to produce HSI of the samples. On these two samples, we achieved an intersection-over-union (IoU) segmentation score of 84.15% and 76.73%, respectively. We showed that our workflow identifies nerve segments that are not easily seen in RGB images. This method is fast, does not rely on special hardware, and can be applied in real time. The hyperspectral imaging and nerve detection approach may provide a powerful tool for image-guided surgery.

Multispectral indices for real-time and non-invasive tissue ischemia monitoring using snapshot cameras

An adequate supply of oxygen-rich blood is vital to maintain cell homeostasis, cellular metabolism, and overall tissue health. While classical methods of measuring tissue ischemia are often invasive, localized and require skin contact or contrast agents, spectral imaging shows promise as a non-invasive, wide field, and contrast-free approach. We evaluate three novel reflectance-based spectral indices from the 460 - 840 nm spectral range. With the aim of enabling real time visualization of tissue ischemia, information is extracted from only 2-3 spectral bands. Video-rate spectral data was acquired from arm occlusion experiments in 27 healthy volunteers. The performance of the indices was evaluated against binary Support Vector Machine (SVM) classification of healthy versus ischemic skin tissue, two other indices from literature, and tissue oxygenation estimated using spectral unmixing. Robustness was tested by evaluating these under various lighting conditions and on both the dorsal and palmar sides of the hand. A novel index with real-time capabilities using reflectance information only from 547 nm and 556 nm achieves an average classification accuracy of 88.48, compared to 92.65 using an SVM trained on all available wavelengths. Furthermore, the index has a higher accuracy compared to reference methods and its time dynamics compare well against the expected clinical responses. This holds promise for robust real-time detection of tissue ischemia, possibly contributing to improved patient care and clinical outcomes.

5-ALA induced PpIX fluorescence spectroscopy in neurosurgery: a review

The review begins with an overview of the fundamental principles/physics underlying light, fluorescence, and other light-matter interactions in biological tissues. It then focuses on 5-aminolevulinic acid (5-ALA)-induced protoporphyrin IX (PpIX) fluorescence spectroscopy methods used in neurosurgery (e.g., intensity, time-resolved) and in so doing, describe their specific features (e.g., hardware requirements, main processing methods) as well as their strengths and limitations. Finally, we review current clinical applications and future directions of 5-ALA-induced protoporphyrin IX (PpIX) fluorescence spectroscopy in neurosurgery.

Fertility detection of unincubated chicken eggs by hyperspectral transmission imaging in the Vis-SWNIR region

Detection of infertile eggs prior to incubation can lead to an increase in the hatchability rate and prevent the wastage of billions of non-fertile eggs ended up by failed incubation. In this study, the feasibility of a line-scan hyperspectral imaging system in the visible and short-wavelength near-infrared region was assessed for early detection of non-fertile eggs on day 0 before incubation. A total of 227 white-shell eggs including 131 fertile and 96 infertile eggs were collected from a flock with similar conditions in terms of hen age, feeding, and management. Hyperspectral images of eggs were captured on day 0 before incubation in a transmittance mode of illumination and then the eggs were incubated in a commercial incubator. The edge detection method was used to segment the egg, including both the white and yolk, from the background, and the image spectral information was extracted from the egg region. After applying various pretreatment methods, different classifiers including soft independent modeling of class analogy (SIMCA), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and artificial neural networks (ANN) classifiers were utilized to extract the predictive models. Following the acceptable results of SIMCA analysis accomplished by 1st derivative pretreatment (accuracy of 86.67%), the discrimination power plot was used to select the most informative wavebands. The results showed that by using fewer variables in effective wavebands better performance (precision and accuracy of 92.59% and 93.33%, respectively) could be obtained in comparison with the ANN classifier based on the whole spectral data (precision and accuracy of 89.29% and 91.11%, respectively). This study revealed the potential application of hyperspectral transmittance imaging in the Vis-SWNIR region to discern the fertile and infertile eggs before starting the incubation process.

Multi-wavelength interference phase imaging for automatic breast cancer detection and delineation using diffuse reflection imaging

Millions of women globally are impacted by the major health problem of breast cancer (BC). Early detection of BC is critical for successful treatment and improved survival rates. In this study, we provide a progressive approach for BC detection using multi-wavelength interference (MWI) phase imaging based on diffuse reflection hyperspectral (HS) imaging. The proposed findings are based on the measurement of the interference pattern between the blue (446.6 nm) and red (632 nm) wavelengths. We consider implementing a comprehensive image processing and categorization method based on the use of Fast Fourier (FF) transform analysis pertaining to a change in the refractive index between tumor and normal tissue. We observed that cancer growth affects tissue organization dramatically, as seen by persistently increased refractive index variance in tumors compared normal areas. Both malignant and normal tissue had different depth data collected from it that was analyzed. To enhance the categorization of ex-vivo BC tissue, we developed and validated a training classifier algorithm specifically designed for categorizing HS cube data. Following the application of signal normalization with the FF transform algorithm, our methodology achieved a high level of performance with a specificity (Spec) of 94% and a sensitivity (Sen) of 90.9% for the 632 nm acquired image categorization, based on preliminary findings from breast specimens under investigation. Notably, we successfully leveraged unstained tissue samples to create 3D phase-resolved images that effectively highlight the distinctions in diffuse reflectance features between cancerous and healthy tissue. Preliminary data revealed that our imaging method might be able to assist specialists in safely excising malignant areas and assessing the tumor bed following resection automatically at different depths. This preliminary investigation might result in an effective "in-vivo" disease description utilizing optical technology using a typical RGB camera with wavelength-specific operation with our quantitative phase MWI imaging methodology.

Gas Plasma Exposure Alters Microcirculation and Inflammation during Wound Healing in a Diabetic Mouse Model

Diabetes can disrupt physiological wound healing, caused by decreased levels or impaired activity of angiogenic factors. This can contribute to chronic inflammation, poor formation of new blood vessels, and delayed re-epithelialization. The present study describes the preclinical application of medical gas plasma to treat a dermal, full-thickness ear wound in streptozotocin (STZ)-induced diabetic mice. Gas plasma-mediated effects occurred in both sexes but with gender-specific differences. Hyperspectral imaging demonstrated gas plasma therapy changing microcirculatory parameters, particularly oxygen saturation levels during wound healing, presumably due to the gas plasma's tissue delivery of reactive species and other bioactive components. In addition, gas plasma treatment significantly affected cell adhesion by regulating focal adhesion kinase and vinculin, which is important in maintaining skin barrier function by regulating syndecan expression and increasing re-epithelialization. An anticipated stimulation of blood vessel formation was detected via transcriptional and translational increase of angiogenic factors in gas plasma-exposed wound tissue. Moreover, gas plasma treatment significantly affected inflammation by modulating systemic growth factors and cytokine levels. The presented findings may help explain the mode of action of successful clinical plasma therapy of wounds of diabetic patients.

Polarized Hyperspectral Microscopic Imaging for Zebrafish

Zebrafish is a well-established animal model for developmental and disease studies. Its optical transparency at early developmental stages is ideal for tissue visualization. Interaction of light with zebrafish tissues provides information on their structure and properties. In this study, we developed a microscopic imaging system for improving the visualization of unstained zebrafish tissues on tissue slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector data in two imaging setups (polarized light imaging and polarized hyperspectral imaging) are capable of improving the visualization of different types of zebrafish tissues (brain, muscle, skin cells, blood vessels, and yolk). Using the images collected from larval zebrafish samples by polarized light imaging, we found that DOP and DOLP could show clearer structural information of the brain and of skin cells, muscle and blood vessels in the tail. Furthermore, DOP and DOLP parameters derived from images collected by polarized hyperspectral imaging could show clearer structural information of skin cells developing around yolk as well as the surrounding blood vessel network. In addition, polarized hyperspectral imaging could provide complementary spectral information to the spatial information on Stokes vector data of zebrafish tissues. The polarized light imaging & polarized hyperspectral imaging systems provide a better insight into the microstructures of zebrafish tissues.

A priori free spectral unmixing with periodic absorbance changes: application for auto-calibrated intraoperative functional brain mapping

Spectral unmixing designates techniques that allow to decompose measured spectra into linear or non-linear combination of spectra of all targets (endmembers). This technique was initially developed for satellite applications, but it is now also widely used in biomedical applications. However, several drawbacks limit the use of these techniques with standard optical devices like RGB cameras. The devices need to be calibrated and a a priori on the observed scene is often necessary. We propose a new method for estimating endmembers and their proportion automatically and without calibration of the acquisition device based on near separable non-negative matrix factorization. This method estimates the endmembers on spectra of absorbance changes presenting periodic events. This is very common in in vivo biomedical and medical optical imaging where hemodynamics dominate the absorbance fluctuations. We applied the method for identifying functional brain areas during neurosurgery using four different RGB cameras (an industrial camera, a smartphone and two surgical microscopes). Results obtained with the auto-calibration method were consistent with the intraoperative gold standards. Endmembers estimated with the auto-calibration method were similar to the calibrated endmembers used in the modified Beer-Lambert law. The similarity was particularly strong when both cardiac and respiratory periodic events were considered. This work can allow a widespread use of spectral imaging in the industrial or medical field.

Registration of hyperspectral images and mass spectrometry data for the correlation of tissue optical spectra and molecular profiles

Hyperspectral imaging (HSI) is a label-free imaging modality that is emerging for non-invasive detection of various diseases including cancers. HSI provides high-resolution spatial images where each pixel has a spectral curve with numerous wavelength bands from the visible to infrared ranges. The rich spatial and spectral information can be used to discriminate various types of tissues and pathophysiological conditions. However, it can be difficult to explain spectral data with respect to the underline cellular and molecular mechanism. In this study, we developed an approach that registers hyperspectral images and mass spectrometry (MS) data where MS provides tissue molecular profiles. Human prostate tissues that were obtained after prostatectomy were used in the experiments. The whole prostate was first sliced every six mm. A customized hyperspectral surgical microscope was used to acquire HSI data from the sliced tissue. For MS data analysis, the sliced tissue of the prostate was divided into 51 small regions and then processed separately for each region. The immediately adjacent tissue was sliced and processed histologically for H&E staining. The MS molecular profiles were correlated with the hyperspectral images in this study.

A High-Resolution Hyperspectral Imaging System for the Retina

In this study, we developed an imaging system that can acquire and produce high-resolution hyperspectral images of the retina. Our system combines the view from a high-resolution RGB camera and a snapshot hyperspectral camera together. The method is fast and can be constructed into a compact imaging device. We tested our system by imaging a calibrated color chart, biological tissues ex vivo, and a phantom of the human retina. By using image pansharpening methods, we were able to produce a high-resolution hyperspectral image. The images from the hyperspectral camera alone have a spatial resolution of 0.2 mm/pixel, whereas the pansharpened images have a spatial resolution of 0.1 mm/pixel, a 2x increase in spatial resolution. Our method has the potential to capture images of the retina rapidly. Our method preserves both the spatial and spectral fidelity, as shown by comparing the original hyperspectral images with the pansharpened images. The high-resolution hyperspectral imaging device can have a variety of applications in retina examinations.

A dual-camera hyperspectral laparoscopic imaging system

Minimally invasive surgery (MIS) has expanded broadly in the field of abdominal and pelvic surgery. However, there are still prevalent issues surrounding intracorporeal surgery, such as iatrogenic injury, anastomotic leakage, or the presence of positive tumor margins after resection. Current approaches to address these issues and advance laparoscopic imaging techniques often involve fluorescence imaging agents, such as indocyanine green (ICG), to improve visualization, but these have drawbacks. Hyperspectral imaging (HSI) is an emerging optical imaging modality that takes advantage of spectral characteristics of different tissues. Various applications include tissue classification and digital pathology. In this study, we developed a dual-camera system for high-speed hyperspectral imaging. This includes the development of a custom application interface and corresponding hardware setup. Characterization of the system was performed, including spectral accuracy and spatial resolution, showing little sacrifice in speed for the approximate doubling of the covered spectral range, with our system acquiring 29 spectral images from 460-850 nm. Reference color tiles with various reflectance profiles were imaged and a RMSE of 3.56 ± 1.36% was achieved. Sub-millimeter resolution was shown at 7 cm working distance for both hyperspectral cameras. Finally, we image ex vivo tissues, including porcine stomach, liver, intestine, and kidney with our system and use a high-resolution, radiometrically calibrated spectrometer for comparison and evaluation of spectral fidelity. The dual-camera hyperspectral laparoscopic imaging system can have immediate applications in various surgeries.

Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma

Significance

Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications.

Aim

The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides.

Approach

We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: S 0 , S 1 , S 2 , and S 3 . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors.

Results

By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides.

Conclusions

With the spatial and spectral information from the Stokes vector data cubes (S 0 , S 1 , S 2 , and S 3 ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

Significance

Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates.

Aim

To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution.

Approach

The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers.

Results

The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156    μ m . The effective frame rate was 20 fps.

Conclusions

High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery.