In-situ detection of microplastics in the aquatic environment: A systematic literature review

Microplastics are ubiquitous in the aquatic environment and have emerged as a significant environmental issue due to their potential impacts on human health and the ecosystem. Current laboratory-based microplastic detection methods suffer from various drawbacks, including a lack of standardisation, limited spatial and temporal coverage, high costs, and time-consuming procedures. Consequently, there is a need for the development of in-situ techniques to detect and monitor microplastics to effectively identify and understand their sources, pathways, and behaviours. Herein, we adopt a systematic literature review method to assess the development and application of experimental and field technologies designed for the in-situ detection and monitoring of aquatic microplastics, without the need for sample preparation. Four scientific databases were searched in March 2023, resulting in a review of 62 relevant studies. These studies were classified into seven sensor categories and their working principles were discussed. The sensor classes include optical devices, digital holography, Raman spectroscopy, other spectroscopy, hyperspectral imaging, remote sensing, and other methods. We also looked at how data from these technologies are integrated with machine learning models to develop classifiers capable of accurately characterising the physical and chemical properties of microplastics and discriminating them from other particles. This review concluded that in-situ detection of microplastics in aquatic environments is feasible and can be achieved with high accuracy, even though the methods are still in the early stages of development. Nonetheless, further research is still needed to enhance the in-situ detection of microplastics. This includes exploring the possibility of combining various detection methods and developing robust machine-learning classifiers. Additionally, there is a recommendation for in-situ implementation of the reviewed methods to assess their effectiveness in detecting microplastics and identify their limitations.

Feasibility Study of Scanning Spectral Imaging Based on a Birefringence Flat Plate

Hyper-spectral imaging (HSI) systems can be divided into two main types as follows: a group of systems that includes a dedicated dispersion/filtering component whose role is to physically separate the different wavelengths and a group of systems that sample all wavelengths in parallel, so that the separation into wavelengths is performed by signal processing (interferometric method). There is a significant advantage to systems of the second type in terms of the integration time required to obtain a signal with a high signal-to-noise ratio since the signal-to-noise ratio of methods based on scanning interferometry (Windowing method) is better compared to methods based on dispersion. The current research deals with the feasibility study of a new concept for an HSI system that is based on scanning interferometry using the "push-broom" method. In this study, we investigated the viability of incorporating a simple birefringent plate into a scanning optical system. By exploiting the motion of the platform on which the system is mounted, we extracted the spectral information of the scanned region. This approach combines the benefits of scanning interferometry with the simplicity of the setup. According to the theory, a chirped cosine-shaped interferogram is obtained for each wavelength due to the nonlinear behavior of the optical path difference of light in the birefringent plate as a function of the angle. An algorithm converts the signal from a superposition of chirped cosine signals to a scaled interferogram such that Fourier transforming (FT) the interferogram retrieves the spectral information. This innovative idea can turn a simple monochrome camera into a hyperspectral camera by adding a relief lens and a birefringent plate.

Metasurface-empowered snapshot hyperspectral imaging with convex/deep (CODE) small-data learning theory

Hyperspectral imaging is vital for material identification but traditional systems are bulky, hindering the development of compact systems. While previous metasurfaces address volume issues, the requirements of complicated fabrication processes and significant footprint still limit their applications. This work reports a compact snapshot hyperspectral imager by incorporating the meta-optics with a small-data convex/deep (CODE) deep learning theory. Our snapshot hyperspectral imager comprises only one single multi-wavelength metasurface chip working in the visible window (500-650 nm), significantly reducing the device area. To demonstrate the high performance of our hyperspectral imager, a 4-band multispectral imaging dataset is used as the input. Through the CODE-driven imaging system, it efficiently generates an 18-band hyperspectral data cube with high fidelity using only 18 training data points. We expect the elegant integration of multi-resonant metasurfaces with small-data learning theory will enable low-profile advanced instruments for fundamental science studies and real-world applications.

Spectrally-shaped illumination for improved optical inspection of lateral III-V-semiconductor oxidation

We report an hyperspectral imaging microscopy system based on a spectrally-shaped illumination and its use to offer an enhanced in-situ inspection of a technological process that is critical in Vertical-Cavity Surface-Emitting Laser (VCSEL) manufacturing, the lateral III-V-semiconductor oxidation (AlOx). The implemented illumination source exploits a digital micromirror device (DMD) to arbitrarily tailor its emission spectrum. When combined to an imager, this source is shown to provide an additional ability to detect minute surface reflectance contrasts on any VCSEL or AlOx-based photonic structure and, in turn, to offer improved in-situ inspection of the oxide aperture shapes and dimensions down to the best-achievable optical resolution. The demonstrated technique is very versatile and could be readily extended to the real-time monitoring of oxidation or other semiconductor technological processes as soon as they rely on a real-time yet accurate measurement of spatio-spectral (reflectance) maps.

Optimized Method Based on Subspace Merging for Spectral Reflectance Recovery

The similarity between samples is an important factor for spectral reflectance recovery. The current way of selecting samples after dividing dataset does not take subspace merging into account. An optimized method based on subspace merging for spectral recovery is proposed from single RGB trichromatic values in this paper. Each training sample is equivalent to a separate subspace, and the subspaces are merged according to the Euclidean distance. The merged center point for each subspace is obtained through many iterations, and subspace tracking is used to determine the subspace where each testing sample is located for spectral recovery. After obtaining the center points, these center points are not the actual points in the training samples. The nearest distance principle is used to replace the center points with the point in the training samples, which is the process of representative sample selection. Finally, these representative samples are used for spectral recovery. The effectiveness of the proposed method is tested by comparing it with the existing methods under different illuminants and cameras. Through the experiments, the results show that the proposed method not only shows good results in terms of spectral and colorimetric accuracy, but also in the selection representative samples.

Comparative analysis of denoising techniques in burn depth discrimination from burn hyperspectral images

This study analyzes and compares the performance of five denoising techniques (Lee filter, gamma filter, principal component analysis, maximum noise fraction, and wavelet transform) in order to identify the most appropriate one that lead to the most accurate classification of burned tissue in hyperspectral images. Fifteen hyperspectral images of burned patients were acquired and denoising techniques were applied to each image. Spectral angle mapper classifier was used for data classification and the confusion matrix was used for quantitative evaluation of the performances of the denoising methods. The results revealed that gamma filter performed better than other denoising techniques with values of overall accuracy and kappa coefficient of 91.18% and 0.8958 respectively. The lowest performance was detected for principal component analysis. In conclusion, the gamma filter could be considered an optimal choice for noise reduction in burn hyperspectral images and could be used for a more accurate diagnosis of burn depth.

Imaging perfusion changes in oncological clinical applications by hyperspectral imaging: a literature review

BACKGROUND

Hyperspectral imaging (HSI) is a promising imaging modality that uses visible light to obtain information about blood flow. It has the distinct advantage of being noncontact, nonionizing, and noninvasive without the need for a contrast agent. Among the many applications of HSI in the medical field are the detection of various types of tumors and the evaluation of their blood flow, as well as the healing processes of grafts and wounds. Since tumor perfusion is one of the critical factors in oncology, we assessed the value of HSI in quantifying perfusion changes during interventions in clinical oncology through a systematic review of the literature.

MATERIALS AND METHODS

The PubMed and Web of Science electronic databases were searched using the terms "hyperspectral imaging perfusion cancer" and "hyperspectral imaging resection cancer". The inclusion criterion was the use of HSI in clinical oncology, meaning that all animal, phantom, ex vivo, experimental, research and development, and purely methodological studies were excluded.

RESULTS

Twenty articles met the inclusion criteria. The anatomic locations of the neoplasms in the selected articles were as follows: kidneys (1 article), breasts (2 articles), eye (1 article), brain (4 articles), entire gastrointestinal (GI) tract (1 article), upper GI tract (5 articles), and lower GI tract (6 articles).

CONCLUSIONS

HSI is a potentially attractive imaging modality for clinical application in oncology, with assessment of mastectomy skin flap perfusion after reconstructive breast surgery and anastomotic perfusion during reconstruction of gastrointenstinal conduit as the most promising at present.

Adaptive Rank and Structured Sparsity Corrections for Hyperspectral Image Restoration

Hyperspectral images (HSIs) are inevitably contaminated by the mixed noise (such as Gaussian noise, impulse noise, deadlines, and stripes), which could influence the subsequent processing accuracy. Generally, HSI restoration can be transformed into the low-rank matrix recovery (LRMR). In the LRMR, the nuclear norm is widely used to substitute the matrix rank, but its effectiveness is still worth improving. Besides, the l₀ -norm cannot capture the sparse noise's structured sparsity property. To handle these issues, the adaptive rank and structured sparsity corrections (ARSSC) are presented for HSI restoration. The ARSSC introduces two convex regularizers, that is: 1) the rank correction (RC) and 2) the structured sparsity correction (SSC), to, respectively, approximate the matrix rank and the l_2,0 -norm. The RC and the SSC can adaptively offset the penalization of large entries from the nuclear norm and the l_2,1 -norm, respectively, where the larger the entry, the greater its offset. Therefore, the proposed ARSSC achieves a tighter approximation of the noise-free HSI low-rank structure and promotes the structured sparsity of sparse noise. An efficient alternative direction method of multipliers (ADMM) algorithm is applied to solve the resulting convex optimization problem. The superiority of the ARSSC in terms of the mixed noise removal and spatial-spectral structure information preserving, is demonstrated by several experimental results both on simulated and real datasets, compared with other state-of-the-art HSI restoration approaches.

All-Dielectric Transreflective Angle-Insensitive Near-Infrared (NIR) Filter

This paper presents an all-dielectric, cascaded, multilayered, thin-film filter, allowing near-infrared filtration for spectral imaging applications. The proposed design is comprised of only eight layers of amorphous silicon (A-Si) and silicon nitride (Si3N4), successively deposited on a glass substrate. The finite difference time domain (FDTD) simulation results demonstrate a distinct peak in the near-infrared (NIR) region with transmission efficiency up to 70% and a full-width-at-half-maximum (FWHM) of 77 nm. The theoretical results are angle-insensitive up to 60° and show polarization insensitivity in the transverse magnetic (TM) and transverse electric (TE) modes. The theoretical response, obtained with the help of spectroscopic ellipsometry (SE), is in good agreement with the experimental result. Likewise, the experimental results for polarization insensitivity and angle invariance of the thin films are in unison with the theoretical results, having an angle invariance up to 50°.

Densely Residual Network with Dual Attention for Hyperspectral Reconstruction from RGB Images

In the last several years, deep learning has been introduced to recover a hyperspectral image (HSI) from a single RGB image and demonstrated good performance. In particular, attention mechanisms have further strengthened discriminative features, but most of them are learned by convolutions with limited receptive fields or require much computational cost, which hinders the function of attention modules. Furthermore, the performance of these deep learning methods is hampered by tackling multi-level features equally. To this end, in this paper, based on multiple lightweight densely residual modules, we propose a densely residual network with dual attention (DRN-DA), which utilizes advanced attention and adaptive fusion strategy for more efficient feature correlation learning and more powerful feature extraction. Specifically, an SE layer is applied to learn channel-wise dependencies, and dual downsampling spatial attention (DDSA) is developed to capture long-range spatial contextual information. All the intermediate-layer feature maps are adaptively fused. Experimental results on four data sets from the NTIRE 2018 and NTIRE 2020 Spectral Reconstruction Challenges demonstrate the superiority of the proposed DRN-DA over state-of-the-art methods (at least −6.19% and −1.43% on NTIRE 2018 “Clean” and “Real World” track, −6.85% and −5.30% on NTIRE 2020 “Clean” and “Real World” track) in terms of mean relative absolute error.

Adversarial Representation Learning for Hyperspectral Image Classification with Small-Sized Labeled Set

Hyperspectral image (HSI) classification is one of the main research contents of hyperspectral technology. Existing HSI classification algorithms that are based on deep learning use a large number of labeled samples to train models to ensure excellent classification effects, but when the labeled samples are insufficient, the deep learning model is prone to overfitting. In practice, there are a large number of unlabeled samples that have not been effectively utilized, so it is meaningful to study a semi-supervised method. In this paper, an adversarial representation learning that is based on a generative adversarial networks (ARL-GAN) method is proposed to solve the small samples problem in hyperspectral image classification by applying GAN to the representation learning domain in a semi-supervised manner. The proposed method has the following distinctive advantages. First, we build a hyperspectral image block generator whose input is the feature vector that is extracted from the encoder and use the encoder as a feature extractor to extract more discriminant information. Second, the distance of the class probability output by the discriminator is used to measure the error between the generated image block and the real image instead of the root mean square error (MSE), so that the encoder can extract more useful information for classification. Third, GAN and conditional entropy are used to improve the utilization of unlabeled data and solve the small sample problem in hyperspectral image classification. Experiments on three public datasets show that the method achieved better classification accuracy with a small number of labeled samples compared to other state-of-the-art methods.

Hyperspectral Image Classification Based on 3D Coordination Attention Mechanism Network

In recent years, due to its powerful feature extraction ability, the deep learning method has been widely used in hyperspectral image classification tasks. However, the features extracted by classical deep learning methods have limited discrimination ability, resulting in unsatisfactory classification performance. In addition, due to the limited data samples of hyperspectral images (HSIs), how to achieve high classification performance under limited samples is also a research hotspot. In order to solve the above problems, this paper proposes a deep learning network framework named the three-dimensional coordination attention mechanism network (3DCAMNet). In this paper, a three-dimensional coordination attention mechanism (3DCAM) is designed. This attention mechanism can not only obtain the long-distance dependence of the spatial position of HSIs in the vertical and horizontal directions, but also obtain the difference of importance between different spectral bands. In order to extract the spectral and spatial information of HSIs more fully, a convolution module based on convolutional neural network (CNN) is adopted in this paper. In addition, the linear module is introduced after the convolution module, which can extract more fine advanced features. In order to verify the effectiveness of 3DCAMNet, a series of experiments were carried out on five datasets, namely, Indian Pines (IP), Pavia University (UP), Kennedy Space Center (KSC), Salinas Valley (SV), and University of Houston (HT). The OAs obtained by the proposed method on the five datasets were 95.81%, 97.01%, 99.01%, 97.48%, and 97.69% respectively, 3.71%, 9.56%, 0.67%, 2.89% and 0.11% higher than the most advanced A2S2K-ResNet. Experimental results show that, compared with some state-of-the-art methods, 3DCAMNet not only has higher classification performance, but also has stronger robustness.

Variational Generative Adversarial Network with Crossed Spatial and Spectral Interactions for Hyperspectral Image Classification

Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have been widely used in hyperspectral image classification (HSIC) tasks. However, the generated HSI virtual samples by VAEs are often ambiguous, and GANs are prone to the mode collapse, which lead the poor generalization abilities ultimately. Moreover, most of these models only consider the extraction of spectral or spatial features. They fail to combine the two branches interactively and ignore the correlation between them. Consequently, the variational generative adversarial network with crossed spatial and spectral interactions (CSSVGAN) was proposed in this paper, which includes a dual-branch variational Encoder to map spectral and spatial information to different latent spaces, a crossed interactive Generator to improve the quality of generated virtual samples, and a Discriminator stuck with a classifier to enhance the classification performance. Combining these three subnetworks, the proposed CSSVGAN achieves excellent classification by ensuring the diversity and interacting spectral and spatial features in a crossed manner. The superior experimental results on three datasets verify the effectiveness of this method.

Optimized light source spectral power distribution for RGB camera based spectral reflectance recovery

The accuracy of recovered spectra from camera responses mainly depends on the spectral estimation algorithm used, the camera and filters selected, and the light source used to illuminate the object. We present and compare different light source spectrum optimization methods together with different spectral estimation algorithms applied to reflectance recovery. These optimization methods include the Monte Carlo (MC) method, particle swarm optimization (PSO) and multi-population genetic algorithm (MPGA). Optimized SPDs are compared with D65, D50 A and three LED light sources in simulation and reality. Results obtained show us that MPGA has superior performance, and optimized light source spectra along with better spectral estimation algorithm can provide a more accurate spectral reflectance estimation of an object surface. Meanwhile, it is found that camera spectral sensitivities weighted by optimized SPDs tend to be mutually orthogonal.

Compressive spectral image reconstruction using deep prior and low-rank tensor representation

Compressive spectral imaging (CSI) has emerged as an alternative spectral image acquisition technology, which reduces the number of measurements at the cost of requiring a recovery process. In general, the reconstruction methods are based on handcrafted priors used as regularizers in optimization algorithms or recent deep neural networks employed as an image generator to learn a non-linear mapping from the low-dimensional compressed measurements to the image space. However, these deep learning methods need many spectral images to obtain good performance. In this work, a deep recovery framework for CSI without training data is presented. The proposed method is based on the fact that the structure of some deep neural networks and an appropriated low-dimensional structure are sufficient to impose a structure of the underlying spectral image from CSI. We analyzed the low-dimensional structure via the Tucker representation, modeled in the first net layer. The proposed scheme is obtained by minimizing the ${\ell _2}$-norm distance between the compressive measurements and the predicted measurements, and the desired recovered spectral image is formed just before the forward operator. Simulated and experimental results verify the effectiveness of the proposed method for the coded aperture snapshot spectral imaging.

Human Stress and StO2: Database, Features, and Classification of Emotional and Physical Stress

Emotional and physical stress can cause various health problems. In this paper, we used tissue blood oxygen saturation (StO2), a newly proposed physiological signal, to classify the human stress. We firstly constructed a public StO2 database including 42 volunteers subjected to two types of stress. During the physical stress experiment, we observed that the facial StO2 right after the stress can be either increased or decreased comparing to the baseline. We investigated the StO2 feature combinations for the classification and found that the average StO2 values from left cheek, chin, and the middle of the eyebrow can provide the highest classification rate of 95.56%. Comparison with other stress classification method shows that StO2 based method can provide best classification performance with lowest feature dimension. These results suggest that facial StO2 can be used as a promising features to identify stress states, including emotional and physical stress.

Spatial Spectral Band Selection for Enhanced Hyperspectral Remote Sensing Classification Applications

Despite the numerous band selection (BS) algorithms reported in the field, most if not all have exhibited maximal accuracy when more spectral bands are utilized for classification. This apparently disagrees with the theoretical model of the ‘curse of dimensionality’ phenomenon, without apparent explanations. If it were true, then BS would be deemed as an academic piece of research without real benefits to practical applications. This paper presents a spatial spectral mutual information (SSMI) BS scheme that utilizes a spatial feature extraction technique as a preprocessing step, followed by the clustering of the mutual information (MI) of spectral bands for enhancing the efficiency of the BS. Through the SSMI BS scheme, a sharp ’bell’-shaped accuracy-dimensionality characteristic that peaks at about 20 bands has been observed for the very first time. The performance of the proposed SSMI BS scheme has been validated through 6 hyperspectral imaging (HSI) datasets (Indian Pines, Botswana, Barrax, Pavia University, Salinas, and Kennedy Space Center (KSC)), and its classification accuracy is shown to be approximately 10% better than seven state-of-the-art BS schemes (Saliency, HyperBS, SLN, OCF, FDPC, ISSC, and Convolution Neural Network (CNN)). The present result confirms that the high efficiency of the BS scheme is essentially important to observe and validate the Hughes’ phenomenon in the analysis of HSI data. Experiments also show that the classification accuracy can be affected by as much as approximately 10% when a single ‘crucial’ band is included or missed out for classification.

Gudalur Spectral Target Detection (GST-D): A New Benchmark Dataset and Engineered Material Target Detection in Multi-Platform Remote Sensing Data

Target detection in remote sensing imagery, mapping of sparsely distributed materials, has vital applications in defense security and surveillance, mineral exploration, agriculture, environmental monitoring, etc. The detection probability and the quality of retrievals are functions of various parameters of the sensor, platform, target–background dynamics, targets’ spectral contrast, and atmospheric influence. Generally, target detection in remote sensing imagery has been approached using various statistical detection algorithms with an assumption of linearity in the image formation process. Knowledge on the image acquisition geometry, and spectral features and their stability across different imaging platforms is vital for designing a spectral target detection system. We carried out an integrated target detection experiment for the detection of various artificial target materials. As part of this work, we acquired a benchmark multi-platform hyperspectral and multispectral remote sensing dataset named as ‘Gudalur Spectral Target Detection (GST-D)’ dataset. Positioning artificial targets on different surface backgrounds, we acquired remote sensing data by terrestrial, airborne, and space-borne sensors on 20th March 2018. Various statistical and subspace detection algorithms were applied on the benchmark dataset for the detection of targets, considering the different sources of reference target spectra, background, and the spectral continuity across the platforms. We validated the detection results using the receiver operation curve (ROC) for different cases of detection algorithms and imaging platforms. Results indicate, for some combinations of algorithms and imaging platforms, consistent detection of specific material targets with a detection rate of about 80% at a false alarm rate between 10−2 to 10−3. Target detection in satellite imagery using reference target spectra from airborne hyperspectral imagery match closely with the satellite imagery derived reference spectra. The ground-based in-situ reference spectra offer a quantifiable detection in airborne or satellite imagery. However, ground-based hyperspectral imagery has also provided an equivalent target detection in the airborne and satellite imagery paving the way for rapid acquisition of reference target spectra. The benchmark dataset generated in this work is a valuable resourcefor addressing intriguing questions in target detection using hyperspectral imagery from a realistic landscape perspective.

A Two-stage Deep Domain Adaptation Method for Hyperspectral Image Classification

Deep learning has attracted extensive attention in the field of hyperspectral images (HSIs) classification. However, supervised deep learning methods heavily rely on a large amount of label information. To address this problem, in this paper, we propose a two-stage deep domain adaptation method for hyperspectral image classification, which can minimize the data shift between two domains and learn a more discriminative deep embedding space with very few labeled target samples. A deep embedding space is first learned by minimizing the distance between the source domain and the target domain based on Maximum Mean Discrepancy (MMD) criterion. The Spatial–Spectral Siamese Network is then exploited to reduce the data shift and learn a more discriminative deep embedding space by minimizing the distance between samples from different domains but the same class label and maximizes the distance between samples from different domains and class labels based on pairwise loss. For the classification task, the softmax layer is replaced with a linear support vector machine, in which learning minimizes a margin-based loss instead of the cross-entropy loss. The experimental results on two sets of hyperspectral remote sensing images show that the proposed method can outperform several state-of-the-art methods.

Endmember Learning with K-Means through SCD Model in Hyperspectral Scene Reconstructions

This paper proposes a simple yet effective method for improving the efficiency of sparse coding dictionary learning (DL) with an implication of enhancing the ultimate usefulness of compressive sensing (CS) technology for practical applications, such as in hyperspectral imaging (HSI) scene reconstruction. CS is the technique which allows sparse signals to be decomposed into a sparse representation “a” of a dictionary Du. The goodness of the learnt dictionary has direct impacts on the quality of the end results, e.g., in the HSI scene reconstructions. This paper proposes the construction of a concise and comprehensive dictionary by using the cluster centres of the input dataset, and then a greedy approach is adopted to learn all elements within this dictionary. The proposed method consists of an unsupervised clustering algorithm (K-Means), and it is then coupled with an advanced sparse coding dictionary (SCD) method such as the basis pursuit algorithm (orthogonal matching pursuit, OMP) for the dictionary learning. The effectiveness of the proposed K-Means Sparse Coding Dictionary (KMSCD) is illustrated through the reconstructions of several publicly available HSI scenes. The results have shown that the proposed KMSCD achieves ~40% greater accuracy, 5 times faster convergence and is twice as robust as that of the classic Spare Coding Dictionary (C-SCD) method that adopts random sampling of data for the dictionary learning. Over the five data sets that have been employed in this study, it is seen that the proposed KMSCD is capable of reconstructing these scenes with mean accuracies of approximately 20–500% better than all competing algorithms adopted in this work. Furthermore, the reconstruction efficiency of trace materials in the scene has been assessed: it is shown that the KMSCD is capable of recovering ~12% better than that of the C-SCD. These results suggest that the proposed DL using a simple clustering method for the construction of the dictionary has been shown to enhance the scene reconstruction substantially. When the proposed KMSCD is incorporated with the Fast non-negative orthogonal matching pursuit (FNNOMP) to constrain the maximum number of materials to coexist in a pixel to four, experiments have shown that it achieves approximately ten times better than that constrained by using the widely employed TMM algorithm. This may suggest that the proposed DL method using KMSCD and together with the FNNOMP will be more suitable to be the material allocation module of HSI scene simulators like the CameoSim package.

Gaussian Process Graph-Based Discriminant Analysis for Hyperspectral Images Classification

Dimensionality Reduction (DR) models are highly useful for tackling Hyperspectral Images (HSIs) classification tasks. They mainly address two issues: the curse of dimensionality with respect to spectral features, and the limited number of labeled training samples. Among these DR techniques, the Graph-Embedding Discriminant Analysis (GEDA) framework has demonstrated its effectiveness for HSIs feature extraction. However, most of the existing GEDA-based DR methods largely rely on manually tuning the parameters so as to obtain the optimal model, which proves to be troublesome and inefficient. Motivated by the nonparametric Gaussian Process (GP) model, we propose a novel supervised DR algorithm, namely Gaussian Process Graph-based Discriminate Analysis (GPGDA). Our algorithm takes full advantage of the covariance matrix in GP to constructing the graph similarity matrix in GEDA framework. In this way, more superior performance can be provided with the model parameters tuned automatically. Experiments on three real HSIs datasets demonstrate that the proposed GPGDA outperforms some classic and state-of-the-art DR methods.

Compressive ultraspectral imaging using multiscale structured illumination

We present a novel compressive spectral imaging technique that attains spatially resolved ultraspectral resolution. The technique employs a multiscale sampling technique based on the Hadamard basis for the single pixel hyperspectral imager. The proposed multiscale sampling method offers high-quality images at a low compression ratio while also facilitating a preview image at a lower resolution by using the fast Hadamard transform.

Hyperspectral imaging for thermal effect monitoring in in vivo liver during laser ablation

Thermal ablation is a minimally invasive technique used to induce a controlled necrosis of malignant cells by increasing the temperature in localized areas. This procedure needs an accurate and real-time monitoring of thermal effects to evaluate and control treatment outcome. In this work, a hyperspectral imaging (HSI) technique is proposed as a new and non-invasive method to monitor ablative therapy. HSI provides images of the target object in several spectral bands, hence the reflectance/absorbance spectrum for each pixel. This paper presents a preliminary and original HSI-based analysis of the thermal state in the in vivo porcine liver undergoing laser ablation. In order to compare the spectral response between treated and untreated areas of the organ, proper Regions of Interest (ROIs) were chosen on the hyperspectral images; for each ROI, the absorbance variation for the selected wavelengths (i.e., 630, 760, and 960nm, for deoxyhemoglobin, methemoglobin, and water respectively) was assessed. Results obtained during and after laser ablation show that the absorbance of the methemoglobin peaks increases up to 40% in the burned region with respect to the non-ablated one. Conversely, the relative change of deoxyhemoglobin and water peaks is less marked. Based on these results, absorbance threshold values were retrieved and used to visualize the ablation zone on the images. This preliminary analysis suggests that a combination of the absorbance information is essential to achieve a more accurate identification of the ablation region. The results encourage further studies on the correlation between thermal effects and the spectral response of biological tissues undergoing thermal ablation, for final clinical use.

In-Vivo and Ex-Vivo Tissue Analysis through Hyperspectral Imaging Techniques: Revealing the Invisible Features of Cancer

In contrast to conventional optical imaging modalities, hyperspectral imaging (HSI) is able to capture much more information from a certain scene, both within and beyond the visual spectral range (from 400 to 700 nm). This imaging modality is based on the principle that each material provides different responses to light reflection, absorption, and scattering across the electromagnetic spectrum. Due to these properties, it is possible to differentiate and identify the different materials/substances presented in a certain scene by their spectral signature. Over the last two decades, HSI has demonstrated potential to become a powerful tool to study and identify several diseases in the medical field, being a non-contact, non-ionizing, and a label-free imaging modality. In this review, the use of HSI as an imaging tool for the analysis and detection of cancer is presented. The basic concepts related to this technology are detailed. The most relevant, state-of-the-art studies that can be found in the literature using HSI for cancer analysis are presented and summarized, both in-vivo and ex-vivo. Lastly, we discuss the current limitations of this technology in the field of cancer detection, together with some insights into possible future steps in the improvement of this technology.

Archimedean spiral push-broom differential thermal imaging for gas leakage detection

An uncooled infrared focal plane array (FPA) for a multiband optical imaging system monitoring small gas leakages is low in cost but limited by its frame rate and sensitivity. We propose the concept of Archimedean spiral push-broom filtering (ASPBF), where the trajectory of an Archimedean spiral over the FPA is approximated as a straight line. The ASPBF precisely matches the electronic pulse scanning of the uncooled infrared FPA row by row to improve the frame rate. We applied differential imaging to promote gas detection sensitivity. Prototype can detect 11 ml/min of ethylene gas at ΔT = 3 °C with frame rate of 8 fps.

Divide-and-Conquer Dual-Architecture Convolutional Neural Network for Classification of Hyperspectral Images

Convolutional neural network (CNN) is well-known for its powerful capability on image classification. In hyperspectral images (HSIs), fixed-size spatial window is generally used as the input of CNN for pixel-wise classification. However, single fixed-size spatial architecture hinders the excellent performance of CNN due to the neglect of various land-cover distributions in HSIs. Moreover, insufficient samples in HSIs may cause the overfitting problem. To address these problems, a novel divide-and-conquer dual-architecture CNN (DDCNN) method is proposed for HSI classification. In DDCNN, a novel regional division strategy based on local and non-local decisions is devised to distinguish homogeneous and heterogeneous regions. Then, for homogeneous regions, a multi-scale CNN architecture with larger spatial window inputs is constructed to learn joint spectral-spatial features. For heterogeneous regions, a fine-grained CNN architecture with smaller spatial window inputs is constructed to learn hierarchical spectral features. Moreover, to alleviate the problem of insufficient training samples, unlabeled samples with high confidences are pre-labeled under adaptively spatial constraint. Experimental results on HSIs demonstrate that the proposed method provides encouraging classification performance, especially region uniformity and edge preservation with limited training samples.

Deep Learning-Based Framework for In Vivo Identification of Glioblastoma Tumor using Hyperspectral Images of Human Brain

The main goal of brain cancer surgery is to perform an accurate resection of the tumor, preserving as much normal brain tissue as possible for the patient. The development of a non-contact and label-free method to provide reliable support for tumor resection in real-time during neurosurgical procedures is a current clinical need. Hyperspectral imaging is a non-contact, non-ionizing, and label-free imaging modality that can assist surgeons during this challenging task without using any contrast agent. In this work, we present a deep learning-based framework for processing hyperspectral images of in vivo human brain tissue. The proposed framework was evaluated by our human image database, which includes 26 in vivo hyperspectral cubes from 16 different patients, among which 258,810 pixels were labeled. The proposed framework is able to generate a thematic map where the parenchymal area of the brain is delineated and the location of the tumor is identified, providing guidance to the operating surgeon for a successful and precise tumor resection. The deep learning pipeline achieves an overall accuracy of 80% for multiclass classification, improving the results obtained with traditional support vector machine (SVM)-based approaches. In addition, an aid visualization system is presented, where the final thematic map can be adjusted by the operating surgeon to find the optimal classification threshold for the current situation during the surgical procedure.

Detecting Happiness Using Hyperspectral Imaging Technology

Hyperspectral imaging (HSI) technology can be used to detect human emotions based on the power of material discrimination from their faces. In this paper, HSI is used to remotely sense and distinguish blood chromophores in facial tissues and acquire an evaluation indicator (tissue oxygen saturation, StO₂) using an optical absorption model. This study explored facial analysis while people were showing spontaneous expressions of happiness during social interaction. Happiness, as a psychological emotion, has been shown to be strongly linked to other activities such as physiological reaction and facial expression. Moreover, facial expression as a communicative motor behavior likely arises from musculoskeletal anatomy, neuromuscular activity, and individual personality. This paper quantified the neuromotor movements of tissues surrounding some regions of interest (ROIs) on smiling happily. Next, we selected six regions—the forehead, eye, nose, cheek, mouth, and chin—according to a facial action coding system (FACS). Nineteen segments were subsequently partitioned from the above ROIs. The affective data (StO₂) of 23 young adults were acquired by HSI while the participants expressed emotions (calm or happy), and these were used to compare the significant differences in the variations of StO₂ between the different ROIs through repeated measures analysis of variance. Results demonstrate that happiness causes different distributions in the variations of StO₂ for the above ROIs; these are explained in depth in the article. This study establishes that facial tissue oxygen saturation is a valid and reliable physiological indicator of happiness and merits further research.

Design of a Tunable Snapshot Multispectral Imaging System through Ray Tracing Simulation

Research on snapshot multispectral imaging has been popular in the remote sensing community due to the high demands of video-rate remote sensing system for various applications. Existing snapshot multispectral imaging techniques are mainly of a fixed wavelength type, which limits their practical usefulness. This paper describes a tunable multispectral snapshot system by using a dual prism assembly as the dispersion element of the coded aperture snapshot spectral imagers (CASSI). Spectral tuning is achieved by adjusting the air gap displacement of the dual prism assembly. Typical spectral shifts of about 1 nm at 400 nm and 12 nm at 700 nm wavelength have been achieved in the present design when the air-gap of the dual prism is changed from 4.24 mm to 5.04 mm. The paper outlines the optical designs, the performance, and the pros and cons of the dual-prism CASSI (DP-CASSI) system. The performance of the system is illustrated by TracePro^TM ray tracing, to allow researchers in the field to repeat or to validate the results presented in this paper.

Identifying molecular contributors to autofluorescence of neoplastic and normal colon sections using excitation-scanning hyperspectral imaging

Autofluorescence, the endogenous fluorescence present in cells and tissues, has historically been considered a nuisance in biomedical imaging. Many endogenous fluorophores, specifically, collagen, elastin, nicotinamide adenine dinucleotide, and flavin adenine dinucleotide (FAD), are found throughout the human body. In fluorescence imaging scenarios, these signals can be prohibitive as they can outcompete signals introduced for diagnostic purposes. However, autofluorescence also contains information that has diagnostic value. Recent advances in hyperspectral imaging have allowed the acquisition of significantly more data in a shorter time period by scanning the excitation spectra of fluorophores. The reduced acquisition time and increased signal-to-noise ratio allow for separation of significantly more fluorophores than previously possible. We propose to utilize excitation-scanning hyperspectral imaging of autofluorescence to differentiate neoplastic lesions from surrounding non-neoplastic "normal" tissue. The spectra of isolated autofluorescent molecules are obtained using a custom inverted microscope (TE-2000, Nikon Instruments) with an Xe arc lamp and thin-film tunable filter array (VersaChrome, Semrock, Inc.). Scans utilize excitation wavelengths from 360 to 550 nm in 5-nm increments. The resultant molecule-specific spectra are used to analyze hyperspectral image stacks from normal and neoplastic colorectal tissues. Due to a limited number of samples, neoplastic tissues examined here are a pool of both colorectal adenocarcinoma and adenomatous polyps. The hyperspectral images are analyzed with ENVI software and custom MATLAB scripts, including linear spectral unmixing. Initial results indicate the ability to separate signals of endogenous fluorophores and measure the relative concentrations of fluorophores among healthy and diseased states, in this case, normal colon versus neoplastic colon. These results suggest pathology-specific changes to endogenous fluorophores can be detected using excitation-scanning hyperspectral imaging. Future work will focus on expanding the library of pure molecules, exploring histogram distance metrics as a means for identifying deviations in spectral signatures, and examining more defined disease states.

Demystifying autofluorescence with excitation-scanning hyperspectral imaging

Autofluorescence has historically been considered a nuisance in medical imaging. Many endogenous fluorophores, specifically, collagen, elastin, NADH, and FAD, are found throughout the human body. Diagnostically, these signals can be prohibitive since they can outcompete signals introduced for diagnostic purposes. Recent advances in hyperspectral imaging have allowed the acquisition of significantly more data in a shorter time period by scanning the excitation spectra of fluorophores. The reduced acquisition time and increased signal-to-noise ratio allow for separation of significantly more fluorophores than previously possible. Here, we propose to utilize excitation-scanning of autofluorescence to examine tissues and diagnose pathologies. Spectra of autofluorescent molecules were obtained using a custom inverted microscope (TE-2000, Nikon Instruments) with a Xe arc lamp and thin film tunable filter array (VersaChrome, Semrock, Inc.) Scans utilized excitation wavelengths from 360 nm to 550 nm in 5 nm increments. The resultant spectra were used to examine hyperspectral image stacks from various collaborative studies, including an atherosclerotic rat model and a colon cancer study. Hyperspectral images were analyzed with ENVI and custom Matlab scripts including linear spectral unmixing (LSU) and principal component analysis (PCA). Initial results suggest the ability to separate the signals of endogenous fluorophores and measure the relative concentrations of fluorophores among healthy and diseased states of similar tissues. These results suggest pathology-specific changes to endogenous fluorophores can be detected using excitation-scanning hyperspectral imaging. Future work will expand the library of pure molecules and will examine more defined disease states.

Computational reconfigurable imaging spectrometer

We demonstrate a novel hyperspectral imaging spectrometer based on computational imaging that enables sensitive measurements from smaller, noisier, and less-expensive components (e.g. uncooled microbolometers), making it useful for applications such as small space and air platforms with strict size, weight, and power requirements. The computational reconfigurable imaging spectrometer (CRISP) system exploits platform motion and a spectrally coded focal-plane mask to temporally modulate the optical spectrum, enabling simultaneous measurement of multiple spectral bins. Demodulation of this coded pattern returns an optical spectrum in each pixel.

Local Spectral Component Decomposition for Multi-Channel Image Denoising

We propose a method for local spectral component decomposition based on the line feature of local distribution. Our aim is to reduce noise on multi-channel images by exploiting the linear correlation in the spectral domain of a local region. We first calculate a linear feature over the spectral components of an M -channel image, which we call the spectral line, and then, using the line, we decompose the image into three components: a single M -channel image and two gray-scale images. By virtue of the decomposition, the noise is concentrated on the two images, and thus our algorithm needs to denoise only the two gray-scale images, regardless of the number of the channels. As a result, image deterioration due to the imbalance of the spectral component correlation can be avoided. The experiment shows that our method improves image quality with less deterioration while preserving vivid contrast. Our method is especially effective for hyperspectral images. The experimental results demonstrate that our proposed method can compete with the other state-of-the-art denoising methods.

Along-track scanning using a liquid crystal compressive hyperspectral imager

In various applications, such as remote sensing and quality inspection, hyperspectral (HS) imaging is performed by spatially scanning an object. In this work, we present a new compressive hyperspectral imaging method that performs along-track scanning. The method relies on the compressive sensing miniature ultra-spectral imaging (CS-MUSI) system, which uses a single liquid crystal (LC) cell for spectral encoding and provides a more efficient way of HS data acquisition, compared to classical spatial scanning based systems. The experimental results show that a compression ratio of about 1:10 can be reached. Owing to the inherent compression, the captured data is preprepared for efficient storage and transmission.

Automated Spectroscopic Tissue Classification in Colorectal Surgery

BACKGROUND

In colorectal surgery, detecting ureters and mesenteric arteries is of utmost importance to prevent iatrogenic injury and to facilitate intraoperative decision making. A tool enabling ureter- and artery-specific image enhancement within (and possibly through) surrounding adipose tissue would facilitate this need, especially during laparoscopy. To evaluate the potential of hyperspectral imaging in colorectal surgery, we explored spectral tissue signatures using single-spot diffuse reflectance spectroscopy (DRS). As hyperspectral cameras with silicon (Si) and indium gallium arsenide (InGaAs) sensor chips are becoming available, we investigated spectral distinctive features for both sensor ranges.

METHODS

In vivo wide-band (wavelength range 350-1830 nm) DRS was performed during open colorectal surgery. From the recorded spectra, 36 features were extracted at predefined wavelengths: 18 gradients and 18 amplitude differences. For classification of respectively ureter and artery in relation to surrounding adipose tissue, the best distinctive feature was selected using binary logistic regression for Si- and InGaAs-sensor spectral ranges separately. Classification performance was evaluated by leave-one-out cross-validation.

RESULTS

In 10 consecutive patients, 253 spectra were recorded on 53 tissue sites (including colon, adipose tissue, muscle, artery, vein, ureter). Classification of ureter versus adipose tissue revealed accuracy of 100% for both Si range and InGaAs range. Classification of artery versus surrounding adipose tissue revealed accuracies of 95% (Si) and 89% (InGaAs).

CONCLUSIONS

Intraoperative DRS showed that Si and InGaAs sensors are equally suited for automated classification of ureter versus surrounding adipose tissue. Si sensors seem better suited for classifying artery versus mesenteric adipose tissue. Progress toward hyperspectral imaging within this field is promising.

Differentiation between nerve and adipose tissue using wide-band (350-1,830 nm) in vivo diffuse reflectance spectroscopy

BACKGROUND

Intraoperative nerve localization is of great importance in surgery. In certain procedures, where nerves show visual resemblance to surrounding adipose tissue, this can be particularly challenging for the human eye. An example of such a delicate procedure is thyroid and parathyroid surgery, where iatrogenic injury of the recurrent laryngeal nerve can result in transient or permanent vocal problems (0.5-2.0% reported incidence). A camera system, enabling nerve-specific image enhancement, would be useful in preventing such complications. This might be realized with hyperspectral camera technology using silicon (Si) or indium gallium arsenide (InGaAs) sensor chips.

METHODS

As a first step towards such a camera, we evaluated the performance of diffuse reflectance spectroscopy by analysing spectra collected during 18 thyroid and parathyroid resections. We assessed the contrast information present in two different spectral ranges, for respectively Si and InGaAs sensors. Two hundred fifty three in vivo, wide-band diffuse reflectance spectra (350-1,830 nm range, 1 nm resolution) were acquired on 52 tissue spots, including nerve (n = 22), muscle (n = 12), and adipose tissue (n = 18). We extracted 36 features from these spectroscopic data: 18 gradients and 18 amplitude differences at predefined points in the tissue spectra. Best distinctive feature combinations were established using binary logistic regression. Classification performance was evaluated in a cross-validation (CV) approach by leave-one-out (LOO). To generalize nerve recognition applicability, we performed a train-test (TT) validation using the thyroid and parathyroid surgery data for training purposes and carpal tunnel release surgery data (10 nerve spots and 5 adipose spots) for classification purposes.

RESULTS

For combinations of two distinctive spectral features, LOO revealed an accuracy of respectively 78% for Si-sensors and 95% for InGaAs-sensors. TT revealed accuracies of respectively 67% and 100%.

CONCLUSIONS

Using diffuse reflectance spectroscopy we have identified that InGaAs sensors are better suited for automated discrimination between nerves and surrounding adipose tissue than Si sensors.