Spectral reflectance recovery using optimal illuminations

Unsupervised spectral reconstruction from RGB images under two lighting conditions

Unsupervised spectral reconstruction (SR) aims to recover the hyperspectral image (HSI) from corresponding RGB images without annotations. Existing SR methods achieve it from a single RGB image, hindered by the significant spectral distortion. Although several deep learning-based methods increase the SR accuracy by adding RGB images, their networks are always designed for other image recovery tasks, leaving huge room for improvement. To overcome this problem, we propose a novel, to our knowledge, approach that reconstructs the HSI from a pair of RGB images captured under two illuminations, significantly improving reconstruction accuracy. Specifically, an SR iterative model based on two illuminations is constructed at first. By unfolding the proximal gradient algorithm solving this SR model, an interpretable unsupervised deep network is proposed. All the modules in the proposed network have precise physical meanings, which enable our network to have superior performance and good generalization capability. Experimental results on two public datasets and our real-world images show the proposed method significantly improves both visually and quantitatively as compared with state-of-the-art methods.

A Spectral Encoding Simulator for Broadband Active Illumination and Reconstruction-Based Spectral Measurement

Spectral reflectance or transmittance measurements provide intrinsic information on the material of an object and are widely used in remote sensing, agriculture, diagnostic medicine, etc. Most reconstruction-based spectral reflectance or transmittance measurement methods based on broadband active illumination use narrow-band LEDs or lamps combined with specific filters as spectral encoding light sources. These light sources cannot achieve the designed spectral encoding with a high resolution and accuracy due to their low degree of freedom for adjustment, leading to inaccurate spectral measurements. To address this issue, we designed a spectral encoding simulator for active illumination. The simulator is composed of a prismatic spectral imaging system and a digital micromirror device. The spectral wavelengths and intensity are adjusted by switching the micromirrors. We used it to simulate spectral encodings according to the spectral distribution on micromirrors and solved the DMD patterns corresponding to the spectral encodings with a convex optimization algorithm. To verify the applicability of the simulator for spectral measurements based on active illumination, we used it to numerically simulate existing spectral encodings. We also numerically simulated a high-resolution Gaussian random measurement encoding for compressed sensing and measured the spectral reflectance of one vegetation type and two minerals through numerical simulations. We reconstructed the spectral transmittance of a calibrated filter through an experiment. The results show that the simulator can measure the spectral reflectance or transmittance with a high resolution and accuracy.

Highly multicolored light-emitting arrays for compressive spectroscopy

Miniaturized, multicolored light-emitting device arrays are promising for applications in sensing, imaging, computing, and more, but the range of emission colors achievable by a conventional light-emitting diode is limited by material or device constraints. In this work, we demonstrate a highly multicolored light-emitting array with 49 different, individually addressable colors on a single chip. The array consists of pulsed-driven metal-oxide-semiconductor capacitors, which generate electroluminescence from microdispensed materials spanning a diverse range of colors and spectral shapes, enabling facile generation of arbitrary light spectra across a broad wavelength range (400 to 1400 nm). When combined with compressive reconstruction algorithms, these arrays can be used to perform spectroscopic measurements in a compact manner without diffractive optics. As an example, we demonstrate microscale spectral imaging of samples using a multiplexed electroluminescent array in conjunction with a monochrome camera.

Sensor simulation using a spectrum tunable LED system

This study developed a method to simulate the sensor responses and verify the effectiveness on spectral reconstruction by a spectrum tunable LED system. Studies have shown that the spectral reconstruction accuracy could be improved by including multiple channels in a digital camera. However, the real sensors with designed spectral sensitivities were hard to manufacture and validate. Therefore, the presence of a quick and reliable validation mechanism was preferred when performing evaluation. In this study, two novel approaches, i.e., channel-first and illumination-first simulations, were proposed to replicate the designed sensors with the use of a monochrome camera and a spectrum-tunable LED illumination system. In the channel-first method, the spectral sensitivities of three extra sensor channels were optimized theoretically for an RGB camera and then simulated by matching the corresponding illuminants in the LED system. The illumination-first method optimized the spectral power distribution (SPD) of the lights using the LED system, and the extra channels could be determined accordingly. The results of practical experiments showed that the proposed methods were effective to simulate the responses of the extra sensor channels.

pHSCNN: CNN-based hyperspectral recovery from a pair of RGB images

To increase the fidelity of hyperspectral recovery from RGB images, we propose a pairwise-image-based hyperspectral convolutional neural network (pHSCNN) to recover hyperspectral images from a pair of RGB images, obtained by the same color sensor with and without an optical filter in front of the imaging lens. The proposed method avoids the pitfall of requiring multiple color sensors to obtain different RGB images and achieves higher accuracy than recovery from single RGB image. Besides, pHSCNN can also optimize the optical filter to further improve the performance. To experiment on real data, we built a dual-camera hyperspectral imaging system and created a real-captured hyperspectral-RGB dataset. Experimental results demonstrate the superiority of pHSCNN with the highest accuracy of the recovered hyperspectral signature perceptually and numerically.

Snapshot spectroscopic microscopy with double spherical slicer mirrors

Snapshot hyperspectral microscopic imaging can obtain the morphological characteristics and chemical specificity of samples simultaneously and instantaneously. We demonstrate a double-slicer spectroscopic microscopy (DSSM) that uses two spherical slicer mirrors to magnify the target image and slice it. These slits are lined up and dispersed, then mapped onto an area-array detector. An anamorphosis unit optimizes the capacity of the limited pixels. With a single shot and image recombination, a data cube can be constructed for sample analysis, and a model of DSSM is simulated. The system covers the spectral range from 500 nm to 642.5 nm with 20 spectral channels. The spatial resolution is 417 nm, and the spectral resolution is 7.5 nm.