DD-Net: spectral imaging from a monochromatic dispersed and diffused snapshot

Snapshot spectral imaging: from spatial-spectral mapping to metasurface-based imaging

Snapshot spectral imaging technology enables the capture of complete spectral information of objects in an extremely short period of time, offering wide-ranging applications in fields requiring dynamic observations such as environmental monitoring, medical diagnostics, and industrial inspection. In the past decades, snapshot spectral imaging has made remarkable breakthroughs with the emergence of new computational theories and optical components. From the early days of using various spatial-spectral data mapping methods, they have evolved to later attempts to encode various dimensions of light, such as amplitude, phase, and wavelength, and then computationally reconstruct them. This review focuses on a systematic presentation of the system architecture and mathematical modeling of these snapshot spectral imaging techniques. In addition, the introduction of metasurfaces expands the modulation of spatial-spectral data and brings advantages such as system size reduction, which has become a research hotspot in recent years and is regarded as the key to the next-generation snapshot spectral imaging techniques. This paper provides a systematic overview of the applications of metasurfaces in snapshot spectral imaging and provides an outlook on future directions and research priorities.

PNMC: Four-dimensional conebeam CT reconstruction combining prior network and motion compensation

Four-dimensional conebeam computed tomography (4D CBCT) is an efficient technique to overcome motion artifacts caused by organ motion during breathing. 4D CBCT reconstruction in a single scan usually divides projections into different groups of sparsely sampled data based on the respiratory phases. The reconstructed images within each group present poor image quality due to the limited number of projections. To improve the image quality of 4D CBCT in a single scan, we propose a novel reconstruction scheme that combines prior knowledge with motion compensation. We apply the reconstructed images of the full projections within a single routine as prior knowledge, providing structural information for the network to enhance the restoration structure. The prior network (PN-Net) is proposed to extract features of prior knowledge and fuse them with the sparsely sampled data using an attention mechanism. The prior knowledge guides the reconstruction process to restore the approximate organ structure and alleviates severe streaking artifacts. The deformation vector field (DVF) extracted using deformable image registration among different phases is then applied in the motion-compensated ordered-subset simultaneous algebraic reconstruction algorithm to generate 4D CBCT images. Proposed method has been evaluated using simulated and clinical datasets and has shown promising results by comparative experiment. Compared with previous methods, our approach exhibits significant improvements across various evaluation metrics.

Dual-Channel Switchable Metasurface Filters for Compact Spectral Imaging with Deep Compressive Reconstruction

Spectral imaging technology, which aims to capture images across multiple spectral channels and create a spectral data cube, has been widely utilized in various fields. However, conventional spectral imaging systems face challenges, such as slow acquisition speed and large size. The rapid development of optical metasurfaces, capable of manipulating light fields versatilely and miniaturizing optical components into ultrathin planar devices, offers a promising solution for compact hyperspectral imaging (HSI). This study proposes a compact snapshot compressive spectral imaging (SCSI) system by leveraging the spectral modulations of metasurfaces with dual-channel switchable metasurface filters and employing a deep-learning-based reconstruction algorithm. To achieve compactness, the proposed system integrates dual-channel switchable metasurface filters using twisted nematic liquid crystals (TNLCs) and anisotropic titanium dioxide (TiO2) nanostructures. These thin metasurface filters are closely attached to the image sensor, resulting in a compact system. The TNLCs possess a broadband linear polarization conversion ability, enabling the rapid switching of the incidence polarization state between x-polarization and y-polarization by applying different voltages. This polarization conversion facilitates the generation of two groups of transmittance spectra for wavelength-encoding, providing richer information for spectral data cube reconstruction compared to that of other snapshot compressive spectral imaging techniques. In addition, instead of employing classic iterative compressive sensing (CS) algorithms, an end-to-end residual neural network (ResNet) is utilized to reconstruct the spectral data cube. This neural network leverages the 2-frame snapshot measurements of orthogonal polarization channels. The proposed hyperspectral imaging technology demonstrates superior reconstruction quality and speed compared to those of the traditional compressive hyperspectral image recovery methods. As a result, it is expected that this technology will have substantial implications in various domains, including but not limited to object detection, face recognition, food safety, biomedical imaging, agriculture surveillance, and so on.

Eliminating Temporal Illumination Variations in Whisk-broom Hyperspectral Imaging

We propose a method for eliminating the temporal illumination variations in whisk-broom (point-scan) hyperspectral imaging. Whisk-broom scanning is useful for acquiring a spatial measurement using a pixel-based hyperspectral sensor. However, when it is applied to outdoor cultural heritages, temporal illumination variations become an issue due to the lengthy measurement time. As a result, the incoming illumination spectra vary across the measured image locations because different locations are measured at different times. To overcome this problem, in addition to the standard raster scan, we propose an additional perpendicular scan that traverses the raster scan. We show that this additional scan allows us to infer the illumination variations over the raster scan. Furthermore, the sparse structure in the illumination spectrum is exploited to robustly eliminate these variations. We quantitatively show that a hyperspectral image captured under sunlight is indeed affected by temporal illumination variations, that a Naïve mitigation method suffers from severe artifacts, and that the proposed method can robustly eliminate the illumination variations. Finally, we demonstrate the usefulness of the proposed method by capturing historic stained-glass windows of a French cathedral.

Spectral imaging with deep learning

The goal of spectral imaging is to capture the spectral signature of a target. Traditional scanning method for spectral imaging suffers from large system volume and low image acquisition speed for large scenes. In contrast, computational spectral imaging methods have resorted to computation power for reduced system volume, but still endure long computation time for iterative spectral reconstructions. Recently, deep learning techniques are introduced into computational spectral imaging, witnessing fast reconstruction speed, great reconstruction quality, and the potential to drastically reduce the system volume. In this article, we review state-of-the-art deep-learning-empowered computational spectral imaging methods. They are further divided into amplitude-coded, phase-coded, and wavelength-coded methods, based on different light properties used for encoding. To boost future researches, we've also organized publicly available spectral datasets.

Convolutional neural network for estimating physical parameters from Newton's rings

By analyzing Newton's rings, often encountered in interferometry, the parameters of spherical surfaces such as the rings' center and the curvature radius can be estimated. First, the classical convolutional neural networks, visual geometry group (VGG) network and U-Net, are applied to parameter estimation of Newton's rings. After these models are trained, the rings' center and curvature radius can be obtained simultaneously. Compared with previous analysis methods of Newton's rings, it is shown that the proposed method has higher precision, better immunity to noise, and lower time consumption. For a Newton's rings pattern of ${{640}} \times {{480}}$ pixels comprising ${-}{{5}}\;{\rm{dB}}$ Gaussian noise or 60% salt-and-pepper noise, the parameters can be estimated by the VGG model in 0.01 s, the error of the rings' center is less than one pixel, and the error of curvature radius is lower than 0.5%.