Compressive spectral imaging via deformable mirror and colored-mosaic detector

Computational spectral imaging: a contemporary overview

Spectral imaging collects and processes information along spatial and spectral coordinates quantified in discrete voxels, which can be treated as a 3D spectral data cube. The spectral images (SIs) allow the identification of objects, crops, and materials in the scene through their spectral behavior. Since most spectral optical systems can only employ 1D or maximum 2D sensors, it is challenging to directly acquire 3D information from available commercial sensors. As an alternative, computational spectral imaging (CSI) has emerged as a sensing tool where 3D data can be obtained using 2D encoded projections. Then, a computational recovery process must be employed to retrieve the SI. CSI enables the development of snapshot optical systems that reduce acquisition time and provide low computational storage costs compared with conventional scanning systems. Recent advances in deep learning (DL) have allowed the design of data-driven CSI to improve the SI reconstruction or, even more, perform high-level tasks such as classification, unmixing, or anomaly detection directly from 2D encoded projections. This work summarizes the advances in CSI, starting with SI and its relevance and continuing with the most relevant compressive spectral optical systems. Then, CSI with DL will be introduced, as well as the recent advances in combining the physical optical design with computational DL algorithms to solve high-level tasks.

Snapshot compressive spectral depth imaging from coded aberrations

Compressive spectral depth imaging (CSDI) is an emerging technology aiming to reconstruct spectral and depth information of a scene from a limited set of two-dimensional projections. CSDI architectures have conventionally relied on stereo setups that require the acquisition of multiple shots attained via dynamically programmable spatial light modulators (SLM). This work proposes a snapshot CSDI architecture that exploits both phase and amplitude modulation and uses a single image sensor. Specifically, we modulate the spectral-depth information in two steps. Firstly, a deformable mirror (DM) is used as a phase modulator to induce a focal length sweeping while simultaneously introducing a controlled aberration. The phase-modulated wavefront is then spatially modulated and spectrally dispersed by a digital micromirror device (DMD) and a prism, respectively. Therefore, each depth plane is modulated by a variable phase and binary code. Complimentary, we also propose a computational methodology to recover the underlying spectral depth hypercube efficiently. Through simulations and our experimental proof-of-concept implementation, we demonstrate that the proposed computational imaging system is a viable approach to capture spectral-depth hypercubes from a single image.

Snapshot multispectral imaging using a pixel-wise polarization color image sensor

This study proposes a new imaging technique for snapshot multispectral imaging in which a multispectral image was captured using an imaging lens that combines a set of multiple spectral filters and polarization filters, as well as a pixel-wise color polarization image sensor. The author produced a prototype nine-band multispectral camera system that covered from visible to near-infrared regions and was very compact. The camera's spectral performance was evaluated using experiments; moreover, the camera was used to detect the freshness of food and the activity of wild plants and was mounted on a vehicle to obtain a multispectral video while driving.

Simultaneous coded aperture and dictionary optimization in compressive spectral imaging via coherence minimization

Coded aperture snapshot spectral imaging (CASSI) reconstructs a hyperspectral image from several two-dimensional (2D) projections via compressive sensing. The reconstruction quality and the sampling efficiency of CASSI can be effectively improved by decreasing the coherence of the underlying sensing matrix. Efforts have been made to minimize the coherence with individual optimization on coded aperture or sparse basis. In this paper, a simultaneous optimization on the system projection and the over-complete dictionary is introduced to minimize the Frobenius norm coherence. The dual-disperser structure and the RGB image sensor are adopted for the lowest coherence in terms of system configuration. The coded aperture and the dictionary are optimized with genetic algorithm and gradient descent respectively, and simultaneous optimization is conducted iteratively. Low coherence of sensing matrix is acquired after the simultaneous optimization, with both reconstruction quality and sampling efficiency significantly improved. Compared to the non-optimized system and state-of-the-art systems with individually optimized coded aperture or dictionary, the simultaneous optimization promotes the peak signal-to-noise ratio by more than 5dB. The coherence minimization via simultaneous optimization on the system matrix and the sparse representation basis may open opportunities for further development of other compressive-sensing-based computational imaging systems.

Dual-camera snapshot spectral imaging with a pupil-domain optical diffuser and compressed sensing algorithms

We propose a snapshot spectral imaging method for the visible spectral range using two digital cameras placed side-by-side: a regular red-green-blue (RGB) camera and a monochromatic camera equipped with a dispersive diffractive diffuser placed at the pupil of the imaging lens. While spectral imaging was shown to be feasible using a single monochromatic camera with a pupil diffuser [Appl. Opt.55, 432 (2016)APOPAI0003-693510.1364/AO.55.000432], adding an RGB camera provides more spatial and spectral information for stable reconstruction of the spectral cube of a scene. Results of optical experiments confirm that the combined data from the two cameras relax the complexity of the underdetermined reconstruction problem and improve the reconstructed image quality obtained using compressed sensing-based algorithms.