Compressive 4D spectro-volumetric imaging

Micro 4D Imaging Sensor Using Snapshot Narrowband Imaging Method

The spectral and depth (SAD) imaging method plays an important role in the field of computer vision. However, accurate depth estimation and spectral image capture from a single image without increasing the volume of the imaging sensor is still an unresolved problem. Our research finds that a snapshot narrow band imaging (SNBI) method can discern wavelength-dependent spectral aberration and simultaneously capture spectral-aberration defocused images for quantitative depth estimation. First, a micro 4D imaging (M4DI) sensor is proposed by integrating a mono-chromatic imaging sensor with a miniaturized narrow-band microarrayed spectral filter mosaic. The appearance and volume of the M4DI sensor are the same as the integrated mono-chromatic imaging sensor. A simple remapping algorithm was developed to separate the raw image into four narrow spectral band images. Then, a depth estimation algorithm is developed to generate 3D data with a dense depth map at every exposure of the M4DI sensor. Compared with existing SAD imaging method, the M4DI sensor has the advantages of simple implementation, low computational burden, and low cost. A proof-of-principle M4DI sensor was applied to sense the depth of objects and to track a tiny targets trajectory. The relative error in the three-dimensional positioning is less than 7% for objects within 1.1 to 2.8 m.

Mid-infrared single-pixel imaging at the single-photon level

Single-pixel cameras have recently emerged as promising alternatives to multi-pixel sensors due to reduced costs and superior durability, which are particularly attractive for mid-infrared (MIR) imaging pertinent to applications including industry inspection and biomedical diagnosis. To date, MIR single-pixel photon-sparse imaging has yet been realized, which urgently calls for high-sensitivity optical detectors and high-fidelity spatial modulators. Here, we demonstrate a MIR single-photon computational imaging with a single-element silicon detector. The underlying methodology relies on nonlinear structured detection, where encoded time-varying pump patterns are optically imprinted onto a MIR object image through sum-frequency generation. Simultaneously, the MIR radiation is spectrally translated into the visible region, thus permitting infrared single-photon upconversion detection. Then, the use of advanced algorithms of compressed sensing and deep learning allows us to reconstruct MIR images under sub-Nyquist sampling and photon-starving illumination. The presented paradigm of single-pixel upconversion imaging is featured with single-pixel simplicity, single-photon sensitivity, and room-temperature operation, which would establish a new path for sensitive imaging at longer infrared wavelengths or terahertz frequencies, where high-sensitivity photon counters and high-fidelity spatial modulators are typically hard to access.

Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution

Simultaneous and efficient ultrafast recording of multiple photon tags contributes to high-dimensional optical imaging and characterization in numerous fields. Existing high-dimensional optical imaging techniques that record space and polarization cannot detect the photon's time of arrival owing to the limited speeds of the state-of-the-art electronic sensors. Here, we overcome this long-standing limitation by implementing stereo-polarimetric compressed ultrafast photography (SP-CUP) to record light-speed high-dimensional events in a single exposure. Synergizing compressed sensing and streak imaging with stereoscopy and polarimetry, SP-CUP enables video-recording of five photon tags (x, y, z: space; t: time of arrival; and ψ: angle of linear polarization) at 100 billion frames per second with a picosecond temporal resolution. We applied SP-CUP to the spatiotemporal characterization of linear polarization dynamics in early-stage plasma emission from laser-induced breakdown. This system also allowed three-dimensional ultrafast imaging of the linear polarization properties of a single ultrashort laser pulse propagating in a scattering medium.

Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures

Compressive multispectral imaging systems comprise a new generation of spectral imagers that capture coded projections of a scene where spectral data cubes are reconstructed computationally. Separately, time-of-flight (ToF) cameras obtain 2D range images where each pixel records the distance from the camera sensor to the target surface. The demand for these imaging modalities is rapidly increasing, and thus, there is strong interest in developing new image sensors that can simultaneously acquire multispectral-color-and-depth imagery (MS+D) using a single aperture. Work in this path has been mainly developed via RGB+D imaging. However, in RGB+D, the multispectral image is limited to three spectral channels, and the imaging system often relies on two image sensors. We recently proposed a compressive MS+D imaging device that used a digital-micromirror-device, requiring a bulky double imaging-and-relay path. To overcome the bulkiness and other difficulties of our previous imaging system, this work presents a more-compact MS+D imaging device with snapshot capabilities. It provides better spectral sensing, relying on a static color-coded-aperture (CCA) and a ToF sensor. To guarantee good quality in the recovery, we develop an optimization method for CCA based-on blue-noise-multitoning, solved via the direct-binary-search algorithm. A testbed-setup is reported along with simulated and real experiments that demonstrate the MS+D capabilities of the proposed system over static and dynamic scenes.

Compressive Sensing Hyperspectral Imaging by Spectral Multiplexing with Liquid Crystal

Hyperspectral (HS) imaging involves the sensing of a scene’s spectral properties, which are often redundant in nature. The redundancy of the information motivates our quest to implement Compressive Sensing (CS) theory for HS imaging. This article provides a review of the Compressive Sensing Miniature Ultra-Spectral Imaging (CS-MUSI) camera, its evolution, and its different applications. The CS-MUSI camera was designed within the CS framework and uses a liquid crystal (LC) phase retarder in order to modulate the spectral domain. The outstanding advantage of the CS-MUSI camera is that the entire HS image is captured from an order of magnitude fewer measurements of the sensor array, compared to conventional HS imaging methods.

Complete plenoptic imaging using a single detector

Multi-dimensional imaging is a powerful technique for many applications, such as biological analysis, remote sensing, and object recognition. Most existing multi-dimensional imaging systems rely on scanning or camera array, which make the system bulky and unstable. To some extent, these problems can be mitigated by employing compressed sensing algorithms. However, they are computationally expensive and highly rely on the ill-posed assumption that the information is sparse in a given domain. Here, we propose a snapshot spectral-volumetric imaging (SSVI) system by introducing the paradigm of light-field imaging into Fourier transform imaging spectroscopy. We demonstrate that SSVI can reconstruct a complete plenoptic function, P(x,y,z,θ,φ,λ,t), of the incoming light rays using a single detector. Compared with other multidimensional imagers, SSVI features prominent advantages in compactness, robustness, and low cost.