Complete plenoptic imaging using a single detector

Coded aperture snapshot hyperspectral light field tomography

Multidimensional imaging has emerged as a powerful technology capable of simultaneously acquiring spatial, spectral, and depth information about a scene. However, existing approaches often rely on mechanical scanning or multi-modal sensing configurations, leading to prolonged acquisition times and increased system complexity. Coded aperture snapshot spectral imaging (CASSI) has introduced compressed sensing to recover three-dimensional (3D) spatial-spectral datacubes from single snapshot two-dimensional (2D) measurements. Despite its advantages, the reconstruction problem remains severely underdetermined due to the high compression ratio, resulting in limited spatial and spectral reconstruction quality. To overcome this challenge, we developed a novel two-stage cascaded compressed sensing scheme called coded aperture snapshot hyperspectral light field tomography (CASH-LIFT). By appropriately distributing the computation load to each stage, this method utilizes the compressibility of natural scenes in multiple domains, reducing the ill-posed nature of datacube recovery and achieving enhanced spatial resolution, suppressed aliasing artifacts, and improved spectral fidelity. Additionally, leveraging the snapshot 3D imaging capability of LIFT, our approach efficiently records a five-dimensional (5D) plenoptic function in a single snapshot.

Micro 4D Imaging Sensor Using Snapshot Narrowband Imaging Method

The spectral and depth (SAD) imaging method plays an important role in the field of computer vision. However, accurate depth estimation and spectral image capture from a single image without increasing the volume of the imaging sensor is still an unresolved problem. Our research finds that a snapshot narrow band imaging (SNBI) method can discern wavelength-dependent spectral aberration and simultaneously capture spectral-aberration defocused images for quantitative depth estimation. First, a micro 4D imaging (M4DI) sensor is proposed by integrating a mono-chromatic imaging sensor with a miniaturized narrow-band microarrayed spectral filter mosaic. The appearance and volume of the M4DI sensor are the same as the integrated mono-chromatic imaging sensor. A simple remapping algorithm was developed to separate the raw image into four narrow spectral band images. Then, a depth estimation algorithm is developed to generate 3D data with a dense depth map at every exposure of the M4DI sensor. Compared with existing SAD imaging method, the M4DI sensor has the advantages of simple implementation, low computational burden, and low cost. A proof-of-principle M4DI sensor was applied to sense the depth of objects and to track a tiny targets trajectory. The relative error in the three-dimensional positioning is less than 7% for objects within 1.1 to 2.8 m.

Super diffraction limit spectral imaging detection and material type identification of distant space objects

The image information of distant objects shows a diffuse speckle pattern due to diffraction limit, non-uniform scattering, etc., which is difficult to achieve object discrimination. In this study, we have developed a staring spectral video imaging system mounted on a ground-based telescope observation platform to detect the high orbit space objects and gain their spectral images for six groups of GEO targets. The speckle remains basically the same characteristic as the projection structure of the object due to "the balloon inflation phenomenon of near parallel light during long-distance atmospheric transmission" under the premise of considering the bi-directional reflection distribution function (BRDF), Rayleigh scattering theory, and the memory effect. Based on this phenomenon, a mathematical model of remote target scattering spectrum imaging is established where the speckle can be treated as both a global speckle and speckle combination of texture blocks caused by various components of the target. The radial basis function (RBF) neural network is separately used to invert the global speckle and the speckle combination of the texture blocks on account of the typical target material database. The results show that the target materials are of relatively fewer kinds in the global inversion with only including gallium arsenide panel (GaAs) and carbon fiber (CF), for which the highest goodness of curve fitting is only 77.97. An improved algorithm makes their goodness of fit reach 90.29 and 93.33, respectively, in view of one conjecture that the target surface contains unknown materials. The spectral inversion result of the texture blocks shows that the types of materials in each target texture block increase significantly, and that the area ratio of different materials inverted in the block is different from each other. It is further confirmed that the speckle image contains the overall projection structure of distant target and the spectral image projection of each component is relatively fixed, which is the result of the comprehensive action of various mechanisms of ultra-long-haul atmospheric transmission and optical system focusing imaging after BRDF spectral scattering. The spectral image fine inversion is expected to restore the clear structure of the target. This discovery provides important support for the remote imaging and identification of distant and ultra-diffractive targets.

Snapshot hyperspectral light field tomography

We present snapshot hyperspectral light field tomography (Hyper-LIFT), a highly efficient method in recording a 5D (x, y, spatial coordinates; θ, φ, angular coordinates; λ, wavelength) plenoptic function. Using a Dove prism array and a cylindrical lens array, we simultaneously acquire multi-angled 1D en face projections of the object like those in standard sparse-view computed tomography. We further disperse those projections and measure the spectra in parallel using a 2D image sensor. Within a single snapshot, the resultant system can capture a 5D data cube with 270 × 270 × 4 × 4 × 360 voxels. We demonstrated the performance of Hyper-LIFT in imaging spectral volumetric scenes.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Quasi-spectral characterization of intracellular regions in bright-field light microscopy images

Investigation of cell structure is hardly imaginable without bright-field microscopy. Numerous modifications such as depth-wise scanning or videoenhancement make this method being state-of-the-art. This raises a question what maximal information can be extracted from ordinary (but well acquired) bright-field images in a model-free way. Here we introduce a method of a physically correct extraction of features for each pixel when these features resemble a transparency spectrum. The method is compatible with existent ordinary bright-field microscopes and requires mathematically sophisticated data processing. Unsupervised clustering of the spectra yields reasonable semantic segmentation of unstained living cells without any a priori information about their structures. Despite the lack of reference data (to prove strictly that the proposed feature vectors coincide with transparency), we believe that this method is the right approach to an intracellular (semi)quantitative and qualitative chemical analysis.

Plenoptic Face Presentation Attack Detection

The vulnerability of current face recognition systems to presentation attacks significantly limits their application in biometrics. Herein, we present a passive presentation attack detection method based on a complete plenoptic imaging system which can derive the complete plenoptic function of light rays using a single detector. Moreover, we constructed a multi-dimensional face database with 50 subjects and seven different types of presentation attacks. We experimentally demonstrated that our approach outperforms the state-of-the-art methods on all types of presentation attacks.

Snapshot hyperspectral light field imaging using image mapping spectrometry

In this Letter, we present a snapshot hyperspectral light field imaging system using a single camera. By integrating an unfocused light field camera with a snapshot hyperspectral imager, the image mapping spectrometer, we captured a five-dimensional (5D) ($x,y,u,v,\lambda $x,y,u,v,λ) ($x,y,$x,y, spatial coordinates; $u,v,$u,v, emittance angles; $\lambda ,$λ, wavelength) datacube in a single camera exposure. The corresponding volumetric image ($x,y,z$x,y,z) at each wavelength is then computed through a scale-depth space transform. We demonstrated the snapshot advantage of our system by imaging the spectral-volumetric scenes in real time.