Feasibility study for compressive multi-dimensional integral imaging

Compound-eye metasurface optics enabling a high-sensitivity, ultra-thin polarization camera

Polarization imaging is key for various applications ranging from biology to machine vision because it can capture valuable optical information about imaged environments, which is usually absent in intensity and spectral content. Conventional polarization cameras rely on a traditional single-eye imaging system with rotating polarizers, cascaded optics, or micropolarizer-patterned image sensors. These cameras, however, have two common issues. The first is low sensitivity resulting from the limited light utilization efficiency of absorptive polarizers or cascaded optics. The other is the difficulty in device miniaturization due to the fact that these devices require at least an optical-path length equivalent to the lens's focal length. Here, we propose a polarization imaging system based on compound-eye metasurface optics and show how it enables the creation of a high-sensitivity, ultra-thin polarization camera. Our imaging system is composed of a typical image sensor and single metasurface layer for forming a vast number of images while sorting the polarization bases. Since this system is based on a filter-free, computational imaging scheme while dramatically reducing the optical-path length required for imaging, it overcomes both efficiency and size limitations of conventional polarization cameras. As a proof of concept, we demonstrated that our system improves the amount of detected light by a factor of ∼2, while reducing device thickness to ∼1/10 that of the most prevalent polarization cameras. Such a sensitive, compact, and passive device could pave the way toward the widespread adoption of polarization imaging in applications in which available light is limited and strict size constraints exist.

Wavelet-based iterative perfect reconstruction in computational integral imaging

We propose a new computational integral imaging (CII) method via the iterative perfect reconstruction technique to improve the visual quality of reconstructed 3D scenes. As is well known, images reconstructed by CII suffer from artifacts and, as a result, degradation of visual quality. As solutions to this problem, the regularization and iterative back-projection (IBP)-based super-resolution (SR) reconstruction algorithms have been shown to be effective for high-visual-quality reconstruction. However, computation of the regularization algorithm is very expensive, and the IBP algorithm is very sensitive to noise in the deblurring process. To address these challenges, we propose an iterative perfect reconstruction algorithm that addresses the issues of low visual quality and noise sensitivity. Experimental results indicate that our proposed method outperforms the conventional SR reconstruction-based CII methods.

Copyright protection for elemental image array by hypercomplex Fourier transform and an adaptive texturized holographic algorithm

In practical applications of three-dimensional integral imaging, the captured elemental image array (EIA) needs to be stored and delivered through the Internet. Therefore, there is an urgent need for protecting the copyright of EIA against piracy and malicious manipulation. In our work, we propose a copyright protection algorithm for EIA by combining the use of the modified hypercomplex Fourier transform (HFT) and the adaptive texturized holographic algorithm. The modified HFT can accurately extract the features from each elemental image. According to these features, we embed watermark into the visually less noticeable regions of the EIA to increase the visual perception. In addition, an adaptive texturized holographic algorithm is proposed to increase the robustness. Finally, the analytical performances are contrasted with simulation results where the imperceptibility and robustness of the proposed method are evaluated against standard attacks.

Compressive 4D spectro-volumetric imaging

In this Letter, we present a method for hyperspectral imaging of three-dimensional objects. A compressive sensing approach is utilized to remedy the acquisition effort required to capture the large amount of data. The spectral dimension is compressively sensed by means of a liquid crystal-based encoder, and the volumetric data are captured using a synthetic aperture integral imaging setup. We demonstrate reconstruction of spectro-volumetric tesseracts with hundreds of spectral bands at different depths without compromise of spatial resolution.

3D compressive spectral integral imaging

A novel compressive 3D imaging spectrometer based on the coded aperture snapshot spectral imager (CASSI) is proposed. By inserting a microlens array (MLA) into the CASSI system, one can capture spectral data of 3D objects in a single snapshot without requiring 3D scanning. The 3D spatio-spectral sensing phenomena is modelled by computational integral imaging in tandem with compressive coded aperture spectral imaging. A set of focal stack images is reconstructed from a single compressive measurement, and presented as images focused on different depth planes where the objects are located. The proposed optical system is demonstrated with simulations and experimental results.

Feasibility study for super-resolution 3D integral imaging using time-multiplexed compressive coding

We propose a novel super-resolution 3D II method using a time-multiplexed coding mask for improving the resolution of 3D imaging based on compressive sensing (CS) theory. Instead of sensing raw pixel data, the recording device measures the compressive samples of the observed 3D scene through a coding mask placed in the aerial pickup plane in a 3D II system. With the aid of CS framework, we design an optimum coding mask pattern and use the time-multiplexed scheme to achieve a sequence of low-resolution elemental images (EIs), which contain the subpixel details of the observed 3D scene. The super-resolution EIs array is further recovered by an optimization algorithm. Both computational reconstruction and optical experimental results show the validity of the proposed method.

A review of snapshot multidimensional optical imaging: measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons' spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition-also dubbed snapshot imaging-has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications.

Recent results of medium wave infrared compressive sensing

The application of compressive sensing (CS) for imaging has been extensively investigated and the underlying mathematical principles are well understood. The theory of CS is motivated by the sparse nature of real-world signals and images, and provides a framework in which high-resolution information can be recovered from low-resolution measurements. This, in turn, enables hardware concepts that require much fewer detectors than a conventional sensor. For infrared imagers there is a significant potential impact on the cost and footprint of the sensor. When smaller focal plane arrays (FPAs) to obtain large images are allowed, large formats FPAs are unnecessary. From a hardware standpoint, this benefit is independent of the actual level of compression and effective data rate reduction, which depend on the choice of codes and information recovery algorithm. Toward this end, we used a CS testbed for mid-wave infrared (MWIR) to experimentally show that information at high spatial resolution can be successfully recovered from measurements made with a small FPA. We describe the highly parallel and scalable CS architecture of the testbed, and its implementation using a reflective spatial light modulator and a focal plane array with variable pixel sizes. We also discuss the impact of real-world devices and the effect of sensor calibration that must be addressed in practice. Finally, we present preliminary results of image reconstruction, which demonstrate the testbed operation. These results experimentally confirm that high-resolution spatial information (for tasks such as imaging and target detection) can be successfully recovered from low-resolution measurements. We also discuss the potential system-level benefits of CS for infrared imaging, and some of the challenges that must be addressed in future infrared CS imagers designs.

Bispectral coding: compressive and high-quality acquisition of fluorescence and reflectance

Fluorescence widely coexists with reflectance in the real world, and an accurate representation of these two components in a scene is vitally important. Despite the rich knowledge of fluorescence mechanisms and behaviors, traditional fluorescence imaging approaches are quite limited in efficiency and quality. To address these two shortcomings, we propose a bispectral coding scheme to capture fluorescence and reflectance: multiplexing code is applied to excitation spectrums to raise the signal-to-noise ratio, and compressive sampling code is applied to emission spectrums for high efficiency. For computational reconstruction from the sparse coded measurements, the redundancy in both components promises recovery from sparse measurements, and the difference between their redundancies promises accurate separation. Mathematically, we cast the reconstruction as a joint optimization, whose solution can be derived by the Augmented Lagrange Method. In our experiment, results on both synthetic data and real data captured by our prototype validate the proposed approach, and we also demonstrate its advantages in two computer vision tasks--photorealistic relighting and segmentation.

Photonic-assisted multi-channel compressive sampling based on effective time delay pattern

In this paper, a photonic-assisted multi-channel compressive sampling scheme is proposed with one pseudo-random binary sequence (PRBS) source and Wavelength Division Multiplexing-based time delay. Meanwhile, the restricted isometry property of sensing matrix determined by the optimized time delay pattern is analyzed. In experiment, a four-channel photonic-assisted system with 5-GHz bandwidth was set up, where four-channel PRBS signals were generated by adding fiber-induced constant time delays to four-wavelength modulated PRBS signal, and a signal composed of twenty tones was recovered faithfully with four analog-to-digital converters (ADCs) with only 120-MHz-bandwidth.