Focal plane filter array engineering I: rectangular lattices

An Automatic and Optimal MPA Design Method

Raw polarimetric images are captured by a focal plane polarimeter which is covered by a micro-polarizer array (MPA). The design of the MPA plays a crucial role in polarimetric imaging. MPAs are predominantly designed according to expert engineering experience and rules of thumb. Typically, only one optimization criterion, maximizing bandwidth, is used to design the MPA. To select a design, an exhaustive search is usually performed on a very limited set of available polarizing patterns, which must be constrained in order to make the search tractable. In contrast, this paper proposes a fully automated and optimal MPA design method (AO-MPA) which generates significantly improved MPAs. Instead of the single criterion of bandwidth, we propose six design principles, and show how they can be utilized to mutually optimize the MPA design by formulating a tri-objective optimization problem with multiple constraints. A much larger set of possible MPA patterns is rapidly and automatically searched by applying advanced multi-objective optimization techniques. We have tested AO-MPA using two groups of experiments, in which AO-MPA is compared against several other leading MPA design methods, and the patterns generated by AO-MPA are compared against state-of-the-art patterns from the literature. The results, obtained using a public benchmark dataset, show that the AO-MPA method is very computationally efficient, and can find all optimal MPA patterns for all array sizes. Moreover, for each size, AO-MPA obtains all optimal layouts simultaneously. AO-MPA generates designs which require fewer polarization orientations, while also yielding better performance in estimating intensity measurements, Stokes vector and the degree of linear polarization. This results in MPAs which are easier to manufacture while also being more robust to noise.

Alternative linear microgrid polarimeters: design, analysis, and demosaicing considerations

Linear division of focal plane (DoFP), or integrated microgrid polarimeters, provide a measurement strategy for obtaining time-synchronized polarized intensity measurements across a scene. This is accomplished by masking pixels in the focal plane array sensor with a repeating pattern of different linear polarizers. The convention in industry has been to use a repeating 2×2 pattern of four linear polarizers with chosen polarizer orientation angles of {0,45,90,135}^∘. Alternative designs based upon other P×Q modulation patterns have been proposed that demonstrate improved performance over conventional microgrid arrays due to better utilization of bandwidth in the frequency domain. Here, we develop a model for linear DoFP snapshot polarimeters that provides an in-depth understanding of these devices in both the spatial and frequency domains and relate this model to previously reported generalized DoFP channeled polarimeter models. We then use the model to identify practical modulation patterns and study their performance through empirical simulations based upon data collected from real polarimeters. We demonstrate the validity of the developed model and compare the performance of the identified modulation schemes against a common set of ground truth images. We find that choosing alternative sets of polarizer angles, in conjunction with modulators that improve bandwidth usage, result in the best overall designs that can improve performance over conventional microgrid polarimeters.

Structured decomposition of a multi-snapshot nine-reconstructables Mueller matrix polarimeter

Snapshot channeled polarimeters forgo temporal modulation in favor of modulating polarization information in either space or wavenumber. We have recently introduced methodologies for describing both channeled and partial polarimeters. In this paper, we focus on the nine-reconstructables design, which limits the resolution loss by reducing the number of carriers. The architecture offers a number of favorable trade-offs: a factor of 5.44 increase in spatial bandwidth or a factor of 3.67 increase in spectral bandwidth, for a smaller amount of temporal bandwidth loss as dictated by the number of snapshots taken. The multi-snapshot structured decomposition given here allows one to analytically shape the measured space with optimal noise characteristics and minimum system complexity. A two-snapshot system can measure a premeditated set of 14 reconstructables; we provide the null space for the subset of optimal systems that also achieve better SNR than the baseline single-snapshot system. A three-snapshot system can measure all 16 Mueller elements while offering an overall 26.3% or 50.4% better bandwidth-SNR figure of merit for the spectral and spatial systems, respectively. Finally, four-snapshot systems provide diminishing returns, but may be more implementable.

Joint Demosaicing and Denoising Based on a Variational Deep Image Prior Neural Network

A joint demosaicing and denoising task refers to the task of simultaneously reconstructing and denoising a color image from a patterned image obtained by a monochrome image sensor with a color filter array. Recently, inspired by the success of deep learning in many image processing tasks, there has been research to apply convolutional neural networks (CNNs) to the task of joint demosaicing and denoising. However, such CNNs need many training data to be trained, and work well only for patterned images which have the same amount of noise they have been trained on. In this paper, we propose a variational deep image prior network for joint demosaicing and denoising which can be trained on a single patterned image and works for patterned images with different levels of noise. We also propose a new RGB color filter array (CFA) which works better with the proposed network than the conventional Bayer CFA. Mathematical justifications of why the variational deep image prior network suits the task of joint demosaicing and denoising are also given, and experimental results verify the performance of the proposed method.

Generalized Polarimetric Dehazing Method Based on Low-Pass Filtering in Frequency Domain

Polarimetric dehazing methods can significantly enhance the quality of hazy images. However, current methods are not robust enough under different imaging conditions. In this paper, we propose a generalized polarimetric dehazing method based on low-pass filtering in the frequency domain. This method can accurately estimate the polarized state of the scattering light automatically without adjusting bias parameters. Experimental results show the effectiveness and robustness of our proposed method in different hazy weather and scattering underwater environments with different densities. Furthermore, computational efficiency is enhanced more than 70% compared to the polarimetric dehazing method we proposed previously.

Imaging dynamic scenes with a spatio-temporally channeled polarimeter

Most channeled polarimeters modulate the intensity in a single independent domain such as space, time, or wavenumber. Recently we proposed and modeled a concept for a system modulated simultaneously in space and time [Opt. Lett.43, 2768 - 2771 (2018)] and demonstrated that superior performance could be obtained by trading off spatial and temporal bandwidth in the system. Here we present results from a prototype realization of such a system and demonstrate quantitatively that the spatial modulation transfer function of the imager can be improved by choosing the appropriate modulation strategy for a given scene spatial and temporal bandwidth. We demonstrate that a hybrid modulation system can achieve the high spatial frequency performance of a time modulated system for static scenes, or it can achieve the high temporal frequency performance of a spatially modulated system for rapidly varying scenes, and it can out perform both systems for scenes with intermediate bandwidth in both domains. Moreover, the physical system implementation is essentially the same for each system type, which in principle allows the reconstruction strategy to be selected in real-time by choosing the appropriate reconstruction filters.

Multi-carrier channeled polarimetry for photoelastic modulator systems

Photoelastic modulator-based polarimeters use multi-carrier modulation schemes that are more complicated than the single carriers of rotating optics. Current state-of-the-art reconstruction implementations favor mathematical simplicity by using significantly abridged subsets of channels. In this Letter, we extend our generalized channeled polarimetry principles to address the challenges associated with multi-carrier modulation schemes. We demonstrate the performance forfeited by existing systems through the use of an incomplete set of information channels, as well as propose a set of more optimal system parameters that achieve better reconstruction noise characteristics. The overall improvement corresponds to a better sensitivity of up to a factor of six.

Polarization state generation and measurement with a single metasurface

The constituent elements of metasurfaces may be designed with explicit polarization dependence, making metasurfaces a fascinating platform for new polarization optics. In this work we show that a metasurface grating can be designed to produce arbitrarily specified polarization states on a set of defined diffraction orders given that the polarization of the incident beam is known. We also demonstrate that, when used in a reverse configuration, the same grating may be used as a parallel snapshot polarimeter, requiring a minimum of bulk polarization optics. We demonstrate its use in measuring partially polarized light, and show that it performs favorably in comparison to a commercial polarimeter. This work is of consequence in any application requiring lightweight, compact, and low-cost polarization optics, polarimetry, or polarization imaging.

Channeled spatio-temporal Stokes polarimeters

We present the analysis and design of spatio-temporal channeled Stokes polarimeters. We extend our recent work on optimal pixelated polarizer arrays by utilizing temporal carrier generation, resulting in polarimeters that achieve super-resolution via the tradeoff between spatial bandwidth and temporal bandwidth. Utilizing the channel space description, we present a linear-Stokes design and two full-Stokes imaging polarimeter designs that have the potential to operate at the full frame rate of the imaging sensor of the system by using hybrid spatio-temporal carriers. If the objects are not spatially bandlimited, the achievable temporal bandwidth is more difficult to analyze; however, a spatio-temporal tradeoff still exists.

Optimal bandwidth and systematic error of full-Stokes micropolarizer arrays

In this paper, we present the first in-depth analysis of the bandwidth tradeoffs, error performance, and noise resiliency of full-Stokes micropolarizer array (MPA) designs. By applying our Fourier domain tools that provide a systematic way for arranging information carriers and allocating bandwidth, we develop a number of new full-Stokes MPA layouts and compare them to the existing full-Stokes MPAs in the literature, all of which use 2×2 pixel unit cells to build the MPA. We compare the reconstruction accuracy afforded by these traditional designs with the generalized 2×L family of MPAs, a 3×3 tiling, as well as a 2×2×3 layout that uses multiple snapshots and trades off temporal resolution for spatial resolution. Of those systems, the hybrid spatiotemporally modulated 2×2×3 MPA provisions the most bandwidth and provides the highest reconstruction accuracy, while the modified 2×L family remains the best performing single-snapshot MPA. Additionally, we study the degradation of reconstruction accuracy under the presence of systematic error in MPA fabrication. We find that reducing the amount of correlated error is by far the largest factor in ensuring robust performance.

Division of amplitude RGB full-Stokes camera using micro-polarizer arrays: erratum

We correct two figures showing the micro-polarizer arrangement on the RGB full-Stokes camera.