Recent results of medium wave infrared compressive sensing

High-Resolution Single-Pixel Imaging of Spatially Sparse Objects: Real-Time Imaging in the Near-Infrared and Visible Wavelength Ranges Enhanced with Iterative Processing or Deep Learning

We demonstrate high-resolution single-pixel imaging (SPI) in the visible and near-infrared wavelength ranges using an SPI framework that incorporates a novel, dedicated sampling scheme and a reconstruction algorithm optimized for the rapid imaging of highly sparse scenes at the native digital micromirror device (DMD) resolution of 1024 × 768. The reconstruction algorithm consists of two stages. In the first stage, the vector of SPI measurements is multiplied by the generalized inverse of the measurement matrix. In the second stage, we compare two reconstruction approaches: one based on an iterative algorithm and the other on a trained neural network. The neural network outperforms the iterative method when the object resembles the training set, though it lacks the generality of the iterative approach. For images captured at a compression of 0.41 percent, corresponding to a measurement rate of 6.8 Hz with a DMD operating at 22 kHz, the typical reconstruction time on a desktop with a medium-performance GPU is comparable to the image acquisition rate. This allows the proposed SPI method to support high-resolution dynamic SPI in a variety of applications, using a standard SPI architecture with a DMD modulator operating at its native resolution and bandwidth, and enabling the real-time processing of the measured data with no additional delay on a standard desktop PC.

Compressive Sensing Imaging Spectrometer for UV-Vis Stellar Spectroscopy: Instrumental Concept and Performance Analysis

Compressive sensing (CS) has been proposed as a disruptive approach to developing a novel class of optical instrumentation used in diverse application domains. Thanks to sparsity as an inherent feature of many natural signals, CS allows for the acquisition of the signal in a very compact way, merging acquisition and compression in a single step and, furthermore, offering the capability of using a limited number of detector elements to obtain a reconstructed image with a larger number of pixels. Although the CS paradigm has already been applied in several application domains, from medical diagnostics to microscopy, studies related to space applications are very limited. In this paper, we present and discuss the instrumental concept, optical design, and performances of a CS imaging spectrometer for ultraviolet-visible (UV-Vis) stellar spectroscopy. The instrument-which is pixel-limited in the entire 300 nm-650 nm spectral range-features spectral sampling that ranges from 2.2 nm@300 nm to 22 nm@650 nm, with a total of 50 samples for each spectrum. For data reconstruction quality, the results showed good performance, measured by several quality metrics chosen from those recommended by CCSDS. The designed instrument can achieve compression ratios of 20 or higher without a significant loss of information. A pros and cons analysis of the CS approach is finally carried out, highlighting main differences with respect to a traditional system.

Single pixel imaging at high pixel resolutions

The usually reported pixel resolution of single pixel imaging (SPI) varies between 32 × 32 and 256 × 256 pixels falling far below imaging standards with classical methods. Low resolution results from the trade-off between the acceptable compression ratio, the limited DMD modulation frequency, and reasonable reconstruction time, and has not improved significantly during the decade of intensive research on SPI. In this paper we show that image measurement at the full resolution of the DMD, which lasts only a fraction of a second, is possible for sparse images or in a situation when the field of view is limited but is a priori unknown. We propose the sampling and reconstruction strategies that enable us to reconstruct sparse images at the resolution of 1024 × 768 within the time of 0.3s. Non-sparse images are reconstructed with less details. The compression ratio is on the order of 0.4% which corresponds to an acquisition frequency of 7Hz. Sampling is differential, binary, and non-adaptive, and includes information on multiple partitioning of the image which later allows us to determine the actual field of view. Reconstruction is based on the differential Fourier domain regularized inversion (D-FDRI). The proposed SPI framework is an alternative to both adaptive SPI, which is challenging to implement in real time, and to classical compressive sensing image recovery methods, which are very slow at high resolutions.

Fast lightweight framework for time-of-flight super-resolution based on block compressed sensing

Compressive time-of-flight (ToF) imaging for super-resolution (SR) has tremendous development potential owing to its cost-effectiveness and simplicity. However, existing compressive ToF methods are difficult to apply in practical situations because of their low efficiency and high data storage requirements. In this paper, we propose a fast and lightweight compressive ToF framework for SR. The block compressed sensing method, which shows distinct characteristics of high efficiency and low implementation cost, is introduced into the SR image acquisition and data transmission processes. Based on this framework, we establish a prototype system and verify it experimentally. Compared with existing compressive ToF systems, both the reconstruction time and data storage requirements are significantly decreased. We believe that this study provides a development direction for compressive ToF imaging and effective guidance for researchers realizing highly efficient and lightweight SR image reconstruction.

Earth Observation via Compressive Sensing: The Effect of Satellite Motion

In the framework of earth observation for scientific purposes, we consider a multiband spatial compressive sensing (CS) acquisition system, based on a pushbroom scanning. We conduct a series of analyses to address the effects of the satellite movement on its performance in a context of a future space mission aimed at monitoring the cryosphere. We initially apply the state-of-the-art techniques of CS to static images, and evaluate the reconstruction errors on representative scenes of the earth. We then extend the reconstruction algorithms to pushframe acquisitions, i.e., static images processed line-by-line, and pushbroom acquisitions, i.e., moving frames, which consider the payload displacement during acquisition. A parallel analysis on the classical pushbroom acquisition strategy is also performed for comparison. Design guidelines following this analysis are then provided.

Differential real-time single-pixel imaging with Fourier domain regularization: applications to VIS-IR imaging and polarization imaging

The speed and quality of single-pixel imaging (SPI) are fundamentally limited by image modulation frequency and by the levels of optical noise and compression noise. In an approach to come close to these limits, we introduce a SPI technique, which is inherently differential, and comprises a novel way of measuring the zeroth spatial frequency of images and makes use of varied thresholding of sampling patterns. With the proposed sampling, the entropy of the detection signal is increased in comparison to standard SPI protocols. Image reconstruction is obtained with a single matrix-vector product so the cost of the reconstruction method scales proportionally with the number of measured samples. A differential operator is included in the reconstruction and following the method is based on finding the generalized inversion of the modified measurement matrix with regularization in the Fourier domain. We demonstrate 256 × 256 SPI at up to 17 Hz at visible and near-infrared wavelength ranges using 2 polarization or spectral channels. A low bit-resolution data acquisition device with alternating-current-coupling can be used in the measurement indicating that the proposed method combines improved noise robustness with a differential removal of the direct current component of the signal.

High-resolution fast mid-wave infrared compressive imaging

In the mid-wave infrared (MIR) band, large detector arrays are extremely costly and technically difficult to be manufactured. Thus, it is difficult to obtain high-resolution images for a conventional MIR camera. Spatial compressive imaging can improve resolution. However, system errors due to misalignment or optical aberrations degrade reconstruction quality significantly. Another common issue for compressive imaging is the slow imaging speed, which is caused by slow measurement collection and reconstruction processes. To deal with the two issues, we use an imaging calibration method to improve reconstruction quality and a sliding window measurement collection strategy plus a reconstruction algorithm accelerated by parallel computing to fasten the speed. We build a prototype of a compressive imaging camera with an angular resolution 1.17 lp/mrad. A four-bar target is used as an object. We reconstruct a moving scene of size $1280 \times 1024$ with a frame rate 20 frames per second.

Stray light correction for medium wave infrared focal plane array-based compressive imaging

With focal plane array-based (FPA) compressive imaging (CI), high-resolution medium wave infrared (MWIR) images can be reconstructed by a low-resolution FPA sensor. However, in MWIR FPA CI system, the stray light is inevitable, which reduces the image contrast and increases the blocky structural artifacts of the reconstructed images. In this work, we focus on the stray light in MWIR FPA CI system. This paper investigates the sources of stray light in MWIR FPA CI system and modifies the systematic radiation model. According to the systematic computation model, we illustrate that stray light impedes the accurate sampling of compressive measurements in the MWIR FPA CI system, which may increase the blocky structural artifacts in the reconstructed high-resolution images. With the help of digital micro-mirror device modulation, we propose an operational method to substantially correct the effect of the stray light in MWIR FPA CI system, which can improve the image contrast and reduce the blocky structural artifacts of the reconstructed images, while not significantly increasing the cost of image acquisition and computation. Based on the experimental results obtained from the actual MWIR FPA CI system, we have verified the effectiveness and practicability of the proposed stray light correction method.

Non-uniformity correction for medium wave infrared focal plane array-based compressive imaging

As a super-resolution imaging method, high-resolution medium wave infrared (MWIR) images can be obtained from a low-resolution focal plane array-based (FPA) sensor using compressive imaging (CI) technology. As a common problem in MWIR FPA imaging, the non-uniformity reduces image quality, which is turning worse in MWIR FPA CI. This paper investigates the source of the non-uniformity of MWIR FPA CI, both in the captured low-resolution MWIR images and in the reconstructed high-resolution ones. According to the system model and the image super-resolution computation process of FPA CI, we propose a calibration-based non-uniformity correction (NUC) method for MWIR FPA CI. Based on the actual MWIR FPA CI system, the effectiveness and practicability of the proposed NUC method are verified, obtaining better results than the traditional method. According to the theoretical analysis and experimental results, the particularities of the non-uniformity in MWIR FPA CI are discovered and discussed, which have certain great guiding significance and practical value.

DMD Mask Construction to Suppress Blocky Structural Artifacts for Medium Wave Infrared Focal Plane Array-Based Compressive Imaging

With medium wave infrared (MWIR) focal plane array-based (FPA) compressive imaging (CI), high-resolution images can be obtained with a low-resolution MWIR sensor. However, restricted by the size of digital micro-mirror devices (DMD), aperture interference is inevitable. According to the system model of FPA CI, aperture interference aggravates the blocky structural artifacts (BSA) in the reconstructed images, which reduces the image quality. In this paper, we propose a novel DMD mask design strategy, which can effectively suppress BSA and maximize the reconstruction efficiency. Compared with random binary codes, the storage space and computation cost can be significantly reduced. Based on the actual MWIR FPA CI system, we demonstrate the proposed DMD masks can effectively suppress the BSA in the reconstructed images. In addition, a new evaluation index, blocky root mean square error, is proposed to indicate the BSA in FPA CI.

Focal plane array-based compressive imaging in medium wave infrared: modeling, implementation, and challenges

As a super-resolution imaging method, compressive imaging (CI) has become a research hotspot in recent years. However, most researchers focus on the visible light and near-infrared regime. Further, most experimental studies are confined to a single-pixel camera, and there are comparatively few reports of the experimental studies about CI in the field of focal plane array-based (FPA) CI in the medium-wave infrared (MWIR). This paper derives the system model for an FPA CI system, describes the generation process of DMD masks, and modifies the block-based compressive sensing algorithm to be applicable to the FPA CI system. Based on the actual FPA CI system in MWIR, high-resolution MWIR images are obtained from a low-resolution MWIR sensor, realizing 16 times MWIR image super-resolution. However, imaging quality is not as ideal as that in visible light FPA CI system because of the particularities of MWIR. We analyze the particularities of an FPA CI system in MWIR and provide suggestions on how to solve them to improve performance. This work could provide guidance for researchers to build an experimental FPA CI system in MWIR.

Computational imaging with a highly parallel image-plane-coded architecture: challenges and solutions

This paper investigates a highly parallel extension of the single-pixel camera based on a focal plane array. It discusses the practical challenges that arise when implementing such an architecture and demonstrates that system-specific optical effects must be measured and integrated within the system model for accurate image reconstruction. Three different projection lenses were used to evaluate the ability of the system to accommodate varying degrees of optical imperfection. Reconstruction of binary and grayscale objects using system-specific models and Nesterov's proximal gradient method produced images with higher spatial resolution and lower reconstruction error than using either bicubic interpolation or a theoretical system model that assumes ideal optical behavior. The high-quality images produced using relatively few observations suggest that higher throughput imaging may be achieved with such architectures than with conventional single-pixel cameras. The optical design considerations and quantitative performance metrics proposed here may lead to improved image reconstruction for similar highly parallel systems.