Focal plane array-based compressive imaging in medium wave infrared: modeling, implementation, and challenges

Mid-infrared single-pixel imaging at the single-photon level

Single-pixel cameras have recently emerged as promising alternatives to multi-pixel sensors due to reduced costs and superior durability, which are particularly attractive for mid-infrared (MIR) imaging pertinent to applications including industry inspection and biomedical diagnosis. To date, MIR single-pixel photon-sparse imaging has yet been realized, which urgently calls for high-sensitivity optical detectors and high-fidelity spatial modulators. Here, we demonstrate a MIR single-photon computational imaging with a single-element silicon detector. The underlying methodology relies on nonlinear structured detection, where encoded time-varying pump patterns are optically imprinted onto a MIR object image through sum-frequency generation. Simultaneously, the MIR radiation is spectrally translated into the visible region, thus permitting infrared single-photon upconversion detection. Then, the use of advanced algorithms of compressed sensing and deep learning allows us to reconstruct MIR images under sub-Nyquist sampling and photon-starving illumination. The presented paradigm of single-pixel upconversion imaging is featured with single-pixel simplicity, single-photon sensitivity, and room-temperature operation, which would establish a new path for sensitive imaging at longer infrared wavelengths or terahertz frequencies, where high-sensitivity photon counters and high-fidelity spatial modulators are typically hard to access.

Compressive Sensing Imaging Spectrometer for UV-Vis Stellar Spectroscopy: Instrumental Concept and Performance Analysis

Compressive sensing (CS) has been proposed as a disruptive approach to developing a novel class of optical instrumentation used in diverse application domains. Thanks to sparsity as an inherent feature of many natural signals, CS allows for the acquisition of the signal in a very compact way, merging acquisition and compression in a single step and, furthermore, offering the capability of using a limited number of detector elements to obtain a reconstructed image with a larger number of pixels. Although the CS paradigm has already been applied in several application domains, from medical diagnostics to microscopy, studies related to space applications are very limited. In this paper, we present and discuss the instrumental concept, optical design, and performances of a CS imaging spectrometer for ultraviolet-visible (UV-Vis) stellar spectroscopy. The instrument-which is pixel-limited in the entire 300 nm-650 nm spectral range-features spectral sampling that ranges from 2.2 nm@300 nm to 22 nm@650 nm, with a total of 50 samples for each spectrum. For data reconstruction quality, the results showed good performance, measured by several quality metrics chosen from those recommended by CCSDS. The designed instrument can achieve compression ratios of 20 or higher without a significant loss of information. A pros and cons analysis of the CS approach is finally carried out, highlighting main differences with respect to a traditional system.

Polarization Super-Resolution Imaging Method Based on Deep Compressed Sensing

The division of focal plane (DoFP) polarization imaging sensors, which can simultaneously acquire the target's two-dimensional spatial information and polarization information, improves the detection resolution and recognition capability by capturing the difference in polarization characteristics between the target and the background. In this paper, we propose a novel polarization imaging method based on deep compressed sensing (DCS) by adding digital micromirror devices (DMD) to an optical system and simulating the polarization transmission model of the optical system to reconstruct high-resolution images under low sampling rate conditions. By building a simulated dataset, training a polarization super-resolution imaging network, and showing excellent reconstructions on real shooting scenes, compared to current algorithms, our model has a higher peak signal-to-noise ratio (PSNR), which validates the feasibility of our approach.

Single pixel imaging at high pixel resolutions

The usually reported pixel resolution of single pixel imaging (SPI) varies between 32 × 32 and 256 × 256 pixels falling far below imaging standards with classical methods. Low resolution results from the trade-off between the acceptable compression ratio, the limited DMD modulation frequency, and reasonable reconstruction time, and has not improved significantly during the decade of intensive research on SPI. In this paper we show that image measurement at the full resolution of the DMD, which lasts only a fraction of a second, is possible for sparse images or in a situation when the field of view is limited but is a priori unknown. We propose the sampling and reconstruction strategies that enable us to reconstruct sparse images at the resolution of 1024 × 768 within the time of 0.3s. Non-sparse images are reconstructed with less details. The compression ratio is on the order of 0.4% which corresponds to an acquisition frequency of 7Hz. Sampling is differential, binary, and non-adaptive, and includes information on multiple partitioning of the image which later allows us to determine the actual field of view. Reconstruction is based on the differential Fourier domain regularized inversion (D-FDRI). The proposed SPI framework is an alternative to both adaptive SPI, which is challenging to implement in real time, and to classical compressive sensing image recovery methods, which are very slow at high resolutions.

Differential real-time single-pixel imaging with Fourier domain regularization: applications to VIS-IR imaging and polarization imaging

The speed and quality of single-pixel imaging (SPI) are fundamentally limited by image modulation frequency and by the levels of optical noise and compression noise. In an approach to come close to these limits, we introduce a SPI technique, which is inherently differential, and comprises a novel way of measuring the zeroth spatial frequency of images and makes use of varied thresholding of sampling patterns. With the proposed sampling, the entropy of the detection signal is increased in comparison to standard SPI protocols. Image reconstruction is obtained with a single matrix-vector product so the cost of the reconstruction method scales proportionally with the number of measured samples. A differential operator is included in the reconstruction and following the method is based on finding the generalized inversion of the modified measurement matrix with regularization in the Fourier domain. We demonstrate 256 × 256 SPI at up to 17 Hz at visible and near-infrared wavelength ranges using 2 polarization or spectral channels. A low bit-resolution data acquisition device with alternating-current-coupling can be used in the measurement indicating that the proposed method combines improved noise robustness with a differential removal of the direct current component of the signal.

Stray light correction for medium wave infrared focal plane array-based compressive imaging

With focal plane array-based (FPA) compressive imaging (CI), high-resolution medium wave infrared (MWIR) images can be reconstructed by a low-resolution FPA sensor. However, in MWIR FPA CI system, the stray light is inevitable, which reduces the image contrast and increases the blocky structural artifacts of the reconstructed images. In this work, we focus on the stray light in MWIR FPA CI system. This paper investigates the sources of stray light in MWIR FPA CI system and modifies the systematic radiation model. According to the systematic computation model, we illustrate that stray light impedes the accurate sampling of compressive measurements in the MWIR FPA CI system, which may increase the blocky structural artifacts in the reconstructed high-resolution images. With the help of digital micro-mirror device modulation, we propose an operational method to substantially correct the effect of the stray light in MWIR FPA CI system, which can improve the image contrast and reduce the blocky structural artifacts of the reconstructed images, while not significantly increasing the cost of image acquisition and computation. Based on the experimental results obtained from the actual MWIR FPA CI system, we have verified the effectiveness and practicability of the proposed stray light correction method.

Biomolecular and bioanalytical applications of infrared spectroscopy - A review

Infrared (IR; or mid-infrared, MIR; 4000-400 cm^-1; 2500-25,000 nm) spectroscopy has become one of the most powerful and versatile tools at the disposal of modern bioscience. Because of its high molecular specificity, applicability to wide variety of samples, rapid measurement and non-invasivity, IR spectroscopy forms a potent approach to elucidate qualitative and quantitative information from various kinds of biological material. For these reasons, it became an established bioanalytical technique with diverse applications. This work aims to be a comprehensive and critical review of the recent accomplishments in the field of biomolecular and bioanalytical IR spectroscopy. That progress is presented on a wider background, with fundamental characteristics, the basic principles of the technique outlined, and its scientific capability directly compared with other methods being used in similar fields (e.g. near-infrared, Raman, fluorescence). The article aims to present a complete examination of the topic, as it touches the background phenomena, instrumentation, spectra processing and data analytical methods, spectra interpretation and related information. To suit this goal, the article includes a tutorial information essential to obtain a thorough perspective of bio-related applications of the reviewed methodologies. The importance of the fundamental factors to the final performance and applicability of IR spectroscopy in various areas of bioscience is explained. This information is interpreted in critical way, with aim to gain deep understanding why IR spectroscopy finds extraordinarily intensive use in this remarkably diverse and dynamic field of research and utility. The major focus is placed on the diversity of the applications in which IR biospectroscopy has been established so far and those onto which it is expanding nowadays. This includes qualitative and quantitative analytical spectroscopy, spectral imaging, medical diagnosis, monitoring of biophysical processes, and studies of physicochemical properties and dynamics of biomolecules. The application potential of IR spectroscopy in light of the current accomplishments and the future prospects is critically evaluated and its significance in the progress of bioscience is comprehensively presented.

Non-uniformity correction for medium wave infrared focal plane array-based compressive imaging

As a super-resolution imaging method, high-resolution medium wave infrared (MWIR) images can be obtained from a low-resolution focal plane array-based (FPA) sensor using compressive imaging (CI) technology. As a common problem in MWIR FPA imaging, the non-uniformity reduces image quality, which is turning worse in MWIR FPA CI. This paper investigates the source of the non-uniformity of MWIR FPA CI, both in the captured low-resolution MWIR images and in the reconstructed high-resolution ones. According to the system model and the image super-resolution computation process of FPA CI, we propose a calibration-based non-uniformity correction (NUC) method for MWIR FPA CI. Based on the actual MWIR FPA CI system, the effectiveness and practicability of the proposed NUC method are verified, obtaining better results than the traditional method. According to the theoretical analysis and experimental results, the particularities of the non-uniformity in MWIR FPA CI are discovered and discussed, which have certain great guiding significance and practical value.

DMD Mask Construction to Suppress Blocky Structural Artifacts for Medium Wave Infrared Focal Plane Array-Based Compressive Imaging

With medium wave infrared (MWIR) focal plane array-based (FPA) compressive imaging (CI), high-resolution images can be obtained with a low-resolution MWIR sensor. However, restricted by the size of digital micro-mirror devices (DMD), aperture interference is inevitable. According to the system model of FPA CI, aperture interference aggravates the blocky structural artifacts (BSA) in the reconstructed images, which reduces the image quality. In this paper, we propose a novel DMD mask design strategy, which can effectively suppress BSA and maximize the reconstruction efficiency. Compared with random binary codes, the storage space and computation cost can be significantly reduced. Based on the actual MWIR FPA CI system, we demonstrate the proposed DMD masks can effectively suppress the BSA in the reconstructed images. In addition, a new evaluation index, blocky root mean square error, is proposed to indicate the BSA in FPA CI.