Broadband near-infrared hyperspectral single pixel imaging for chemical characterization

Single-Pixel Imaging Based on Deep Learning Enhanced Singular Value Decomposition

We propose and demonstrate a single-pixel imaging method based on deep learning network enhanced singular value decomposition. The theoretical framework and the experimental implementation are elaborated and compared with the conventional methods based on Hadamard patterns or deep convolutional autoencoder network. Simulation and experimental results show that the proposed approach is capable of reconstructing images with better quality especially under a low sampling ratio down to 3.12%, or with fewer measurements or shorter acquisition time if the image quality is given. We further demonstrate that it has better anti-noise performance by introducing noises in the SPI systems, and we show that it has better generalizability by applying the systems to targets outside the training dataset. We expect that the developed method will find potential applications based on single-pixel imaging beyond the visible regime.

Optical single-pixel volumetric imaging by three-dimensional light-field illumination

Three-dimensional single-pixel imaging (3D SPI) has become an attractive imaging modality for both biomedical research and optical sensing. 3D-SPI techniques generally depend on time-of-flight or stereovision principle to extract depth information from backscattered light. However, existing implementations for these two optical schemes are limited to surface mapping of 3D objects at depth resolutions, at best, at the millimeter level. Here, we report 3D light-field illumination single-pixel microscopy (3D-LFI-SPM) that enables volumetric imaging of microscopic objects with a near-diffraction-limit 3D optical resolution. Aimed at 3D space reconstruction, 3D-LFI-SPM optically samples the 3D Fourier spectrum by combining 3D structured light-field illumination with single-element intensity detection. We build a 3D-LFI-SPM prototype that provides an imaging volume of ∼390 × 390 × 3,800 μm3 and achieves 2.7-μm lateral resolution and better than 37-μm axial resolution. Its capability of 3D visualization of label-free optical absorption contrast is demonstrated by imaging single algal cells in vivo. Our approach opens broad perspectives for 3D SPI with potential applications in various fields, such as biomedical functional imaging.

Spatially encoded hyperspectral compressive microscope for ultrabroadband VIS/NIR hyperspectral imaging

Hyperspectral imaging (HSI) has become a valuable tool in sample characterization in various scientific fields. While many approaches have been tested, specific applications and technology usually lead to only a narrow part of the spectrum being studied. We demonstrate the use of a broadband HSI setup based on compressed sensing capable of capturing data in visible (VIS), near-infrared (NIR), and short-wave infrared (SWIR) spectral regions. Using a tested design, we developed a dual configuration and tested its performance on a set of samples demonstrating spatial resolution and spectral reconstruction. Samples showing a potential use of the setup in optical defect detection are also tested. The setup showcases a dual single-pixel camera configuration capable of combining various detectors with a shared spatial modulation, further improving data efficiency and providing an affordable instrument from broadband spectral studies.

Single-pixel imaging with high spectral and spatial resolution

It has long been a challenge to obtain high spectral and spatial resolution simultaneously for the field of measurement and detection. Here we present a measurement system based on single-pixel imaging with compressive sensing that can realize excellent spectral and spatial resolution at the same time, as well as data compression. Our method can achieve high spectral and spatial resolution, which is different from the mutually restrictive relationship between the two in traditional imaging. In our experiments, 301 spectral channels are obtained in the band of 420-780 nm with a spectral resolution of 1.2 nm and a spatial resolution of 1.11 mrad. A sampling rate of 12.5% for a 64×64p i x e l image is obtained by using compressive sensing, which also reduces the measurement time; thus, high spectral and spatial resolution are realized simultaneously, even at a low sampling rate.

Diffraction-limited hyperspectral mid-infrared single-pixel microscopy

In this contribution, we demonstrate a wide-field hyperspectral mid-infrared (MIR) microscope based on multidimensional single-pixel imaging (SPI). The microscope employs a high brightness MIR supercontinuum source for broadband (1.55 [Formula: see text]-4.5 [Formula: see text]) sample illumination. Hyperspectral imaging capability is achieved by a single micro-opto-electro-mechanical digital micromirror device (DMD), which provides both spatial and spectral differentiation. For that purpose the operational spectral bandwidth of the DMD was significantly extended into the MIR spectral region. In the presented design, the DMD fulfills two essential tasks. On the one hand, as standard for the SPI approach, the DMD sequentially masks captured scenes enabling diffraction-limited imaging in the tens of millisecond time-regime. On the other hand, the diffraction at the micromirrors leads to dispersion of the projected field and thus allows for wavelength selection without the application of additional dispersive optical elements, such as gratings or prisms. In the experimental part, first of all, the imaging and spectral capabilities of the hyperspectral microscope are characterized. The spatial and spectral resolution is assessed by means of test targets and linear variable filters, respectively. At a wavelength of 4.15 [Formula: see text] a spatial resolution of 4.92 [Formula: see text] is achieved with a native spectral resolution better than 118.1 nm. Further, a post-processing method for drastic enhancement of the spectral resolution is proposed and discussed. The performance of the MIR hyperspectral microsopce is demonstrated for label-free chemical imaging and examination of polymer compounds and red blood cells. The acquisition and reconstruction of Hadamard sampled 64 [Formula: see text] 64 images is achieved in 450 ms and 162 ms, respectively. Thus, combined with an unprecedented intrinsic flexibiliy gained by a tunable field of view and adjustable spatial resolution, the demonstrated design drastically improves the sample throughput in MIR chemical and biomedical imaging.

Ghost translation: an end-to-end ghost imaging approach based on the transformer network

Artificial intelligence has recently been widely used in computational imaging. The deep neural network (DNN) improves the signal-to-noise ratio of the retrieved images, whose quality is otherwise corrupted due to the low sampling ratio or noisy environments. This work proposes a new computational imaging scheme based on the sequence transduction mechanism with the transformer network. The simulation database assists the network in achieving signal translation ability. The experimental single-pixel detector's signal will be 'translated' into a 2D image in an end-to-end manner. High-quality images with no background noise can be retrieved at a sampling ratio as low as 2%. The illumination patterns can be either well-designed speckle patterns for sub-Nyquist imaging or random speckle patterns. Moreover, our method is robust to noise interference. This translation mechanism opens a new direction for DNN-assisted ghost imaging and can be used in various computational imaging scenarios.

Single-pixel imaging with Gao-Boole patterns

Single-pixel imaging (SPI) can perceive the world using only a single-pixel detector, but long sampling times with a series of patterns are inevitable for SPI, which is the bottleneck for its practical application. Developing new patterns to reduce the sampling times might provide opportunities to address this challenge. Based on the Kronecker product of Hadamard matrix, we here design a complete set of new patterns, called Gao-Boole patterns, for SPI. Compared to orthogonal Hadamard basis patterns with elements valued as +1 or -1, our Gao-Boole patterns are non-orthogonal ones and the element values are designed as +1 or 0. Using our Gao-Boole patterns, the reconstructed quality of a target image (N × N pixels) is as high as the Hadamard one but only with half pattern numbers of the Hadamard ones, for both full sampling (N² for Gao-Boole patterns, 2N² for Hadamard basis patterns) and undersampling cases in experiment. Effectively reducing the patterns numbers and sampling times without sacrificing imaging quality, our designed Gao-Boole patterns provide a superior option for structural patterns in SPI and help to steer SPI toward practical imaging application.

Deep spatial-spectral prior with an adaptive dual attention network for single-pixel hyperspectral reconstruction

Recently, single-pixel imaging has shown great promise in developing cost-effective imaging systems, where coding and reconstruction are the keys to success. However, it also brings challenges in capturing hyperspectral information accurately and instantly. Many works have attempted to improve reconstruction performance in single-pixel hyperspectral imaging by applying various hand-crafted priors, leading to sub-optimal solutions. In this paper, we present the deep spatial-spectral prior with adaptive dual attention network for single-pixel hyperspectral reconstruction. Specifically, the spindle structure of the parameter sharing method is developed to integrate information across spatial and spectral dimensions of HSI, which can synergistically and efficiently extract global and local prior information of hyperspectral images from both shallow and deep layers. Particularly, a sequential adaptive dual attention block (SADAB), i.e., spatial attention and spectral attention, are devised to adaptively rescale informative features of spatial locations and spectral channels simultaneously, which can effectively boost the reconstruction accuracy. Experiment results on public HSI datasets demonstrate that the proposed method significantly outperforms the state-of-the-art algorithm in terms of reconstruction accuracy and speed.

Active hyperspectral imager using a tunable supercontinuum light source based on a MEMS Fabry-Perot interferometer

We have developed an active hyperspectral imager based on a tunable near-infrared supercontinuum light source. Non-dispersive wavelength selection of the supercontinuum laser source is achieved with a microelectromechanical Fabry-Perot interferometer. The tunable light source enables the use of any monochromatic imaging sensor with a suitable spectral sensitivity for hyperspectral imaging. The imager is characterized and demonstrated in the laboratory for remote detection of ice.

Spectral-Coding-Based Compressive Single-Pixel NIR Spectroscopy in the Sub-Millisecond Regime

In this contribution, we present a high-speed, multiplex, grating spectrometer based on a spectral coding approach that is founded on principles of compressive sensing. The spectrometer employs a single-pixel InGaAs detector to measure the signals encoded by an amplitude spatial light modulator (digital micromirror device, DMD). This approach leads to a speed advantage and multiplex sensitivity advantage atypical for standard dispersive systems. Exploiting the 18.2 kHz pattern rate of the DMD, we demonstrated 4.2 ms acquisition times for full spectra with a bandwidth of 450 nm (5250–4300 cm⁻¹; 1.9–2.33 µm). Due to the programmability of the DMD, spectral regions of interest can be chosen freely, thus reducing acquisition times further, down to the sub-millisecond regime. The adjustable resolving power of the system accessed by means of computer simulations is discussed, quantified for different measurement modes, and verified by comparison with a state-of-the-art Fourier-transform infrared spectrometer. We show measurements of characteristic polymer absorption bands in different operation regimes of the spectrometer. The theoretical multiplex advantage of 8 was experimentally verified by comparison of the noise behavior of the spectral coding approach and a standard line-scan approach.

Hadamard transform-based hyperspectral imaging using a single-pixel detector

In this paper, a single-pixel hyperspectral imager is developed based on the Hadamard transformation. The imager's design, fabrication, signal processing method, and experimental results are discussed. The single-pixel hyperspectral imager works in pushbroom mode and employs both spatial encoding and spectral encoding to acquire the hyperspectral data cube. Hadamard encoding patterns, which are known for their multiplexing advantage to achieve high signal-to-noise ratio (SNR), are used in both encoding schemes. A digital micromirror device (DMD) from Texas Instruments (TI) is used for slow spatial encoding and a resonant scanning mirror in combination with a fixed Hadamard mask is used for fast spectral encoding. In addition, the pushbroom operation can be achieved internally by spatially shifting the location of the Hadamard encoded slit on the DMD, thus the imager is able to acquire 3D data cubes without the need to scan it across the object. Although our experimental results demonstrate the hyperspectral data cubes of various objects over a 450 nm ∼ 750 nm visible spectral range, the proposed imager can be easily configured to be used at other wavelengths due to the single-pixel detection mechanism used.

Enriching absorption features for hyperspectral materials identification

Many materials have certain unique 'spectral fingerprints' in electromagnetic spectrum, which enables identification of materials based on hyperspectral imaging technique. In this paper, besides using the location information of absorptions, we propose to extract a group of real-valued parameters based on a detected absorption valley. These absorption parameters are chosen to characterize the details of the spectral absorption quantitatively, and are measured without human intervention. Moreover, we design an orientation descriptor to explore the local characterization for the shape representation of a hyperspectral absorption. According to the idea of information fusion, the augmentation of the absorption parameters and the orientation descriptor may increase the discriminatory ability and lead to an improved hyperspectral material identification. Simulations of material identification accuracy were carried out on two hyperspectral data sets, including a 7 classes of materials from ASD sensor, and a 16 classes of vegetation data from the AVIRIS 92AV3C. Results conclude the effectiveness of the method, which increases the identification accuracy compared to two classical approaches.

Probeless non-invasive near-infrared spectroscopic bioprocess monitoring using microspectrometer technology

Real-time measurements and adjustments of critical process parameters are essential for the precise control of fermentation processes and thus for increasing both quality and yield of the desired product. However, the measurement of some crucial process parameters such as biomass, product, and product precursor concentrations usually requires time-consuming offline laboratory analysis. In this work, we demonstrate the in-line monitoring of biomass, penicillin (PEN), and phenoxyacetic acid (POX) in a Penicilliumchrysogenum fed-batch fermentation process using low-cost microspectrometer technology operating in the near-infrared (NIR). In particular, NIR reflection spectra were taken directly through the glass wall of the bioreactor, which eliminates the need for an expensive NIR immersion probe. Furthermore, the risk of contaminations in the reactor is significantly reduced, as no direct contact with the investigated medium is required. NIR spectra were acquired using two sensor modules covering the spectral ranges 1350–1650 nm and 1550–1950 nm. Based on offline reference analytics, partial least squares (PLS) regression models were established for biomass, PEN, and POX either using data from both sensors separately or jointly. The established PLS models were tested on an independent validation fed-batch experiment. Root mean squared errors of prediction (RMSEP) were 1.61 g/L, 1.66 g/L, and 0.67 g/L for biomass, PEN, and POX, respectively, which can be considered an acceptable accuracy comparable with previously published results using standard process spectrometers with immersion probes. Altogether, the presented results underpin the potential of low-cost microspectrometer technology in real-time bioprocess monitoring applications.