Low-cost single-pixel 3D imaging by using an LED array

Efficient single-pixel imaging based on a compact fiber laser array and untrained neural network

This paper presents an efficient scheme for single-pixel imaging (SPI) utilizing a phase-controlled fiber laser array and an untrained deep neural network. The fiber lasers are arranged in a compact hexagonal structure and coherently combined to generate illuminating light fields. Through the utilization of high-speed electro-optic modulators in each individual fiber laser module, the randomly modulated fiber laser array enables rapid speckle projection onto the object of interest. Furthermore, the untrained deep neural network is incorporated into the image reconstructing process to enhance the quality of the reconstructed images. Through simulations and experiments, we validate the feasibility of the proposed method and successfully achieve high-quality SPI utilizing the coherent fiber laser array at a sampling ratio of 1.6%. Given its potential for high emitting power and rapid modulation, the SPI scheme based on the fiber laser array holds promise for broad applications in remote sensing and other applicable fields.

Ghost imaging based on Fermat spiral laser array designed for remote sensing

We propose a Fermat spiral laser array as illumination source in ghost imaging. Due to the aperiodic structure, the Fermat spiral laser array generates illuminating light field without spatial periodicity on the normalized second-order intensity correlation function. A single-pixel detector is used to receive the signal light from object for image reconstruction. The effects of laser array parameters on the quality of ghost imaging are analyzed comprehensively. Through experimental demonstration, the Fermat spiral laser array successfully achieves ghost imaging with high quality by combining with the compressive sensing reconstruction algorithm. This method is expected to be applied in remote sensing by combining with phased and collimated fiber laser array equipped with the high emitting power and high-speed modulation frequency.

Efficient compressed sensing reconstruction for 3D fluorescence microscopy using OptoMechanical Modulation Tomography (OMMT) with a 1+2D regularization

OptoMechanical Modulation Tomography (OMMT) exploits compressed sensing to reconstruct high resolution microscopy volumes from fewer measurement images compared to exhaustive section sampling in conventional light sheet microscopy. Nevertheless, the volumetric reconstruction process is computationally expensive, making it impractically slow to use on large-size images, and prone to generating visual artefacts. Here, we propose a reconstruction approach that uses a 1+2D Total Variation (TV1+2) regularization that does not generate such artefacts and is amenable to efficient implementation using parallel computing. We evaluate our method for accuracy and scaleability on simulated and experimental data. Using a high quality, but computationally expensive, Plug-and-Play (PnP) method that uses the BM4D denoiser as a benchmark, we observe that our approach offers an advantageous trade-off between speed and accuracy.

Advances in Ghost Imaging of Moving Targets: A Review

Ghost imaging is a novel imaging technique that utilizes the intensity correlation property of an optical field to retrieve information of the scene being measured. Due to the advantages of simple structure, high detection efficiency, etc., ghost imaging exhibits broad application prospects in the fields of space remote sensing, optical encryption transmission, medical imaging, and so on. At present, ghost imaging is gradually developing toward practicality, in which ghost imaging of moving targets is becoming a much-needed breakthrough link. At this stage, we can improve the imaging speed and improve the imaging quality to seek a more optimized ghost imaging scheme for moving targets. Based on the principle of moving target ghost imaging, this review summarizes and compares the existing methods for ghost imaging of moving targets. It also discusses the research direction and the technical challenges at the current stage to provide references for further promotion of the instantiation of ghost imaging applications.

Comparison of Common Algorithms for Single-Pixel Imaging via Compressed Sensing

Single-pixel imaging (SPI) uses a single-pixel detector instead of a detector array with a lot of pixels in traditional imaging techniques to realize two-dimensional or even multi-dimensional imaging. For SPI using compressed sensing, the target to be imaged is illuminated by a series of patterns with spatial resolution, and then the reflected or transmitted intensity is compressively sampled by the single-pixel detector to reconstruct the target image while breaking the limitation of the Nyquist sampling theorem. Recently, in the area of signal processing using compressed sensing, many measurement matrices as well as reconstruction algorithms have been proposed. It is necessary to explore the application of these methods in SPI. Therefore, this paper reviews the concept of compressive sensing SPI and summarizes the main measurement matrices and reconstruction algorithms in compressive sensing. Further, the performance of their applications in SPI through simulations and experiments is explored in detail, and then their advantages and disadvantages are summarized. Finally, the prospect of compressive sensing with SPI is discussed.

Real-time single-pixel imaging using a system on a chip field-programmable gate array

Unlike conventional imaging, the single-pixel imaging technique uses a single-element detector, which enables high sensitivity, broad wavelength, and noise robustness imaging. However, it has several challenges, particularly requiring extensive computations for image reconstruction with high image quality. Therefore, high-performance computers are required for real-time reconstruction with higher image quality. In this study, we developed a compact dedicated computer for single-pixel imaging using a system on a chip field-programmable gate array (FPGA), which enables real-time reconstruction at 40 frames per second with an image size of 128 × 128 pixels. An FPGA circuit was implemented with the proposed reconstruction algorithm to obtain higher image quality by introducing encoding mask pattern optimization. The dedicated computer can accelerate the reconstruction 10 times faster than a recent CPU. Because it is very compact compared with typical computers, it can expand the application of single-pixel imaging to the Internet of Things and outdoor applications.

Single pixel imaging at high pixel resolutions

The usually reported pixel resolution of single pixel imaging (SPI) varies between 32 × 32 and 256 × 256 pixels falling far below imaging standards with classical methods. Low resolution results from the trade-off between the acceptable compression ratio, the limited DMD modulation frequency, and reasonable reconstruction time, and has not improved significantly during the decade of intensive research on SPI. In this paper we show that image measurement at the full resolution of the DMD, which lasts only a fraction of a second, is possible for sparse images or in a situation when the field of view is limited but is a priori unknown. We propose the sampling and reconstruction strategies that enable us to reconstruct sparse images at the resolution of 1024 × 768 within the time of 0.3s. Non-sparse images are reconstructed with less details. The compression ratio is on the order of 0.4% which corresponds to an acquisition frequency of 7Hz. Sampling is differential, binary, and non-adaptive, and includes information on multiple partitioning of the image which later allows us to determine the actual field of view. Reconstruction is based on the differential Fourier domain regularized inversion (D-FDRI). The proposed SPI framework is an alternative to both adaptive SPI, which is challenging to implement in real time, and to classical compressive sensing image recovery methods, which are very slow at high resolutions.

Compressed sensing in fluorescence microscopy

Compressed sensing (CS) is a signal processing approach that solves ill-posed inverse problems, from under-sampled data with respect to the Nyquist criterium. CS exploits sparsity constraints based on the knowledge of prior information, relative to the structure of the object in the spatial or other domains. It is commonly used in image and video compression as well as in scientific and medical applications, including computed tomography and magnetic resonance imaging. In the field of fluorescence microscopy, it has been demonstrated to be valuable for fast and high-resolution imaging, from single-molecule localization, super-resolution to light-sheet microscopy. Furthermore, CS has found remarkable applications in the field of mesoscopic imaging, facilitating the study of small animals' organs and entire organisms. This review article illustrates the working principles of CS, its implementations in optical imaging and discusses several relevant uses of CS in the field of fluorescence imaging from super-resolution microscopy to mesoscopy.

Single-pixel imaging: An overview of different methods to be used for 3D space reconstruction in harsh environments

Different imaging solutions have been proposed over the last few decades, aimed at three-dimensional (3D) space reconstruction and obstacle detection, either based on stereo-vision principles using active pixel sensors operating in the visible part of the spectra or based on active Near Infra-Red (NIR) illumination applying the time-of-flight principle, to mention just a few. If extremely low quantum efficiencies for NIR active illumination yielded by silicon-based detector solutions are considered together with the huge photon noise levels produced by the background illumination accompanied by Rayleigh scattering effects taking place in outdoor applications, the operating limitations of these systems under harsh weather conditions, especially if relatively low-power active illumination is used, are evident. If longer wavelengths for active illumination are applied to overcome these issues, indium gallium arsenide (InGaAs)-based photodetectors become the technology of choice, and for low-cost solutions, using a single InGaAs photodetector or an InGaAs line-sensor becomes a promising choice. In this case, the principles of Single-Pixel Imaging (SPI) and compressive sensing acquire a paramount importance. Thus, in this paper, we review and compare the different SPI developments reported. We cover a variety of SPI system architectures, modulation methods, pattern generation and reconstruction algorithms, embedded system approaches, and 2D/3D image reconstruction methods. In addition, we introduce a Near Infra-Red Single-Pixel Imaging (NIR-SPI) sensor aimed at detecting static and dynamic objects under outdoor conditions for unmanned aerial vehicle applications.

Differential real-time single-pixel imaging with Fourier domain regularization: applications to VIS-IR imaging and polarization imaging

The speed and quality of single-pixel imaging (SPI) are fundamentally limited by image modulation frequency and by the levels of optical noise and compression noise. In an approach to come close to these limits, we introduce a SPI technique, which is inherently differential, and comprises a novel way of measuring the zeroth spatial frequency of images and makes use of varied thresholding of sampling patterns. With the proposed sampling, the entropy of the detection signal is increased in comparison to standard SPI protocols. Image reconstruction is obtained with a single matrix-vector product so the cost of the reconstruction method scales proportionally with the number of measured samples. A differential operator is included in the reconstruction and following the method is based on finding the generalized inversion of the modified measurement matrix with regularization in the Fourier domain. We demonstrate 256 × 256 SPI at up to 17 Hz at visible and near-infrared wavelength ranges using 2 polarization or spectral channels. A low bit-resolution data acquisition device with alternating-current-coupling can be used in the measurement indicating that the proposed method combines improved noise robustness with a differential removal of the direct current component of the signal.

Synchronization-free top-down illumination photometric stereo imaging using light-emitting diodes and a mobile device

Three dimensional reconstruction of objects using a top-down illumination photometric stereo imaging setup and a hand-held mobile phone device is demonstrated. By employing binary encoded modulation of white light-emitting diodes for scene illumination, this method is compatible with standard lighting infrastructure and can be operated without the need for temporal synchronization of the light sources and camera. The three dimensional reconstruction is robust to unmodulated background light. An error of 2.69 mm is reported for an object imaged at a distance of 42 cm and with the dimensions of 48 mm. We also demonstrate the three dimensional reconstruction of a moving object with an effective off-line reconstruction rate of 25 fps.

Camera-free three-dimensional dual photography

We report camera-free three-dimensional (3D) dual photography. Inspired by the linkage between fringe projection profilometry (FPP) and dual photography, we propose to implement coordinate mapping to simultaneously sense the direct component of the light transport matrix and the surface profiles of 3D objects. By exploiting Helmholtz reciprocity, dual photography and scene relighting can thus be performed on 3D images. To verify the proposed imaging method, we have developed a single-pixel imaging system based on two digital micromirror devices (DMDs). Binary cyclic S-matrix patterns and binary sinusoidal fringe patterns are loaded on each DMD for scene encoding and virtual fringe projection, respectively. Using this system, we have demonstrated viewing and relighting 3D images at user-selectable perspectives. Our work extends the conceptual scope and the imaging capability of dual photography.

Image-free real-time 3-D tracking of a fast-moving object using dual-pixel detection

Real-time 3-D tracking of a fast-moving object has found important applications in industry, traffic control, sports, biomedicine, defense, etc. However, it is difficult to adopt typical image-based object tracking systems in a fast-moving object tracking in real time and for a long duration, because reliable and robust image processing and analysis algorithms are often computationally exhausted, and limited storage and bandwidth can hardly fulfill the great demand of high-speed photography. Here we report an image-free 3-D tracking approach. The approach uses only two single-pixel detectors and a high-speed spatial light modulator for data acquisition. By illuminating the target moving object with six single-period Fourier basis patterns, the approach is able to analytically calculate the position of the object with the corresponding single-pixel measurements. The approach is low-cost, and data- and computation-efficient. We experimentally demonstrate that the proposed approach can detect and track a fast-moving object at a frame rate of 1666 frames per second by using a 10,000 Hz digital micromirror device. Benefiting from the wide working spectrum of single-pixel detectors, the reported approach might be applicable for hidden fast-moving object tracking.

DeepGhost: real-time computational ghost imaging via deep learning

The potential of random pattern based computational ghost imaging (CGI) for real-time applications has been offset by its long image reconstruction time and inefficient reconstruction of complex diverse scenes. To overcome these problems, we propose a fast image reconstruction framework for CGI, called "DeepGhost", using deep convolutional autoencoder network to achieve real-time imaging at very low sampling rates (10-20%). By transferring prior-knowledge from STL-10 dataset to physical-data driven network, the proposed framework can reconstruct complex unseen targets with high accuracy. The experimental results show that the proposed method outperforms existing deep learning and state-of-the-art compressed sensing methods used for ghost imaging under similar conditions. The proposed method employs deep architecture with fast computation, and tackles the shortcomings of existing schemes i.e., inappropriate architecture, training on limited data under controlled settings, and employing shallow network for fast computation.

Image-free classification of fast-moving objects using "learned" structured illumination and single-pixel detection

Object classification generally relies on image acquisition and subsequent analysis. Real-time classification of fast-moving objects is a challenging task. Here we propose an approach for real-time classification of fast-moving objects without image acquisition. The key to the approach is to use structured illumination and single-pixel detection to acquire the object features directly. A convolutional neural network (CNN) is trained to learn the object features. The "learned" object features are then used as structured patterns for structured illumination. Object classification can be achieved by picking up the resulting light signals by a single-pixel detector and feeding the single-pixel measurements to the trained CNN. In our experiments, we show that accurate and real-time classification of fast-moving objects can be achieved. Potential applications of the proposed approach include rapid classification of flowing cells, assembly-line inspection, and aircraft classification in defense applications. Benefiting from the use of a single-pixel detector, the approach might be applicable for hidden moving object classification.

Non-Lambertian Photometric Stereo Network based on Inverse Reflectance Model with Collocated Light

Current non-Lambertian photometric stereo methods generally require a large number of images to ensure accurate surface normal estimation. To achieve accurate surface normal recovery under a sparse set of lights, this paper proposes a non-Lambertian photometric stereo network based on a derived inverse reflectance model with collocated light. The model is deduced using monotonicity of isotropic reflectance and the univariate property of collocated light to decouple the surface normal from the reflectance function. Thus, the surface normal can be estimated by three steps, i.e., model fitting, shadow rejection, and normal estimation. We leverage a supervised deep learning technique to enhance the shadow rejection ability and the flexibility of the inverse reflectance model. Shadows are handled through max-pooling. Information from a neighborhood image patch is utilized to improve the flexibility to various reflectances. Experiments using both synthetic and real images demonstrate that the proposed method achieves state-of-the-art accuracy in surface normal estimation.

Visual cryptography in single-pixel imaging

Two novel visual cryptography (VC) schemes are proposed by combining VC with single-pixel imaging (SPI) for the first time. It is pointed out that the overlapping of visual key images in VC is similar to the superposition of pixel intensities by a single-pixel detector in SPI. In the first scheme, QR-code VC is designed by using opaque sheets instead of transparent sheets. The secret image can be recovered when identical illumination patterns are projected onto multiple visual key images and a single detector is used to record the total light intensities. In the second scheme, the secret image is shared by multiple illumination pattern sequences and it can be recovered when the visual key patterns are projected onto identical items. The application of VC can be extended to more diversified scenarios by our proposed schemes.

Single-Pixel MEMS Imaging Systems

Single-pixel imaging technology is an attractive technology considering the increasing demand of imagers that can operate in wavelengths where traditional cameras have limited efficiency. Meanwhile, the miniaturization of imaging systems is also desired to build affordable and portable devices for field applications. Therefore, single-pixel imaging systems based on microelectromechanical systems (MEMS) is an effective solution to develop truly miniaturized imagers, owing to their ability to integrate multiple functionalities within a small device. MEMS-based single-pixel imaging systems have mainly been explored in two research directions, namely the encoding-based approach and the scanning-based approach. The scanning method utilizes a variety of MEMS scanners to scan the target scenery and has potential applications in the biological imaging field. The encoding-based system typically employs MEMS modulators and a single-pixel detector to encode the light intensities of the scenery, and the images are constructed by harvesting the power of computational technology. This has the capability to capture non-visible images and 3D images. Thus, this review discusses the two approaches in detail, and their applications are also reviewed to evaluate the efficiency and advantages in various fields.

Image-free real-time detection and tracking of fast moving object using a single-pixel detector

Real-time detection and tracking for fast moving object has important applications in various fields. However, available methods, especially low-cost ones, can hardly achieve real-time and long-duration object detection and tracking. Here we report an image-free and cost-effective method for detecting and tracking a fast moving object in real time and for long duration. The method employs a spatial light modulator and a single-pixel detector for data acquisition. It uses Fourier basis patterns to illuminate the target moving object and collects the resulting light signal with a single-pixel detector. The proposed method is able to detect and track the object with the single-pixel measurements directly without image reconstruction. The detection and tracking algorithm of the proposed method is computationally efficient. We experimentally demonstrate that the method can achieve a temporal resolution of 1,666 frames per second by using a 10,000 Hz digital micro-mirror device. The latency time of the method is on the order of microseconds. Additionally, the method acquires only 600 bytes of data for each frame. The method therefore allows fast moving object detection and tracking in real time and for long duration. This image-free approach might open up a new avenue for spatial information acquisition in a highly efficient manner.

Adaptive compressed 3D ghost imaging based on the variation of surface normals

Three-dimensional (3D) imaging can be reconstructed by a computational ghost imaging system with single pixel detectors based on a photometric stereo, but the requirement of large measurements and long imaging times are obstacles to its development. Also, the compressibility of the target's surface normals has not been fully studied, which causes the waste in sampling efficiency in single-pixel imaging. In this paper, we propose a method to adaptively measure the object's 3D information based on surface normals. In the proposed method, the regions of object's surface are illuminated by patterns of different spatial resolutions according to the variation of surface normals. The experimental results demonstrate that our proposed scheme can reduce measurements and preserve the quality of the formed 3D image.

Motion estimation and quality enhancement for a single image in dynamic single-pixel imaging

In single-pixel imaging (SPI), a large number of illuminations is usually required to capture one single image. Consequently, SPI may only achieve a very low frame rate for a fast-moving object and the reconstructed images are contaminated with blur and noise. In previous works, some attempts are made to perform motion estimation between neighboring frames in a SPI video to enhance the image quality. However, the motion estimation and quality enhancement from one single image frame in dynamic SPI was seldom investigated. In this work, it assumed that some prior knowledge about the type of motion the object undergoes is known. A motion model of the target object is constructed and the motion parameters can be optimized within a search space. Our proposed scheme is different from common motion deblur techniques for photographs since the motion blur mechanism in SPI is significantly different from a conventional camera. Experimental results demonstrate that the reconstructed images with our proposed scheme in dynamic SPI have much better quality.

Single-Pixel Imaging and Its Application in Three-Dimensional Reconstruction: A Brief Review

Whereas modern digital cameras use a pixelated detector array to capture images, single-pixel imaging reconstructs images by sampling a scene with a series of masks and associating the knowledge of these masks with the corresponding intensity measured with a single-pixel detector. Though not performing as well as digital cameras in conventional visible imaging, single-pixel imaging has been demonstrated to be advantageous in unconventional applications, such as multi-wavelength imaging, terahertz imaging, X-ray imaging, and three-dimensional imaging. The developments and working principles of single-pixel imaging are reviewed, a mathematical interpretation is given, and the key elements are analyzed. The research works of three-dimensional single-pixel imaging and their potential applications are further reviewed and discussed.

Improving the noise immunity of 3D computational ghost imaging

Computational ghost imaging (CGI) can build a three-dimensional (3D) image using the reconstructed shading images. However, it could be easily affected by the noise accumulated in the 3D reconstruction. More importantly, the selection of initial growing position will also affect the quality of the formed 3D image significantly. In this paper, we apply the technique of sub-pixel displacement to achieve smooth shading images in noisy environments and propose a method for selecting the optimal initial growing position to preserve the stereo feature of the object. We demonstrate that the surfaces of the reconstructed 3D images are more accurate using our proposed method as compared to the ones achieved by previously used methods. Our research would promote the development of 3D imaging using CGI in noisy environments.

Lensless single-pixel imaging by using LCD: application to small-size and multi-functional scanner

Single-pixel imaging commonly uses a spatial light modulator (SLM) to encode a scene's spatial information into a one-dimensional light signal sequence so that a single-pixel detector can be used to capture a scene. Digital micromirror device, liquid crystal on silicon, or light emitted diode matrix is a common choice of SLM, but it requires a certain lens system in order to project the structured light pattern that is generated by the SLM onto the scene. Using a lens would not only lead to aberration but also result in difficulty for establishing a compact imaging system. Therefore, we propose to use a liquid crystal display (LCD) as an intensity-only SLM to conduct structured illumination. As such, single-pixel imaging can be performed in a lensless way. As an instance of the proposed technique, a small-size and multi-functional scanner is designed and established to prove the lensless single-pixel imaging concept. As experimentally demonstrated, the single-pixel scanner can not only achieve grayscale and true-color scanning as a typical scanner does, but also achieve distinctive functionalities, such as accurate optical character recognition from under-sampled data, on-the-fly encryption, and genuine document identification. This compact scanner is as thin as 2.48 millimeters. The proposed lensless single-pixel imaging technique might find applications in various fields.