A residual-based deep learning approach for ghost imaging

Extendible ghost imaging with high reconstruction quality in strong scattering medium

Ghost imaging (GI) possesses significant application prospects in scattering imaging, which is a classic example of underdetermined conversion problem in optical field. However, even under the framework of single-pixel imaging (SPI), a challenge remains unresolved, i.e., structured patterns may be damaged by scattering media in both the emissive and receiving optical paths. In this study, an extendible ghost imaging, a numerical reproduction of the qualitative process using deep learning (DL)-based GI is presented. First, we propose and experimentally verify a brief degradation-guided reconstruction (DR) approach with a neural network to demonstrate the degradation principle of scattering, including realistic dataset simulations and a new training structure in the form of a convolutional neural network (CNN). Then, a novel photon contribution model (PCM) with redundant parameters is proposed to generate intensity sequences from the forward direction through volumetric scattering media; the redundant parameters are constructed and relate to the special output configuration in a lightweight CNN with two branches, based on a reformulated atmospheric scattering model. The proposed scheme recovers the semantics of targets and suppresses the imaging noise in the strong scattering medium, and the obtained results are very satisfactory for applications to scattering media of more practical scenarios and are available for various scattering coefficients and work distances of an imaging prototype. After using DL methods in computational imaging, we conclude that strategies embedded in optics or broader physical factors can result in solutions with better effects for unanalyzable processes.

Semantic ghost imaging based on recurrent-neural-network

Ghost imaging (GI) illuminates an object with a sequence of light patterns and obtains the corresponding total echo intensities with a bucket detector. The correlation between the patterns and the bucket signals results in the image. Due to such a mechanism different from the traditional imaging methods, GI has received extensive attention during the past two decades. However, this mechanism also makes GI suffer from slow imaging speed and poor imaging quality. In previous work, each sample, including an illumination pattern and its detected bucket signal, was treated independently with each other. The correlation is therefore a linear superposition of the sequential data. Inspired by human's speech, where sequential words are linked with each other by a certain semantic logic and an incomplete sentence could still convey a correct meaning, we here propose a different perspective that there is potentially a non-linear connection between the sequential samples in GI. We therefore built a system based on a recurrent neural network (RNN), called GI-RNN, which enables recovering high-quality images at low sampling rates. The test with MNIST's handwriting numbers shows that, under a sampling rate of 1.28%, GI-RNN have a 12.58 dB higher than the traditional basic correlation algorithm and a 6.61 dB higher than compressed sensing algorithm in image quality. After trained with natural images, GI-RNN exhibits a strong generalization ability. Not only does GI-RNN work well with the standard images such as "cameraman", but also it can recover the natural scenes in reality at the 3% sampling rate while the SSIMs are greater than 0.7.

Color ghost imaging through the scattering media based on A-cGAN

Ghost imaging plays an important role in the field of optical imaging. To realize color ghost imaging through the scattering media, we propose a deep learning method with high generation ability. Through our method, we can efficiently reconstruct color images with rich details, in line with human perception and close to the target color pictures. Experimental results show that our method can image through the scattering media with different scattering intensities and achieve good results even at a sampling rate of 0.1.

Single-pixel neural network object classification of sub-Nyquist ghost imaging

A single-pixel neural network object classification scenario in the sub-Nyquist ghost imaging system is proposed. Based on the neural network, objects are classified directly by bucket measurements without reconstructing images. Classification accuracy can still be maintained at 94.23% even with only 16 measurements (less than the Nyquist limit of 1.56%). A parallel computing scheme is applied in data processing to reduce the object acquisition time significantly. Random patterns are used as illumination patterns to illuminate objects. The proposed method performs much better than existing methods for both binary and grayscale images in the sub-Nyquist condition, which is also robust to environment noise turbulence. Benefiting from advantages of ghost imaging, it may find applications for target recognition in the fields of remote sensing, military defense, and so on.

SP-ILC: Concurrent Single-Pixel Imaging, Object Location, and Classification by Deep Learning

We propose a concurrent single-pixel imaging, object location, and classification scheme based on deep learning (SP-ILC). We used multitask learning, developed a new loss function, and created a dataset suitable for this project. The dataset consists of scenes that contain different numbers of possibly overlapping objects of various sizes. The results we obtained show that SP-ILC runs concurrent processes to locate objects in a scene with a high degree of precision in order to produce high quality single-pixel images of the objects, and to accurately classify objects, all with a low sampling rate. SP-ILC has potential for effective use in remote sensing, medical diagnosis and treatment, security, and autonomous vehicle control.

Ghost imaging based on asymmetric learning

Ghost imaging (GI) is an unconventional optical imaging method making use of the correlation measurement between a test beam and a reference beam. GI using deep learning (GIDL) has earned increasing attention, as it can reconstruct images of high quality more effectively than traditional GI methods. It has been demonstrated that GIDL can be trained completely with simulation data, which makes it even more practical. However, most GIDLs proposed so far appear to have limited performance for random noise distributed patterns. This is because traditional GIDLs are sensitive to the under-estimation error but robust to the over-estimation error. An asymmetric learning framework is proposed here to tackle the unbalanced sensitivity to estimation errors of GIDL. The experimental results show that it can achieve much better reconstructed images than GIDL with a symmetric loss function, and the structural similarity index of GI is quadrupled for randomly selected objects.