Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling

3D photon counting integral imaging by using multi-level decomposition

In this paper, we propose three-dimensional (3D) photon counting integral imaging by using multi-level decomposition such as discrete wavelet transform to improve the visual quality and measurement accuracy under photon-starved conditions. Conventional 3D integral imaging can visualize 3D objects and acquire their depth information. However, the amount of irradiated light on the object causes the degradation of visual quality for 3D images under photon-starved conditions. To visualize 3D objects, photon counting integral imaging has been utilized. It can detect photons from 3D scenes by using a computational photon counting model, which is modelled by the Poisson random process. However, photons occur not only from objects but also in areas where objects do not exist. Moreover, photon fluctuation may occur in the scene through shot noise. Since these noise photons are measurement errors, it may decrease the image quality and accuracy. In contrast, our proposed method uses 2D discrete wavelet transform, which can emphasize the object photons effectively. Finally, our proposed method can enhance the visual quality of 3D images and provide more accurate depth information under photon-starved conditions. To prove the feasibility of our proposed method, we implement the optical experiment and calculate various image quality metrics.

A multispectral photon-counting double random phase encoding scheme for image authentication

In this paper, we propose a new method for color image-based authentication that combines multispectral photon-counting imaging (MPCI) and double random phase encoding (DRPE) schemes. The sparsely distributed information from MPCI and the stationary white noise signal from DRPE make intruder attacks difficult. In this authentication method, the original multispectral RGB color image is down-sampled into a Bayer image. The three types of color samples (red, green and blue color) in the Bayer image are encrypted with DRPE and the amplitude part of the resulting image is photon counted. The corresponding phase information that has nonzero amplitude after photon counting is then kept for decryption. Experimental results show that the retrieved images from the proposed method do not visually resemble their original counterparts. Nevertheless, the original color image can be efficiently verified with statistical nonlinear correlations. Our experimental results also show that different interpolation algorithms applied to Bayer images result in different verification effects for multispectral RGB color images.

Photon counting range-intensity image strategy in low-light level environments

We present a photon counting range-intensity image strategy based on a single-photon detector in low-light level environments. In this Letter, a composite modulation method over the pulse sequence was used for the first time, to the best of our knowledge, which combined pulse-position modulation and pulse-intensity modulation. This composite modulation method could obtain range and intensity of the detected target at the same time. Besides, angle-angle information could be provided from the scanner or detector array. Thus, a range-intensity image of the target became feasible. For demonstrating this photon counting range-intensity image strategy, a proof-of-principle laboratory system was established. In low-light level environments, a range-intensity image of multiple similar targets was obtained successfully with the range accuracy of centimeter level and intensity error of 1%. Compared with the range image, a range-intensity image could better reorganize and identify similar targets.

Advances in three-dimensional integral imaging: sensing, display, and applications [Invited]

Three-dimensional (3D) sensing and imaging technologies have been extensively researched for many applications in the fields of entertainment, medicine, robotics, manufacturing, industrial inspection, security, surveillance, and defense due to their diverse and significant benefits. Integral imaging is a passive multiperspective imaging technique, which records multiple two-dimensional images of a scene from different perspectives. Unlike holography, it can capture a scene such as outdoor events with incoherent or ambient light. Integral imaging can display a true 3D color image with full parallax and continuous viewing angles by incoherent light; thus it does not suffer from speckle degradation. Because of its unique properties, integral imaging has been revived over the past decade or so as a promising approach for massive 3D commercialization. A series of key articles on this topic have appeared in the OSA journals, including Applied Optics. Thus, it is fitting that this Commemorative Review presents an overview of literature on physical principles and applications of integral imaging. Several data capture configurations, reconstruction, and display methods are overviewed. In addition, applications including 3D underwater imaging, 3D imaging in photon-starved environments, 3D tracking of occluded objects, 3D optical microscopy, and 3D polarimetric imaging are reviewed.

Dual-camera enabled real-time three-dimensional integral imaging pick-up and display

A new real-time integral imaging pick-up and display method is demonstrated. This proposed method utilizes the dual-camera optical pick-up part to collect 3D information of real scene in real-time without pre-calibration. Elemental images are then provided by a computer-generated integral imaging part and displayed by a projection-type integral imaging display part. The theoretical analysis indicates the method is robust to the camera position deviation, which profits the real-time data processing. Experimental results show that the fully continuous, real 3D scene pick-up and display system is feasible with a throughput of 8 fps in real time. Further analysis predicts that the parallel optimization can be adopted by the proposed method for real-time 3D pick-up and display with a throughput of 25 fps.

Homogeneous light field model for interactive control of viewing parameters of integral imaging displays

A novel model for three dimensional (3D) interactive control of viewing parameters of integral imaging systems is established in this paper. Specifically, transformation matrices are derived in an extended homogeneous light field coordinate space based on interactive controllable requirement of integral imaging displays. In this model, new elemental images can be synthesized directly from the ones captured in the record process to display 3D images with expected viewing parameters, and no extra geometrical information of the 3D scene is required in the synthesis process. Computer simulation and optical experimental results show that the reconstructed 3D scenes with depth control, lateral translation and rotation can be achieved.

Depth extraction of three-dimensional objects using block matching for slice images in synthetic aperture integral imaging

We describe a computational method for depth extraction of three-dimensional (3D) objects using block matching for slice images in synthetic aperture integral imaging (SAII). SAII is capable of providing high-resolution 3D slice images for 3D objects because the picked-up elemental images are high-resolution ones. In the proposed method, the high-resolution elemental images are recorded by moving a camera; a computational reconstruction algorithm based on ray backprojection generates a set of 3D slice images from the recorded elemental images. To extract depth information of the 3D objects, we propose a new block-matching algorithm between a reference elemental image and a set of 3D slice images. The property of the slices images is that the focused areas are the right location for an object, whereas the blurred areas are considered to be empty space; thus, this can extract robust and accurate depth information of the 3D objects. To demonstrate our method, we carry out the preliminary experiments of 3D objects; the results indicate that our method is superior to a conventional method in terms of depth-map quality.

Three-dimensional photon counting integral imaging reconstruction using penalized maximum likelihood expectation maximization

Recent works have demonstrated that three-dimensional (3D) object reconstruction is possible from integral images captured in severely photon starved conditions. In this paper we propose an iterative approach to implement a maximum likelihood expectation maximization estimator with several types of regularization for 3D reconstruction from photon counting integral images. We show that the proposed algorithms outperform the previously reported approaches for photon counting 3D integral imaging reconstruction. To the best of our knowledge, this is the first report on using iterative statistical reconstruction techniques for 3D photon counting integral imaging.

Artifact analysis and image enhancement in three-dimensional computational integral imaging using smooth windowing technique

We propose an artifact analysis in computational integral imaging and the image enhancement method based on the analysis using the smooth windowing technique. Blurring and lenslet artifacts, which are major problems in computational integral imaging, are defined and analyzed using a signal model. Applying a smooth and continuous window such as the triangular window to computational integral imaging reconstruction provides a dramatic improvement in terms of image quality. Experimental results are presented to show the validity of our method. To our best knowledge, this is the first trial to control a window function in computational integral imaging.

Information theoretic approach for assessing image fidelity in photon-counting arrays

The method of photon-counting integral imaging has been introduced recently for three-dimensional object sensing, visualization, recognition and classification of scenes under photon-starved conditions. This paper presents an information-theoretic model for the photon-counting imaging (PCI) method, thereby providing a rigorous foundation for the merits of PCI in terms of image fidelity. This, in turn, can facilitate our understanding of the demonstrated success of photon-counting integral imaging in compressive imaging and classification. The mutual information between the source and photon-counted images is derived in a Markov random field setting and normalized by the source-image's entropy, yielding a fidelity metric that is between zero and unity, which respectively corresponds to complete loss of information and full preservation of information. Calculations suggest that the PCI fidelity metric increases with spatial correlation in source image, from which we infer that the PCI method is particularly effective for source images with high spatial correlation; the metric also increases with the reduction in photon-number uncertainty. As an application to the theory, an image-classification problem is considered showing a congruous relationship between the fidelity metric and classifier's performance.

Recent progress in three-dimensional information processing based on integral imaging

Recently developed integral imaging techniques are reviewed. Integral imaging captures and reproduces the light rays from the object space, enabling the acquisition and the display of the three-dimensional information of the object in an efficient way. Continuous effort on integral imaging has been improving the performance of the capture and display process in various aspects, including distortion, resolution, viewing angle, and depth range. Digital data processing of the captured light rays can now visualize the three-dimensional structure of the object with a high degree of freedom and enhanced quality. This recent progress is of high interest for both industrial applications and academic research.

Three dimensional imaging and recognition using truncated photon counting model and parametric maximum likelihood estimator

In this paper, a statistical approach is presented for three-dimensional (3D) visualization and recognition of objects having very small number of photons based on a parametric estimator. A truncated Poisson probability density function is assumed for modeling the distribution of small number of photons count observation. For 3D visualization and recognition of photon-limited objects, an integral imaging system is employed. We utilize virtual geometrical ray propagation for 3D reconstruction of objects. A maximum likelihood estimator (MLE) and statistical inference algorithms are applied to small number of photons counted elemental images captured with integral imaging. We have demonstrated that the MLE using a truncated Poisson model for estimating the average number of photon for each voxel of a photon starved 3D object has a small estimation error compared with the MLE using a Poisson model. Also, we present experiments to investigate the effect of 3D sensing parallax on the recognition performance under a fixed mean number of photons.