Distortion-tolerant three-dimensional object recognition with digital holography

Self-adjustment of spectral components to object wave field extraction in off-axis digital holography

In this paper, we introduce a positioning system to self-adjust the spectral components of holograms recorded by an off-axis digital holography setup. The incidence angle of the reference arm is carried out by two motorized actuators controlled by an algorithm that automatically adjusts it to avoid the overlapping of the spectral components. The right positioning of the spectral components allows selection of a spectral region to extract only the virtual component so that, after an inverse Fourier transformation, the object wave field can be obtained, thus eliminating the undesired components and increasing the image quality of the reconstructed image.

Spatially multiplexed interferometric microscopy with partially coherent illumination

We have recently reported on a simple, low cost, and highly stable way to convert a standard microscope into a holographic one [Opt. Express 22, 14929 (2014)]. The method, named spatially multiplexed interferometric microscopy (SMIM), proposes an off-axis holographic architecture implemented onto a regular (nonholographic) microscope with minimum modifications: the use of coherent illumination and a properly placed and selected one-dimensional diffraction grating. In this contribution, we report on the implementation of partially (temporally reduced) coherent illumination in SMIM as a way to improve quantitative phase imaging. The use of low coherence sources forces the application of phase shifting algorithm instead of off-axis holographic recording to recover the sample’s phase information but improves phase reconstruction due to coherence noise reduction. In addition, a less restrictive field of view limitation (1/2) is implemented in comparison with our previously reported scheme (1/3). The proposed modification is experimentally validated in a regular Olympus BX-60 upright microscope considering a wide range of samples (resolution test, microbeads, swine sperm cells, red blood cells, and prostate cancer cells).

Review of holographic-based three-dimensional object recognition techniques [invited]

With the advancement of computing and optical technologies, it is now possible to capture digital holograms of real-life object scenes. Theoretically, through the analysis of a hologram, the three-dimensional (3D) objects coded on the hologram can be identified. However, being different from an optical image, a hologram is composed of complicated fringes that cannot be analyzed easily with traditional computer vision methods. Over the years, numerous important research investigations have been attempted to provide viable solutions to this problem. The aim of this work is three-fold. First, we provide a quick walkthrough on the overall development of holographic-based 3D object recognition (H3DOR) in the past five decades, from film-based approaches to digital-based innovation. Second, we describe in more detail a number of selected H3DOR methods that are introduced at different timelines, starting from the late sixties and then from the seventies, where viable digital holographic-based 3D recognition methods began to emerge. Finally, we present our work on digital holographic, pose-invariant 3D object recognition that is based on a recently introduced virtual diffraction plane framework. As our method has not been reported elsewhere, we have included some experimental results to demonstrate the feasibility of the approach.

Method to calculate the far field of three-dimensional objects for computer-generated holography

Here, a new method for calculating the computer-generated holograms of three-dimensional (3D) objects is presented along with a review of current techniques. The method, the planar layers method (PLM), is established on the idea of representing 3D objects in discrete planar layers perpendicular to the observation plane, then calculating the total far field pattern by summing up the far field patterns of each layer. Simulation results, computational complexity, and error comparisons reveal that this new method can be used to calculate far field patterns--hence, the holograms--of computer-synthesized objects very efficiently.

Computational integral-imaging reconstruction-based 3-D volumetric target object recognition by using a 3-D reference object

In this paper, we propose a novel computational integral-imaging reconstruction (CIIR)-based three-dimensional (3-D) image correlator system for the recognition of 3-D volumetric objects by employing a 3-D reference object. That is, a number of plane object images (POIs) computationally reconstructed from the 3-D reference object are used for the 3-D volumetric target recognition. In other words, simultaneous 3-D image correlations between two sets of target and reference POIs, which are depth-dependently reconstructed by using the CIIR method, are performed for effective recognition of 3-D volumetric objects in the proposed system. Successful experiments with this CIIR-based 3-D image correlator confirmed the feasibility of the proposed method.

Single-shot digital holography by use of the fractional Talbot effect

We present a method for recording in-line single-shot digital holograms based on the fractional Talbot effect. In our system, an image sensor records the interference between the light field scattered by the object and a properly codified parallel reference beam. A simple binary two-dimensional periodic grating is used to codify the reference beam generating a periodic three-step phase distribution over the sensor plane by fractional Talbot effect. This provides a method to perform single-shot phase-shifting interferometry at frame rates only limited by the sensor capabilities. Our technique is well adapted for dynamic wavefront sensing applications. Images of the object are digitally reconstructed from the digital hologram. Both computer simulations and experimental results are presented.

Parallel two-step phase-shifting digital holography

We propose a parallel two-step phase-shifting digital holography technique capable of instantaneous measurement of three-dimensional objects, with a view toward measurement of dynamically moving objects. The technique is based on phase-shifting interferometry. The proposed technique carries out the two-step phase-shifting method at one time and can be optically implemented by using a phase-shifting array device located in the reference beam. The array device has a periodic two-step phase distribution, and its configuration is simplified compared with that required for three-step and four-step parallel phase-shifting digital holographies. Therefore the optical system of the proposed technique is more suitable for the realization of a parallel phase-shifting digital holography system. We conduct both a numerical simulation and a preliminary experiment in the proposed technique. The results of the simulation and the experiment agree well with those of sequential phase-shifting digital holography, and results are superior to those obtained by conventional digital holography using the Fresnel transform alone. Thus the effectiveness of the proposed technique is verified.

Three-dimensional color object visualization and recognition using multi-wavelength computational holography

In this paper, we address 3D object visualization and recognition with multi-wavelength digital holography. Color features of 3D objects are obtained by the multiple-wavelengths. Perfect superimposition technique generates reconstructed images of the same size. Statistical pattern recognition techniques: principal component analysis and mixture discriminant analysis analyze multi-spectral information in the reconstructed images. Class-conditional probability density functions are estimated during the training process. Maximum likelihood decision rule categorizes unlabeled images into one of trained-classes. It is shown that a small number of training images is sufficient for the color object classification.

Histogram approaches for lossy compression of digital holograms of three-dimensional objects

We present a novel nonuniform quantization compression technique-histogram quantization-for digital holograms of 3-D real-world objects. We exploit a priori knowledge of the distribution of the values in our data. We compare this technique to another histogram based approach: a modified version of Max's algorithm that has been adapted in a straight-forward manner to complex-valued 2-D signals. We conclude the compression procedure by applying lossless techniques to our quantized data. We demonstrate improvements over previous results obtained by applying uniform and nonuniform quantization techniques to the hologram data.

Determination of large-scale out-of-plane displacements in digital Fourier holography

A novel approach for the determination of large-scale out-of-plane displacements from digital Fourier holograms is presented. The proposed method is invariant to lateral object shifts. It is based on the determination of the scaling of the reconstructed image that occurs when the recording distance is changed. For a precise determination of the scaling factor, we utilize the Mellin transform. After the discussion of mathematical and computational issues, experimental results are presented to verify the theoretical considerations. The results show that displacements of at least up to 8.4% from the initial recording distance can be detected with this approach. The displacements could be determined with a deviation of typically less than 1.0% from the set values.

Reduction of speckle in digital holography by discrete Fourier filtering

We present a digital signal processing technique that reduces the speckle content in reconstructed digital holograms. The method is based on sequential sampling of the discrete Fourier transform of the reconstructed image field. Speckle reduction is achieved at the expense of a reduced intensity and resolution, but this trade-off is shown to be greatly superior to that imposed by the traditional mean and median filtering techniques. In particular, we show that the speckle can be reduced by half with no loss of resolution (according to standard definitions of both metrics).

Theoretical analysis of three-dimensional imaging and recognition of micro-organisms with a single-exposure on-line holographic microscope

Single-exposure on-line (SEOL) digital holography is a recently proposed technique for monitoring, visualization, and recognition of three-dimensional (3D) objects. In contrast to traditional multi-exposure on-line digital holography, it uses only one exposure, which makes it particularly suitable for imaging and recognizing moving micro-organisms. However, the cost of using only one exposure is the superposition of a conjugate image on the desired reconstructed image. The influence of the conjugate image on the visualization and recognition performance is investigated. The conditions for which the cross-talk noise induced by the conjugate image is negligible are derived. It is demonstrated that with conditions common in imaging of microscopic 3D biological objects, SEOL digital holography is highly tolerant of cross-talk noise induced by the conjugate image.

Recognition and classification of three-dimensional phase objects by digital Fresnel holography

We demonstrate the validity of wavelet-based processing for recognition and classification of three-dimensional phase objects. One Fresnel digital hologram of each of the three-dimensional (3-D) phase objects to be classified is recorded. The electronic holograms are processed digitally to permit 3-D object information to be retrieved as two-dimensional digital complex images. We use a Mexican-hat wavelet- matched filter (WMF) to enhance the correlation peak and discriminate between the objects. The WMF performs a wavelet transform (WT) to enhance the significant features of the images and the correlation of the WT coefficients thus obtained. We compare the feasibility of a WMF-based object classifier with the matched-filter-based classifier to classify our four 3-D phase objects in a 3-D scene into true or false classes with minimal error.

Improved-resolution digital holography using the generalized sampling theorem for locally band-limited fields

We describe the recording conditions that, together with the appropriate numerical reconstruction process, permit high-lateral-resolution reconstruction of in-line digital holograms. By high resolution, we mean a resolution that is beyond the Nyquist frequency, which is achieved by common methods. The proposed method is based on a previously reported generalized sampling theory that presents the conditions to precisely reconstruct fields that in certain cases may be sampled with a sampling rate lower than the Nyquist rate. We examine the hologram-recording process in the Wigner space. On the basis of this analysis, we demonstrate a simple high-resolution numerical reconstruction method.

Reconstruction of partially occluded objects encoded in three-dimensional scenes by using digital holograms

We propose a task-specific digital holographic capture system for three-dimensional scenes, which can reduce the amount of data sent from the camera system to the receiver and can effectively reconstruct partially occluded objects. The system requires knowledge of the object of interest, but it does not require a priori knowledge of either the occlusion or the distance the object is from the camera. Subwindows of the camera-plane Fresnel field are digitally propagated to reveal different perspectives of the scene, and these are combined to overcome the unknown foreground occlusions. The nature of the occlusions and the effect of subwindows are analyzed thoroughly by using the Wigner distribution function. We demonstrate that a careful combination of reconstructions from subwindows can reveal features that are not apparent in a reconstruction from the whole hologram. We provide results by using optically captured digital holograms of real-world objects and simulated occlusions.

Parallel three-step phase-shifting digital holography

We propose parallel three-step phase-shifting digital holography as a technique capable of noiseless instantaneous measurement of three-dimensional objects based on phase-shifting interferometry. The proposed digital holography carries out three-step phase shifting at the same time by using a phase-shifting array device located in the reference beam. The array device has a periodic three-step phase distribution, and its configuration is simplified compared with that required for conventional parallel phase-shifting digital holography. Therefore the optical system of the proposed parallel phase-shifting digital holography is more suitable for the realization of the proposed holography. We conduct both a numerical simulation and a preliminary experiment. The results of the simulation and experiment agree well with those of the conventional phase-shifting method and are superior to the results obtained by conventional digital holography by using the Fresnel transform alone. Thus the effectiveness of the proposed technique is verified.

Scheme to improve the reconstructed image in parallel quasi-phase-shifting digital holography

We propose a scheme to improve the reconstructed image in parallel quasi-phase-shifting digital holography. Parallel quasi-phase-shifting digital holography is a technique capable of noiseless instantaneous measurement of three-dimensional objects, and it implements four kinds of phase shifting at a time with an array of 2 x 2 phase-shifting devices located in the reference wave. In the phase-shifting calculation in the reconstruction process of the technique, the scheme assigns the 2 x 2 cell configuration for each pixel in the vertical direction and for each 1-pixel interval in the horizontal direction of the hologram recorded by the image sensor. We conduct both a numerical simulation and a preliminary experiment. The results show that the proposed scheme can improve the quality of the reconstructed image calculated by the conventional scheme of parallel quasi-phase-shifting digital holography we previously proposed, and then the effectiveness of the proposed scheme is verified.

Speckle removal using a maximum-likelihood technique with isoline gray-level regularization

We propose a method based on the maximum-likelihood technique for removing speckle patterns that plague coherent images. The proposed method is designed for images whose gray levels vary continuously in space. The image model is based on a lattice of nodes corresponding to vertices of triangles in which the gray level of each pixel is produced by linear interpolation. A constraint on isoline gray levels is introduced to regularize the solution.

Comparison of passive ranging integral imaging and active imaging digital holography for three-dimensional object recognition

We present an overview of three-dimensional (3D) object recognition techniques that use active sensing by interferometric imaging (digital holography) and passive sensing by integral imaging. We describe how each technique can be used to retrieve the depth information of a 3D scene and how this information can then be used for 3D object recognition. We explore various algorithms for 3D recognition such as nonlinear correlation and target distortion tolerance. We also provide a comparison of the advantages and disadvantages of the two techniques.

Three-dimensional object feature extraction and classification with computational holographic imaging

We address three-dimensional (3D) object classification with computational holographic imaging. A 3D object can be reconstructed at different planes by use of a single hologram. We apply principal component and Fisher linear discriminant analyses based on Gabor-wavelet feature vectors to classify 3D objects measured by digital interferometry. Experimental and simulation results are presented for regional filtering concentrated at specific positions and for overall grid filtering. The proposed technique substantially reduces the dimensionality of the 3D classification problem. To the best of our knowledge, this is the first report on the use of the proposed technique for 3D object classification.

Efficient compression of fresnel fields for Internet transmission of three-dimensional images

We compress phase-shift digital holograms (whole Fresnel fields) for the transmission of three-dimensional images. For real-time networking applications, the time required to compress can be as critical as the compression rate. We achieve lossy compression through quantization of both the real and imaginary streams, followed by a bit packing operation. Compression losses in the reconstructed objects were quantified. We define a speedup metric that combines space gains due to compression with temporal overheads due to the compression routine and the transmission serialization. We empirically verify transmission speedup due to compression using a special-purpose Internet-based networking application.

Real-time three-dimensional object reconstruction by use of a phase-encoded digital hologram

A three-dimensional (3D) object reconstruction technique that uses only phase information of a phase-shifting digital hologram and a phase-only spatial-light modulator is proposed. It is well known that a digital hologram can store both amplitude and phase information of an optical electric field and can reconstruct the original 3D object in a computer. We demonstrate that it is possible to reconstruct optically 3D objects using only phase information of the optical field calculated from phase-shifting digital holograms. The use of phase-only information enables us to reduce the amount of data in the digital hologram and reconstruct optically the 3D objects using a liquid-crystal spatial light modulator without optical power loss. Numerical evaluation of the reconstructed 3D object is presented.

Digital three-dimensional image correlation by use of computer-reconstructed integral imaging

We use integral images of a three-dimensional (3D) scene to estimate the longitudinal depth of multiple objects present in the scene. With this information, we digitally reconstruct the objects in three dimensions and compute 3D correlations of input objects. We investigate the use of nonlinear techniques for 3D correlations. We present experimental results for 3D reconstruction and correlation of 3D objects. We demonstrate that it is possible to perform 3D segmentation of 3D objects in a scene. We finally present experiments to demonstrate that the 3D correlation is more discriminant than the two-dimensional correlation.

Object recognition using three-dimensional optical quasi-correlation

A novel method of three-dimensional (3-D) object recognition is proposed. Several projections of a 3-D target are recorded under white-light illumination and fused into a single complex two-dimensional function. After proper filtering, the resulting function is coded into a computer-generated hologram. When this hologram is coherently illuminated, a correlation space is reconstructed such that light peaks indicate the existence and locations of true targets in the observed 3-D scene. Experimental results and comparisons with results of another 3-D object recognition technique are presented.

Compression of digital holograms for three-dimensional object reconstruction and recognition

We present the results of applying lossless and lossy data compression to a three-dimensional object reconstruction and recognition technique based on phase-shift digital holography. We find that the best lossless (Lempel-Ziv, Lempel-Ziv-Welch, Huffman, Burrows-Wheeler) compression rates can be expected when the digital hologram is stored in an intermediate coding of separate data streams for real and imaginary components. The lossy techniques are based on subsampling, quantization, and discrete Fourier transformation. For various degrees of speckle reduction, we quantify the number of Fourier coefficients that can be removed from the hologram domain, and the lowest level of quantization achievable, without incurring significant loss in correlation performance or significant error in the reconstructed object domain.

Pattern recognition with a digital holographic microscope working in partially coherent illumination

We describe the implementation of the automatic spatial-frequency-selection filter for recognition of patterns obtained with a digital holographic microscope working with a partially coherent source. The microscope provides the complex-optical-amplitude field that allows a refocusing plane-by-plane of the sample under investigation by numerical computation of the optical propagation. By inserting a correlation filter in the propagation equation, the correlation between the filter and the propagated optical field is obtained. In this way, the pattern is located in the direction of the optical axis. Owing to the very weak noise level generated by the partially coherent source, the correlation process is shift invariant. Therefore the samples can be located in the three dimensions. To have a robust recognition process, a generalized version of the automatic spatial-frequency-selection filters has been implemented. The method is experimentally demonstrated in a two-class problem for the recognition of protein crystals.

Neural network for three-dimensional object recognition based on digital holography

We present a two-layer neural network for processing of three-dimensional (3D) images that are obtained by digital holography. The network is trained with a real 3D object to compute the weights of the layers. Experiments are presented to illustrate the system performance. The system is designed to detect a 3D object in the presence of various distortions. As an example, experiments are presented to illustrate how the system is able to recognize a 3D object with 360 degrees out-of-plane rotation.