Depth extraction of three-dimensional objects using block matching for slice images in synthetic aperture integral imaging

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

This study aims to achieve accurate three-dimensional (3D) localization of multiple objects in a complicated scene using passive imaging. It is challenging, as it requires accurate localization of the objects in all three dimensions given recorded 2D images. An integral imaging system captures the scene from multiple angles and is able to computationally produce blur-based depth information about the objects in the scene. We propose a method to detect and segment objects in a 3D space using integral-imaging data obtained by a video camera array. Using objects' two-dimensional regions detected via deep learning, we employ local computational integral imaging in detected objects' depth tubes to estimate the depth positions of the objects along the viewing axis. This method analyzes object-based blurring characteristics in the 3D environment efficiently. Our camera array produces an array of multiple-view videos of the scene, called elemental videos. Thus, the proposed 3D object detection applied to the video frames allows for 3D tracking of the objects with knowledge of their depth positions along the video. Results show successful 3D object detection with depth localization in a real-life scene based on passive integral imaging. Such outcomes have not been obtained in previous studies using integral imaging; mainly, the proposed method outperforms them in its ability to detect the depth locations of objects that are in close proximity to each other, regardless of the object size. This study may contribute when robust 3D object localization is desired with passive imaging, but it requires a camera or lens array imaging apparatus.

Depth-of-field extension in integral imaging using multi-focus elemental images

In integral imaging, one of the main challenges is the limited depth of field (DOF), which is mainly caused by the short focal length of the microlenses. In this paper, we propose a method to extend the DOF of a synthetic aperture integral imaging (SAII) system by realizing the image fusion method on the multi-focus elemental images with different perspectives. In the proposed system, the contour-based object extraction method combined with size correction is developed to solve size inconsistency of the objects in the misaligned elemental images. The all-in-focus elemental images (EIs) combining selected features of multi-focus elemental images are then obtained by the block-based image fusion method. In the last step, the reconstructed images with the extended DOF can be generated based on the all-in-focus EIs in the SAII system. Experimental results are presented to demonstrate the feasibility of the proposed system.

Effects of elemental images' quantity on three-dimensional segmentation using computational integral imaging

3D object detection and isolation can be achieved algorithmically using computational integral-imaging data. The 3D scene is acquired by a multi-channel system, where each channel (elemental image) captures the scene from a shifted perspective angle. The number of these channels affects the weight, the cost, and the computational load of the segmentation process, while a lower number of channels may reduce the performance of the objects' separation in the 3D scene. This research examines the effect of the elemental images' quantity on the 3D object detection and segmentation, under both regular and noisy conditions. Moreover, based on our previous works, we perform an improvement of the 3D object segmentation quality using an adapted active-contour method.

Disparity pattern-based autostereoscopic 3D metrology system for in situ measurement of microstructured surfaces

This paper presents a disparity pattern-based autostereoscopic (DPA) 3D metrology system that makes use of a microlens array to capture raw 3D information of the measured surface in a single snapshot through a CCD camera. Hence, a 3D digital model of the target surface with the measuring data is generated through a system-associated direct extraction of disparity information (DEDI) method. The DEDI method is highly efficient for performing the direct 3D mapping of the target surface based on tomography-like operation upon every depth plane with the defocused information excluded. Precise measurement results are provided through an error-elimination process based on statistical analysis. Experimental results show that the proposed DPA 3D metrology system is capable of measuring 3D microstructured surfaces with submicrometer measuring repeatability for high precision and in situ measurement of microstructured surfaces.

Automatic 3D object localization and isolation using computational integral imaging

Detecting objects in three-dimensional (3D) space may be useful for various applications. We present a method to detect the 3D locations of objects using computationally reconstructed images obtained by integral imaging. The new algorithm exploits the space-variant blurring properties of the reconstructed images to detect and isolate objects at their depth locations, while removing traces of objects from other depths. With regard to previous work, the proposed method is more efficient and more resistant to noise; it gives more information about the detected object's depth, and improves object isolation and presentation.

Autostereoscopy-based three-dimensional on-machine measuring system for micro-structured surfaces

Traditional off-line measuring systems find it difficult to measure micro-structured workpieces which have a large volume and heavy weight, such as micro-structured patterned precision roller drums. This paper proposes an autostereoscopy-based three-dimensional (3D) measuring method and develops an innovative measuring system for the 3D on-machine measurement of the micro-structured surfaces, an Autostereoscopy-based Three-Dimensional On-machine Measuring (ATDOM) system. The ATDOM system is compact and capable of fast data acquisition and high accuracy in 3D computational reconstruction of complex surfaces under different measuring environments. A prototype ATDOM system is experimentally verified through a series of measurement experiments conducted on a precision machine tool. The results indicate that the ATDOM system provides an important means for efficient and reliable on-machine measurement of micro-structured surfaces.

Distortion-free wide-angle 3D imaging and visualization using off-axially distributed image sensing

We propose a new off-axially distributed image sensing (ODIS) using a wide-angle lens for reconstructing distortion-free wide-angle slice images computationally. In the proposed system, the wide-angle image sensor captures a wide-angle 3D scene, and thus the collected information of the 3D objects is severely distorted. To correct this distortion, we introduce a new correction process involving a wide-angle lens to the computational reconstruction in ODIS. This enables us to reconstruct distortion-free, wide-angle slice images for visualization of 3D objects. Experimental results are carried out to verify the proposed method. To the best of our knowledge, this is the first time the use of a wide-angle lens in a multiple-perspective 3D imaging system is described.

Dynamic 3D scene depth reconstruction via optical flow field rectification

In this paper, we propose a depth propagation scheme based on optical flow field rectification towards more accurate depth reconstruction. In depth reconstruction, the occlusions and low-textural regions easily result in optical flow field errors, which lead ambiguous depth value or holes without depth in the obtained depth map. In this work, a scheme is proposed to improve the precision of depth propagation and the quality of depth reconstruction for dynamic scene. The proposed scheme first adaptively detects the occlusive or low-textural regions, and the obtained vectors in optical flow field are rectified properly. Subsequently, we process the occluded and ambiguous vectors for more precise depth propagation. We further leverage the boundary enhancement filters as a post-processing to sharpen the object boundaries in obtained depth maps for better quality. Quantitative evaluations show that the proposed scheme can reconstruct depth map with higher accuracy and better quality compared with the state-of-the-art methods.