Stereo matching based on multi-scale fusion and multi-type support regions

Occlusion disparity refinement for stereo matching through the geometric prior-based adaptive label search

In stereo matching, occlusion disparity refinement is one of the challenges when attempting to improve disparity accuracy. In order to refine the disparity in occluded regions, a geometric prior guided adaptive label search method and sequential disparity filling strategy are proposed. In our method, considering the scene structural correlation between pixels, the geometric prior information such as image patch similarity, matching distance, and disparity constraint is used in the proposed label search energy function and the disparity labels are searched by superpixel matching. Thus, the reliable disparity labels are adaptively searched and propagated for occlusion filling. In order to improve the accuracy in large occluded regions, by using the proposed sequential filling strategy, occluded regions are decomposed into multiple blocks and filled in multiple steps from the periphery; thus, reliable labels are iteratively propagated to the interior of occluded regions without violating the smooth disparity assumption. Experimental results on the Middlebury V3 benchmark show that, compared with other state-of-the-art algorithms, the proposed method achieves better disparity results under multiple criteria. The proposed method can provide better disparity refinement for typical stereo matching algorithms.

Enhanced Soft 3D Reconstruction Method with an Iterative Matching Cost Update Using Object Surface Consensus

In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.