Asymmetric correlation: a noise robust similarity measure for template matching

A rotation invariant template matching algorithm based on Sub-NCC

This paper proposes an anti-rotation template matching method based on a portion of the whole pixels. To solve the problem that the speed of the original template matching method based on NCC (Normalized cross correlation) is too slow for the rotated image, a template matching method based on Sub-NCC is proposed, which improves the anti-jamming ability of the algorithm. At the same time, in order to improve the matching speed, the rotation invariant edge points are selected from the rotation invariant pixels, and the selected points are used for rough matching to quickly screen out the unmatched areas. The theoretical analysis and experimental results show that the accuracy of this method is more than 95%. For the search map at any angle with the resolution at the level of 300,000 pixel, after selecting the appropriate pyramid series and threshold, the matching time can be controlled to within 0.1 s.

Reliable Template Matching for Image Detection in Vision Sensor Systems

Template matching is a simple image detection algorithm that can easily detect different types of objects just by changing the template without tedious training procedures. Despite these advantages, template matching is not currently widely used. This is because traditional template matching is not very reliable for images that differ from the template. The reliability of template matching can be improved by using additional information (depths for the template) available from the vision sensor system. Methods of obtaining the depth of a template using stereo vision or a few (two or more) template images or a short template video via mono vision are well known in the vision literature and have been commercialized. In this strategy, this paper proposes a template matching vision sensor system that can easily detect various types of objects without prior training. To this end, by using the additional information provided by the vision sensor system, we study a method to increase the reliability of template matching, even when there is a difference in the 3D direction and size between the template and the image. Template images obtained through the vision sensor provide a depth template. Using this depth template, it is possible to predict the change of the image according to the difference in the 3D direction and the size of the object. Using the predicted changes in these images, the template is calibrated close to the given image, and then template matching is performed. For ease of use, the algorithm is proposed as a closed form solution that avoids tedious recursion or training processes. For wider application and more accurate results, the proposed method considers the 3D direction and size difference in the perspective projection model and the general 3D rotation model.

Automated Surface Defect Inspection Based on Autoencoders and Fully Convolutional Neural Networks

This study aims to develop a novel automated computer vision algorithm for quality inspection of surfaces with complex patterns. The proposed algorithm is based on both an autoencoder (AE) and a fully convolutional neural network (FCN). The AE is adopted for the self-generation of templates from test targets for defect detection. Because the templates are produced from the test targets, the position alignment issues for the matching operations between templates and test targets can be alleviated. The FCN is employed for the segmentation of a template into a number of coherent regions. Because the AE has the limitation that its capacities for the regeneration of each coherent region in the template may be different, the segmentation of the template by FCN is beneficial for allowing the inspection of each region to be independently carried out. In this way, more accurate detection results can be achieved. Experimental results reveal that the proposed algorithm has the advantages of simplicity for training data collection, high accuracy for defect detection, and high flexibility for online inspection. The proposed algorithm is therefore an effective alternative for the automated inspection in smart factories with a growing demand for the reliability for high quality production.

Sub-pixel registration of multi-resolution imagery by correlation matching of the bathymetry-related features

Multispectral imaging plays a significant role in coastal mapping and monitoring applications. For tasks involving the integration of multiple overlapped images, precise co-registration of the multisource satellite images is a crucial preliminary step. However, due to the limited terrestrial area and insufficient landscape features, the traditional methods become less efficient or even invalid in offshore island environments. This study addresses the problem by exploring the feasibility of using bathymetry information for geometric registration of satellite imagery. Instead of using the ground control points (GCPs) or extracting the tie points from the landscape features, the band ratio values are extracted from the multispectral images and are subsequently matched between different images through a correlation-based similarity measure. By searching the optimum correlation within the positioning uncertainty radius, the translation between two satellite images is estimated. Thus, the geometric inconsistency between the multispectral images of different sources and resolutions is effectively reduced. This result is obtained by using the ample bathymetry features without the aid of the GCPs and the in-situ bathymetry data. The experimental results using GeoEye-1, Sentinel-2, and Landsat-8 images at Ganquan Island show that for an island setting with a limited terrestrial area, the developed method achieves sub-pixel registration accuracy (less than 2 m) in planimetry. The effect of the nonlinearity and outliers are accounted for using the Spearman correlation measure. The improvement in image alignment enables the integration of multispectral images of different sources and resolutions for producing an accurate and consistent interpretation for coastal comparative and synergistic applications.

Evaluation of Matching Costs for High-Quality Sea-Ice Surface Reconstruction from Aerial Images

Satellite remote sensing can be used effectively with a wide coverage and repeatability in large-scale Arctic sea-ice analysis. To produce reliable sea-ice information, satellite remote-sensing methods should be established and validated using accurate field data, but obtaining field data on Arctic sea-ice is very difficult due to limited accessibility. In this situation, digital surface models derived from aerial images can be a good alternative to topographical field data. However, to achieve this, we should discuss an additional issue, i.e., that low-textured surfaces on sea-ice can reduce the matching accuracy of aerial images. The matching performance is dependent on the matching cost and search window size used. Therefore, in order to generate high-quality sea-ice surface models, we first need to examine the influence of matching costs and search window sizes on the matching performance on low-textured sea-ice surfaces. For this reason, in this study, we evaluate the performance of matching costs in relation to changes of the search window size, using acquired aerial images of Arctic sea-ice. The evaluation concerns three factors. The first is the robustness of matching to low-textured surfaces. Matching costs for generating sea-ice surface models should have a high discriminatory power on low-textured surfaces, even with small search windows. To evaluate this, we analyze the accuracy, uncertainty, and optimal window size in terms of template matching. The second is the robustness of positioning to low-textured surfaces. One of the purposes of image matching is to determine the positions of object points that constitute digital surface models. From this point of view, we analyze the accuracy and uncertainty in terms of positioning object points. The last is the processing speed. Since the computation complexity is also an important performance indicator, we analyze the elapsed time for each of the processing steps. The evaluation results showed that the image domain costs were more effective for low-textured surfaces than the frequency domain costs. In terms of matching robustness, the image domain costs showed a better performance, even with smaller search windows. In terms of positioning robustness, the image domain costs also performed better because of the lower uncertainty. Lastly, in terms of processing speed, the PC (phase correlation) of the frequency domain showed the best performance, but the image domain costs, except MI (mutual information), were not far behind. From the evaluation results, we concluded that, among the compared matching costs, ZNCC (zero-mean normalized cross-correlation) is the most effective for sea-ice surface model generation. In addition, we found that it is necessary to adjust search window sizes properly, according to the number of textures required for reliable image matching on sea-ice surfaces, and that various uncertainties due to low-textured surfaces should be considered to determine the positions of object points.

Best-Buddies Similarity-Robust Template Matching Using Mutual Nearest Neighbors

We propose a novel method for template matching in unconstrained environments. Its essence is the Best-Buddies Similarity (BBS), a useful, robust, and parameter-free similarity measure between two sets of points. BBS is based on counting the number of Best-Buddies Pairs (BBPs)-pairs of points in source and target sets that are mutual nearest neighbours, i.e., each point is the nearest neighbour of the other. BBS has several key features that make it robust against complex geometric deformations and high levels of outliers, such as those arising from background clutter and occlusions. We study these properties, provide a statistical analysis that justifies them, and demonstrate the consistent success of BBS on a challenging real-world dataset while using different types of features.

Mapping the Individual Trees in Urban Orchards by Incorporating Volunteered Geographic Information and Very High Resolution Optical Remotely Sensed Data: A Template Matching-Based Approach

This paper presents a collective sensing approach that integrates imperfect Volunteered Geographic Information (VGI) obtained through Citizen Science (CS) tree mapping projects with very high resolution (VHR) optical remotely sensed data for low-cost, fine-scale, and accurate mapping of trees in urban orchards. To this end, an individual tree crown (ITC) detection technique utilizing template matching (TM) was developed for extracting urban orchard trees from VHR optical imagery. To provide the training samples for the TM algorithm, remotely sensed VGI about trees including the crowdsourced data about ITC locations and their crown diameters was adopted in this study. A data quality assessment of the proposed approach in the study area demonstrated that the detected trees had a very high degree of completeness (92.7%), a high thematic accuracy (false discovery rate (FDR) = 0.090, false negative rate (FNR) = 0.073, and F1 score (F1) = 0.918), and a fair positional accuracy (root mean square error(RMSE) = 1.02 m). Overall, the proposed approach based on the crowdsourced training samples generally demonstrated a promising ITC detection performance in our pilot project.