Automatic Registration Method for Optical Remote Sensing Images with Large Background Variations Using Line Segments

A Robust 3D Density Descriptor Based on Histogram of Oriented Primary Edge Structure for SAR and Optical Image Co-Registration

The co-registration between SAR and optical images is a challenging task because of the speckle noise of SAR and the nonlinear radiation distortions (NRD), particularly in the one-look situation. In this paper, we propose a novel density descriptor based on the histogram of oriented primary edge structure (HOPES) for the co-registration of SAR and optical images, aiming to describe the shape structure of patches more firm. In order to extract the primary edge structure, we develop the novel multi-scale sigmoid Gabor (MSG) detector and a primary edge fusion algorithm. Based on the HOPES, we propose the co-registration method. To obtain stable and uniform keypoints, the non-maximum suppressed SAR-Harris (NMS-SAR-Harris) and deviding grids methods are used. NMS-SSD fast template matching and fast sample consensus (FSC) algorithm are used to further complete and optimize matching. We use two one-look simulated SAR images to demonstrate that the signal-to-noise ratio (SNR) of MSG is more than 10 dB higher than other state-of-the-stage detectors; the binary edge maps and F-score show that MSG has more accurate positioning performance. Compared with the other state-of-the-stage co-registration methods, the image co-registration results obtained on seven pairs of test images show that, the correct match rate (CMR) and the root mean squared error (RMSE) improve by more than 25% and 15% on average, respectively. It is experimentally demonstrated that the HOPES is robust against speckle noise and NRD, which can effectively improve the matching success rate and accuracy.

A Line Matching Method Based on Multiple Intensity Ordering with Uniformly Spaced Sampling

This paper presents a line matching method based on multiple intensity ordering with uniformly spaced sampling. Line segments are extracted from the image pyramid, with the aim of adapting scale changes and addressing fragmentation problem. The neighborhood of line segments was divided into sub-regions adaptively according to intensity order to overcome the difficulty brought by various line lengths. An intensity-based local feature descriptor was introduced by constructing multiple concentric ring-shaped structures. The dimension of the descriptor was reduced significantly by uniformly spaced sampling and dividing sample points into several point sets while improving the discriminability. The performance of the proposed method was tested on public datasets which cover various scenarios and compared with another two well-known line matching algorithms. The experimental results show that our method achieves superior performance dealing with various image deformations, especially scale changes and large illumination changes, and provides much more reliable correspondences.

Registration Algorithm Based on Line-Intersection-Line for Satellite Remote Sensing Images of Urban Areas

Image registration is an important step in remote sensing image processing, especially for images of urban areas, which are often used for urban planning, environmental assessment, and change detection. Urban areas have many artificial objects whose contours and edges provide abundant line features. However, the locations of line endpoints are greatly affected by large background variations. Considering that line intersections remain relatively stable and have high positioning accuracy even with large background variations, this paper proposes a high-accuracy remote sensing image registration algorithm that is based on the line-intersection-line (LIL) structure, with two line segments and their intersection. A double-rectangular local descriptor and a spatial relationship-based outlier removal strategy are designed on the basis of the LIL structure. First, the LILs are extracted based on multi-scale line segments. Second, LIL local descriptors are built with pixel gradients in the LIL neighborhood to realize initial matching. Third, the spatial relations between initial matches are described with the LIL structure and simple affine properties. Finally, the graph-based LIL outlier removal strategy is conducted and incorrect matches are eliminated step by step. The proposed algorithm is tested on simulated and real images and compared with state-of-the-art methods. The experiments prove that the proposed algorithm can achieve sub-pixel registration accuracy, high precision, and robust performance even with significant background variations.

Augmented Reality Mapping of Rock Mass Discontinuities and Rockfall Susceptibility Based on Unmanned Aerial Vehicle Photogrammetry

In rockfall hazard management, the investigation and detection of potential rockfall source areas on rock cliffs by remote-sensing-based susceptibility analysis are of primary importance. However, when the rockfall analysis results are used as feedback to the fieldwork, the irregular slope surface morphology makes it difficult to objectively locate the risk zones of hazard maps on the real slopes, and the problem of straightforward on-site visualization of rockfall susceptibility remains a research gap. This paper presents some of the pioneering studies on the augmented reality (AR) mapping of geospatial information from cyberspace within 2D screens to the physical world for on-site visualization, which directly recognizes the rock mass and superimposes corresponding rock discontinuities and rockfall susceptibility onto the real slopes. A novel method of edge-based tracking of the rock mass target for mobile AR is proposed, where the model edges extracted from unmanned aerial vehicle (UAV) structure-from-motion (SfM) 3D reconstructions are aligned with the corresponding actual rock mass to estimate the camera pose accurately. Specifically, the visually prominent edges of dominant structural planes were first explored and discovered to be a robust visual feature of rock mass for AR tracking. The novel approaches of visual-geometric synthetic image (VGSI) and prominent structural plane (Pro-SP) were developed to extract structural planes with identified prominent edges as 3D template models which could provide a pose estimation reference. An experiment verified that the proposed Pro-SP template model could effectively improve the edge tracking performance and quality, and this approach was relatively robust to the changes of sunlight conditions. A case study was carried out on a typical roadcut cliff in the Mentougou District of Beijing, China. The results validate the scalability of the proposed mobile AR strategy, which is applicable and suitable for cliff-scale fieldwork. The results also demonstrate the feasibility, efficiency, and significance of the geoinformation AR mapping methodology for on-site zoning and locating of potential rockfalls, and providing relevant guidance for subsequent detailed site investigation.

Remote Sensing Image Registration with Line Segments and Their Intersections

Image registration is a basic but essential step for remote sensing image processing, and finding stable features in multitemporal images is one of the most considerable challenges in the field. The main shape contours of artificial objects (e.g., roads, buildings, farmlands, and airports) can be generally described as a group of line segments, which are stable features, even in images with evident background changes (e.g., images taken before and after a disaster). In this study, a registration method that uses line segments and their intersections is proposed for multitemporal remote sensing images. First, line segments are extracted in image pyramids to unify the scales of the reference image and the test image. Then, a line descriptor based on the gradient distribution of local areas is constructed, and the segments are matched in image pyramids. Lastly, triplets of intersections of matching lines are selected to estimate affine transformation between two images. Additional corresponding intersections are provided based on the estimated transformation, and an iterative process is adopted to remove outliers. The performance of the proposed method is tested on a variety of optical remote sensing image pairs, including synthetic and real data. Compared with existing methods, our method can provide more accurate registration results, even in images with significant background changes.