An Automatic and Novel SAR Image Registration Algorithm: A Case Study of the Chinese GF-3 Satellite

A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions

As an all-weather and all-day remote sensing image data source, SAR (Synthetic Aperture Radar) images have been widely applied, and their registration accuracy has a direct impact on the downstream task effectiveness. The existing registration algorithms mainly focus on small sub-images, and there is a lack of available accurate matching methods for large-size images. This paper proposes a high-precision, rapid, large-size SAR image dense-matching method. The method mainly includes four steps: down-sampling image pre-registration, sub-image acquisition, dense matching, and the transformation solution. First, the ORB (Oriented FAST and Rotated BRIEF) operator and the GMS (Grid-based Motion Statistics) method are combined to perform rough matching in the semantically rich down-sampled image. In addition, according to the feature point pairs, a group of clustering centers and corresponding images are obtained. Subsequently, a deep learning method based on Transformers is used to register images under weak texture conditions. Finally, the global transformation relationship can be obtained through RANSAC (Random Sample Consensus). Compared with the SOTA algorithm, our method's correct matching point numbers are increased by more than 2.47 times, and the root mean squared error (RMSE) is reduced by more than 4.16%. The experimental results demonstrate that our proposed method is efficient and accurate, which provides a new idea for SAR image registration.

An Image Registration Method Based on Correlation Matching of Dominant Scatters for Distributed Array ISAR

Distributed array radar provides new prospects for three-dimensional (3D) inverse synthetic aperture radar (ISAR) imaging. The accuracy of image registration, as an essential part of 3D ISAR imaging, affects the performance of 3D reconstruction. In this paper, the imaging process of distributed array ISAR is proposed according to the imaging model. The ISAR images of distributed array radar at different APCs have different distribution of scatters. When the local distribution of scatters for the same target are quite different, the performance of the existing ISAR image registration methods may not be optimal. Therefore, an image registration method is proposed by integrating the feature-based method and the area-based method. The proposed method consists of two stages: coarse registration and fine registration. In the first stage, a dominant scatters model is established based on scale-invariant feature transform (SIFT). In the second stage, sub-pixel precision registration is achieved using the local correlation matching method. The effectiveness of the proposed method is verified by comparison with other image registration methods. The 3D reconstruction of the registered experimental data is carried out to assess the practicability of the proposed method.

A Novel Mosaic Method for Spaceborne ScanSAR Images Based on Homography Matrix Compensation

Accurate and efficient image mosaicking is essential for generating wide-range swath images of spaceborne scanning synthetic aperture radar (ScanSAR). However, the existing methods cannot guarantee the accuracy and efficiency of stitching simultaneously, especially when mosaicking multiple large-area images. In this paper, we propose a novel image mosaic method based on homography matrix compensation to solve the mentioned problem. A set of spaceborne ScanSAR images from the Gaofen-3 (GF-3) satellite were selected to test the performance of the new method. First, images are preprocessed by an improved Wallis filter to eliminate intensity inconsistencies. Then, to reduce the enormous computational redundancy of registration, the overlapping areas of adjacent images are coarsely extracted using geolocation technologies. Furthermore, to improve the efficiency of stitching and maintain the original information and resolution of images, we deduce a compensation of homography matrix to implement downsampled images registration and original-size images projection. After stitching, the transitions at the edges of the images were smooth and seamless, the information and resolution of the original images were preserved successfully, and the efficiency of the mosaic was improved by approximately one thousand-fold. The validity, high efficiency and reliability of the method are verified.

Modality-Free Feature Detector and Descriptor for Multimodal Remote Sensing Image Registration

The nonlinear radiation distortions (NRD) among multimodal remote sensing images bring enormous challenges to image registration. The traditional feature-based registration methods commonly use the image intensity or gradient information to detect and describe the features that are sensitive to NRD. However, the nonlinear mapping of the corresponding features of the multimodal images often results in failure of the feature matching, as well as the image registration. In this paper, a modality-free multimodal remote sensing image registration method (SRIFT) is proposed for the registration of multimodal remote sensing images, which is invariant to scale, radiation, and rotation. In SRIFT, the nonlinear diffusion scale (NDS) space is first established to construct a multi-scale space. A local orientation and scale phase congruency (LOSPC) algorithm are then used so that the features of the images with NRD are mapped to establish a one-to-one correspondence, to obtain sufficiently stable key points. In the feature description stage, a rotation-invariant coordinate (RIC) system is adopted to build a descriptor, without requiring estimation of the main direction. The experiments undertaken in this study included one set of simulated data experiments and nine groups of experiments with different types of real multimodal remote sensing images with rotation and scale differences (including synthetic aperture radar (SAR)/optical, digital surface model (DSM)/optical, light detection and ranging (LiDAR) intensity/optical, near-infrared (NIR)/optical, short-wave infrared (SWIR)/optical, classification/optical, and map/optical image pairs), to test the proposed algorithm from both quantitative and qualitative aspects. The experimental results showed that the proposed method has strong robustness to NRD, being invariant to scale, radiation, and rotation, and the achieved registration precision was better than that of the state-of-the-art methods.

Robust SAR Automatic Target Recognition Based on Transferred MS-CNN with L2-Regularization

Though Synthetic Aperture Radar (SAR) Automatic Target Recognition (ATR) via Convolutional Neural Networks (CNNs) has made huge progress toward deep learning, some key issues still remain unsolved due to the lack of sufficient samples and robust model. In this paper, we proposed an efficient transferred Max-Slice CNN (MS-CNN) with L²-Regularization for SAR ATR, which could enrich the features and recognize the targets with superior performance. Firstly, the data amplification method is presented to reduce the computational time and enrich the raw features of SAR targets. Secondly, the proposed MS-CNN framework with L²-Regularization is trained to extract robust features, in which the L²-Regularization is incorporated to avoid the overfitting phenomenon and further optimizing our proposed model. Thirdly, transfer learning is introduced to enhance the feature representation and discrimination, which could boost the performance and robustness of the proposed model on small samples. Finally, various activation functions and dropout strategies are evaluated for further improving recognition performance. Extensive experiments demonstrated that our proposed method could not only outperform other state-of-the-art methods on the public and extended MSTAR dataset but also obtain good performance on the random small datasets.

A Point Pattern Chamfer Registration of Optical and SAR Images Based on Mesh Grids

Automatic image registration of optical-to-Synthetic aperture radar (SAR) images is difficult because of the inconsistency of radiometric and geometric properties between the optical image and the SAR image. The intensity-based methods may require many calculations and be ineffective when there are geometric distortions between these two images. The feature-based methods have high requirements on features, and there are certain challenges in feature extraction and matching. A new automatic optical-to-SAR image registration framework is proposed in this paper. First, modified holistically nested edge detection is employed to detect the main contours in both the optical and SAR images. Second, a mesh grid strategy is presented to perform a coarse-to-fine registration. The coarse registration calculates the feature matching and summarizes the preliminary results for the fine registration process. Finally, moving direct linear transformation is introduced to perform a homography warp to alleviate parallax. The experimental results show the effectiveness and accuracy of our proposed method.

Improved WαSH Feature Matching Based on 2D-DWT for Stereo Remote Sensing Images

Image matching is an outstanding issue because of the existing of geometric and radiometric distortion in stereo remote sensing images. Weighted α-shape (WαSH) local invariant features are tolerant to image rotation, scale change, affine deformation, illumination change, and blurring. However, since the number of WαSH features is small, it is difficult to get enough matches to estimate the satisfactory homography matrix or fundamental matrix. In addition, the WαSH detector is extremely sensitive to image noise because it is built on sampled edges. Considering the shortcomings of the WαSH detector, this paper improves the WαSH feature matching method based on the 2D discrete wavelet transform (2D-DWT). The method firstly performs 2D-DWT on the image, and then detects WαSH features on the transformed images. According to the methods of descriptor construction for WαSH features, three matching methods on the basis of wavelet transform WαSH features (WWF), improved wavelet transform WαSH features (IWWF), and layered IWWF (LIWWF) are distinguished with respect to the character of the sub-images. The experimental results on the dataset containing affine distortion, scale distortion, illumination change, and noise images, showed that the proposed methods acquired more matches and better stableness than WαSH. Experimentation on remote sensing images with less affine distortion and slight noise showed that the proposed methods obtained the correct matching rate greater than 90%. For images containing severe distortion, KAZE obtained a 35.71% correct matching rate, which is unacceptable for calculating the homography matrix, while IWWF achieved a 71.42% correct matching rate. IWWF was the only method that achieved the correct matching rate of no less than 50% for all four test stereo remote sensing image pairs and was the most stable compared to MSER, DWT-MSER, WαSH, DWT-WαSH, KAZE, WWF, and LIWWF.

A PolSAR Image Segmentation Algorithm Based on Scattering Characteristics and the Revised Wishart Distance

A novel segmentation algorithm for polarimetric synthetic aperture radar (PolSAR) images is proposed in this paper. The method is composed of two essential components: a merging order and a merging predicate. The similarity measured by the complex-kind Hotelling–Lawley trace (HLT) statistic is used to decide the merging order. The merging predicate is determined by the scattering characteristics and the revised Wishart distance between adjacent pixels, which can greatly improve the performance in speckle suppression and detail preservation. A postprocessing step is applied to obtain a satisfactory result after the merging operation. The decomposition and merging processes are iteratively executed until the termination criterion is met. The superiority of the proposed method was verified with experiments on two RADARSAT-2 PolSAR images and a Gaofen-3 PolSAR image, which demonstrated that the proposed method can obtain more accurate segmentation results and shows a better performance in speckle suppression and detail preservation than the other algorithms.