Multiscale and Multifeature Segmentation of High-Spatial Resolution Remote Sensing Images Using Superpixels with Mutual Optimal Strategy

Object-based multiscale segmentation incorporating texture and edge features of high-resolution remote sensing images

Multiscale segmentation (MSS) is crucial in object-based image analysis methods (OBIA). How to describe the underlying features of remote sensing images and combine multiple features for object-based multiscale image segmentation is a hotspot in the field of OBIA. Traditional object-based segmentation methods mostly use spectral and shape features of remote sensing images and pay less attention to texture and edge features. We analyze traditional image segmentation methods and object-based MSS methods. Then, on the basis of comparing image texture feature description methods, a method for remote sensing image texture feature description based on time-frequency analysis is proposed. In addition, a method for measuring the texture heterogeneity of image objects is constructed on this basis. Using bottom-up region merging as an MSS strategy, an object-based MSS algorithm for remote sensing images combined with texture feature is proposed. Finally, based on the edge feature of remote sensing images, a description method of remote sensing image edge intensity and an edge fusion cost criterion are proposed. Combined with the heterogeneity criterion, an object-based MSS algorithm combining spectral, shape, texture, and edge features is proposed. Experiment results show that the comprehensive features object-based MSS algorithm proposed in this article can obtain more complete segmentation objects when segmenting ground objects with rich texture information and slender shapes and is not prone to over-segmentation. Compare with the traditional object-based segmentation algorithm, the average accuracy of the algorithm is increased by 4.54%, and the region ratio is close to 1, which will be more conducive to the subsequent processing and analysis of remote sensing images. In addition, the object-based MSS algorithm proposed in this article can effectively obtain more complete ground objects and can be widely used in scenes such as building extraction.

A survey on the utilization of Superpixel image for clustering based image segmentation

Superpixel become increasingly popular in image segmentation field as it greatly helps image segmentation techniques to segment the region of interest accurately in noisy environment and also reduces the computation effort to a great extent. However, selection of proper superpixel generation techniques and superpixel image segmentation techniques play a very crucial role in the domain of different kinds of image segmentation. Clustering is a well-accepted image segmentation technique and proved their effective performance over various image segmentation field. Therefore, this study presents an up-to-date survey on the employment of superpixel image in combined with clustering techniques for the various image segmentation. The contribution of the survey has four parts namely (i) overview of superpixel image generation techniques, (ii) clustering techniques especially efficient partitional clustering techniques, their issues and overcoming strategies, (iii) Review of superpixel combined with clustering strategies exist in literature for various image segmentation, (iv) lastly, the comparative study among superpixel combined with partitional clustering techniques has been performed over oral pathology and leaf images to find out the efficacy of the combination of superpixel and partitional clustering approaches. Our evaluations and observation provide in-depth understanding of several superpixel generation strategies and how they apply to the partitional clustering method.

CMSuG: Competitive mechanism-based superpixel generation method for image segmentation

During the last years, object-based image segmentation (OBIA) has seen a considerable increase in the image segmentation. OBIA is generally based on superpixel methods, in which the clustering-based method plays an increasingly important role. Most clustering methods for generating superpixels suffer from inaccurate classification points with inappropriate cluster centers. To solve the problem, we propose a competitive mechanism-based superpixel generation (CMSuG) method, which both accelerates convergence and promotes robustness for noise sensitivity. Then, image segmentation results will be obtained by a region adjacent graph (RAG)-based merging algorithm after constructing an RAG. However, high segmentation accuracy is customarily accompanied by expensive time-consuming costs. To improve computational efficiency, we address a parallel CMSuG algorithm, the time of which is much less than the CMSuG method. In addition, we present a parallel RAG method to decrease the expensive time-consuming cost in serial RAG construction. By leveraging parallel techniques, the running time of the whole image segmentation method decline sharply with the time complexity from O (N) + O (K 2) to O (N/K) or O (K 2), in which N is the size of an input image and K is the given number of the superpixel. In the experiments, both nature image and remote sensing image segmentation results demonstrate that our CMSuG method outperforms the state-of-the-art superpixel generation methods, and then performs well for image segmentation in turn. Compared with the serial segmentation method, our parallel techniques gain more than four times acceleration in both remote sensing image dataset and nature image dataset.

Introducing GEOBIA to Landscape Imageability Assessment: A Multi-Temporal Case Study of the Nature Reserve “Kózki”, Poland

Geographic object-based image analysis (GEOBIA) is a primary remote sensing tool utilized in land-cover mapping and change detection. Land-cover patches are the primary data source for landscape metrics and ecological indicator calculations; however, their application to visual landscape character (VLC) indicators was little investigated to date. To bridge the knowledge gap between GEOBIA and VLC, this paper puts forward the theoretical concept of using viewpoint as a landscape imageability indicator into the practice of a multi-temporal land-cover case study and explains how to interpret the indicator. The study extends the application of GEOBIA to visual landscape indicator calculations. In doing so, eight different remote sensing imageries are the object of GEOBIA, starting from a historical aerial photograph (1957) and CORONA declassified scene (1965) to contemporary (2018) UAV-delivered imagery. The multi-temporal GEOBIA-delivered land-cover patches are utilized to find the minimal isovist set of viewpoints and to calculate three imageability indicators: the number, density, and spacing of viewpoints. The calculated indicator values, viewpoint rank, and spatial arrangements allow us to describe the scale, direction, rate, and reasons for VLC changes over the analyzed 60 years of landscape evolution. We found that the case study nature reserve (“Kózki”, Poland) landscape imageability transformed from visually impressive openness to imageability due to the impression of several landscape rooms enclosed by forest walls. Our results provide proof that the number, rank, and spatial arrangement of viewpoints constitute landscape imageability measured with the proposed indicators. Discussing the method’s technical limitations, we believe that our findings contribute to a better understanding of land-cover change impact on visual landscape structure dynamics and further VLC indicator development.

SPMF-Net: Weakly Supervised Building Segmentation by Combining Superpixel Pooling and Multi-Scale Feature Fusion

The lack of pixel-level labeling limits the practicality of deep learning-based building semantic segmentation. Weakly supervised semantic segmentation based on image-level labeling results in incomplete object regions and missing boundary information. This paper proposes a weakly supervised semantic segmentation method for building detection. The proposed method takes the image-level label as supervision information in a classification network that combines superpixel pooling and multi-scale feature fusion structures. The main advantage of the proposed strategy is its ability to improve the intactness and boundary accuracy of a detected building. Our method achieves impressive results on two 2D semantic labeling datasets, which outperform some competing weakly supervised methods and are close to the result of the fully supervised method.

Correlation Analysis between UBD and LST in Hefei, China, Using Luojia1-01 Night-Time Light Imagery

The urban heat island (UHI) is one of the essential phenomena of the modern urban climate. In recent years, urbanization in China has gradually accelerated, and the heat island effect has also intensified as the urban impervious surface area and the number of buildings is increasing. Urban building density (UBD) is one of the main factors affecting UHI, but there is little discussion on the relationship between the two. This paper takes Hefei as the research area, combines UBD data estimated by Luojia1-01 night-time light (NTL) imagery as the research object with land surface temperature (LST) data obtained from Landsat8 images, and carries out spatial correlation analysis on 0.5 × 0.5 km to 2 × 2 km resolution for them, so as to explore the relationship between UBD and UHI. The results show the following: (1) Luojia1-01 data have a good ability to estimate UBD and have fewer errors when compared with the actual UBD data; (2) At the four spatial scales, UBD and LST present a significant positive correlation that increases with the enlargement of the spatial scale; and (3) Moreover, the fitting effect of the Geographically Weighted Regression (GWR) model is better than that of the ordinary least squares (OLS) regression model.

Region Merging Method for Remote Sensing Spectral Image Aided by Inter-Segment and Boundary Homogeneities

Image segmentation is extensively used in remote sensing spectral image processing. Most of the existing region merging methods assess the heterogeneity or homogeneity using global or pre-defined parameters, which lack the flexibility to further improve the goodness-of-fit. Recently, the local spectral angle (SA) threshold was used to produce promising segmentation results. However, this method falls short of considering the inherent relationship between adjacent segments. In order to overcome this limitation, an adaptive SA thresholds methods, which combines the inter-segment and boundary homogeneities of adjacent segment pairs by their respective weights to refine predetermined SA threshold, is employed in a hybrid segmentation framework to enhance the image segmentation accuracy. The proposed method can effectively improve the segmentation accuracy with different kinds of reference objects compared to the conventional segmentation approaches based on the global SA and local SA thresholds. The results of the visual comparison also reveal that our method can match more accurately with reference polygons of varied sizes and types.

Mapping of Coastal Cities Using Optimized Spectral–Spatial Features Based Multi-Scale Superpixel Classification

The high interior heterogeneity of land surface covers in high-resolution image of coastal cities makes classification challenging. To meet this challenge, a Multi-Scale Superpixels-based Classification method using Optimized Spectral–Spatial features, denoted as OSS-MSSC, is proposed in this paper. In the proposed method, the multi-scale superpixels are firstly generated to capture the local spatial structures of the ground objects with various sizes. Then, the normalized difference vegetation index and extend multi-attribute profiles are introduced to extract the spectral–spatial features from the multi-spectral bands of the image. To reduce the redundancy of the spectral–spatial features, the crossover-based search algorithm is utilized for feature optimization. The pre-classification results at each single scale are, therefore, obtained based on the optimized spectral–spatial features and random forest classifier. Finally, the ultimate classification is generated via the majority voting of those pre-classification results in each scale. Experimental results on the Gaofen-2 image of Qingdao and WorldView-2 image of Hong Kong, China confirmed the effectiveness of the proposed method. The experiments verify that the OSS-MSSC method not only works effectively on the homogeneous regions, but also is able to preserve the small local spatial structures in the high-resolution remote sensing images of coastal cities.

Fusion of Multiscale Convolutional Neural Networks for Building Extraction in Very High-Resolution Images

Extracting buildings from very high resolution (VHR) images has attracted much attention but is still challenging due to their large varieties in appearance and scale. Convolutional neural networks (CNNs) have shown effective and superior performance in automatically learning high-level and discriminative features in extracting buildings. However, the fixed receptive fields make conventional CNNs insufficient to tolerate large scale changes. Multiscale CNN (MCNN) is a promising structure to meet this challenge. Unfortunately, the multiscale features extracted by MCNN are always stacked and fed into one classifier, which make it difficult to recognize objects with different scales. Besides, the repeated sub-sampling processes lead to a blurred boundary of the extracted features. In this study, we proposed a novel parallel support vector mechanism (SVM)-based fusion strategy to take full use of deep features at different scales as extracted by the MCNN structure. We firstly designed a MCNN structure with different sizes of input patches and kernels, to learn multiscale deep features. After that, features at different scales were individually fed into different support vector machine (SVM) classifiers to produce rule images for pre-classification. A decision fusion strategy is then applied on the pre-classification results based on another SVM classifier. Finally, superpixels are applied to refine the boundary of the fused results using region-based maximum voting. For performance evaluation, the well-known International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam dataset was used in comparison with several state-of-the-art algorithms. Experimental results have demonstrated the superior performance of the proposed methodology in extracting complex buildings in urban districts.