Structure-guided statistical textural distinctiveness for salient region detection in natural images

An Ensemble of Generation- and Retrieval-based Image Captioning with Dual Generator Generative Adversarial Network

Image captioning, which aims to generate a sentence to describe the key content of a query image, is an important but challenging task. Existing image captioning approaches can be categorised into two types: generation-based methods and retrieval-based methods. Retrieval-based methods describe images by retrieving pre-existing captions from a repository. Generation-based methods synthesize a new sentence that verbalizes the query image. Both ways have certain advantages but suffer from their own disadvantages. In the paper, we propose a novel EnsCaption model, which aims at enhancing an ensemble of retrieval-based and generation-based image captioning methods through a novel dual generator generative adversarial network. Specifically, EnsCaption is composed of a caption generation model that synthesizes tailored captions for the query image, a caption re-ranking model that retrieves the best-matching caption from a candidate caption pool consisting of generated captions and pre-retrieved captions, and a discriminator that learns the multi-level difference between the generated/retrieved captions and the ground-truth captions. During the adversarial training process, the caption generation model and the caption re-ranking model provide improved synthetic and retrieved candidate captions with high ranking scores from the discriminator, while the discriminator based on multi-level ranking is trained to assign low ranking scores to the generated and retrieved image captions. Our model absorbs the merits of both generation-based and retrieval-based approaches. We conduct comprehensive experiments to evaluate the performance of EnsCaption on two benchmark datasets: MSCOCO and Flickr-30K. Experimental results show that EnsCaption achieves impressive performance compared to the strong baseline methods.

Saliency Region Detection Method Based on Background and Spatial Position

Saliency region detection methods have become one of the hotspots in the field of image processing as an important method to improve the real-time and accurate analysis of massive data. Integrating more effective prior knowledge is a viable direction for improving the performance of saliency region detection methods. Most of the methods based on background prior and boundary connectivity prior assume the boundary area of the image as the background, by restraining the background to highlight the salient area. When the boundary area of the image does not describe the background well (such as a large difference in border area features), if the entire frame of the image is put together to compute the background feature, the calculation of the background feature will be inaccurate. In view of the above shortcomings, this paper proposed a saliency region detection method based on background and spatial position. This method carried on the image boundary super pixel clustering, determined the background feature according to the clustering center, and used the difference between the super pixel on the image and the background super pixel, and its spatial position to calculate the salient of the super pixels. This approach used MATLAB to program and experiment. The method was compared with a series of the state-of-the-art methods. The AUC of proposed algorithm reaches 0.839, and the MAE is 0.220, showing the effectiveness of the proposed algorithm.

The Semi-Variogram and Spectral Distortion Measures for Image Texture Retrieval

Semi-variogram estimators and distortion measures of signal spectra are utilized in this paper for image texture retrieval. On the use of the complete Brodatz database, most high retrieval rates are reportedly based on multiple features and the combinations of multiple algorithms, while the classification using single features is still a challenge to the retrieval of diverse texture images. The semi-variogram, which is theoretically sound and the cornerstone of spatial statistics, has the characteristics shared between true randomness and complete determinism and, therefore, can be used as a useful tool for both the structural and statistical analysis of texture images. Meanwhile, spectral distortion measures derived from the theory of linear predictive coding provide a rigorously mathematical model for signal-based similarity matching and have been proven useful for many practical pattern classification systems. Experimental results obtained from testing the proposed approach using the complete Brodatz database, and the the University of Illinois at Urbana-Champaign texture database suggests the effectiveness of the proposed approach as a single-feature-based dissimilarity measure for real-time texture retrieval.