Fast object detection based on selective visual attention

Saliency-CCE: Exploiting colour contextual extractor and saliency-based biomedical image segmentation

Biomedical image segmentation is one critical component in computer-aided system diagnosis. However, various non-automatic segmentation methods are usually designed to segment target objects with single-task driven, ignoring the potential contribution of multi-task, such as the salient object detection (SOD) task and the image segmentation task. In this paper, we propose a novel dual-task framework for white blood cell (WBC) and skin lesion (SL) saliency detection and segmentation in biomedical images, called Saliency-CCE. Saliency-CCE consists of a preprocessing of hair removal for skin lesions images, a novel colour contextual extractor (CCE) module for the SOD task and an improved adaptive threshold (AT) paradigm for the image segmentation task. In the SOD task, we perform the CCE module to extract hand-crafted features through a novel colour channel volume (CCV) block and a novel colour activation mapping (CAM) block. We first exploit the CCV block to generate a target object's region of interest (ROI). After that, we employ the CAM block to yield a refined salient map as the final salient map from the extracted ROI. We propose a novel adaptive threshold (AT) strategy in the segmentation task to automatically segment the WBC and SL from the final salient map. We evaluate our proposed Saliency-CCE on the ISIC-2016, the ISIC-2017, and the SCISC datasets, which outperform representative state-of-the-art SOD and biomedical image segmentation approaches. Our code is available at https://github.com/zxg3017/Saliency-CCE.

Detecting Salient Image Objects Using Color Histogram Clustering for Region Granularity

Salient object detection represents a novel preprocessing stage of many practical image applications in the discipline of computer vision. Saliency detection is generally a complex process to copycat the human vision system in the processing of color images. It is a convoluted process because of the existence of countless properties inherent in color images that can hamper performance. Due to diversified color image properties, a method that is appropriate for one category of images may not necessarily be suitable for others. The selection of image abstraction is a decisive preprocessing step in saliency computation and region-based image abstraction has become popular because of its computational efficiency and robustness. However, the performances of the existing region-based salient object detection methods are extremely hooked on the selection of an optimal region granularity. The incorrect selection of region granularity is potentially prone to under- or over-segmentation of color images, which can lead to a non-uniform highlighting of salient objects. In this study, the method of color histogram clustering was utilized to automatically determine suitable homogenous regions in an image. Region saliency score was computed as a function of color contrast, contrast ratio, spatial feature, and center prior. Morphological operations were ultimately performed to eliminate the undesirable artifacts that may be present at the saliency detection stage. Thus, we have introduced a novel, simple, robust, and computationally efficient color histogram clustering method that agglutinates color contrast, contrast ratio, spatial feature, and center prior for detecting salient objects in color images. Experimental validation with different categories of images selected from eight benchmarked corpora has indicated that the proposed method outperforms 30 bottom-up non-deep learning and seven top-down deep learning salient object detection methods based on the standard performance metrics.

Ensemble model with cascade attention mechanism for high-resolution remote sensing image scene classification

Scene classification of high-resolution remote sensing images is a fundamental task of earth observation. And numerous methods have been proposed to achieve this. However, these models are inadequate as the number of labelled training data limits them. Most of the existing methods entirely rely on global information, while regions with class-specific ground objects determine the categories of high-resolution remote sensing images. An ensemble model with a cascade attention mechanism, which consists of two kinds of the convolutional neural network, is proposed to address these issues. To improve the generality of the feature extractor, each branch is trained on different large datasets to enrich the prior knowledge. Moreover, to force the model to focus on the most class-specific region in each high-resolution remote sensing image, a cascade attention mechanism is proposed to combine the branches and capture the most discriminative information. By experiments on four benchmark datasets, OPTIMAL-31, UC Merced Land-Use Dataset, Aerial Image Dataset and NWPU-RESISC45, the proposed end-to-end model cascade attention-based double branches model in this paper achieves state-of-the-art performance on each benchmark dataset.

Video Salient Object Detection Using Spatiotemporal Deep Features

This paper presents a method for detecting salient objects in videos, where temporal information in addition to spatial information is fully taken into account. Following recent reports on the advantage of deep features over conventional handcrafted features, we propose a new set of spatiotemporal deep (STD) features that utilize local and global contexts over frames. We also propose new spatiotemporal conditional random field (STCRF) to compute saliency from STD features. STCRF is our extension of CRF to the temporal domain and describes the relationships among neighboring regions both in a frame and over frames. STCRF leads to temporally consistent saliency maps over frames, contributing to accurate detection of salient objects' boundaries and noise reduction during detection. Our proposed method first segments an input video into multiple scales and then computes a saliency map at each scale level using STD features with STCRF. The final saliency map is computed by fusing saliency maps at different scale levels. Our experiments, using publicly available benchmark datasets, confirm that the proposed method significantly outperforms the state-of-the-art methods. We also applied our saliency computation to the video object segmentation task, showing that our method outperforms existing video object segmentation methods.

A NOVEL BIOMECHANICS-BASED APPROACH FOR PERSON RE-IDENTIFICATION BY GENERATING DENSE COLOR SIFT SALIENCE FEATURES

Currently, identifying humans using biomechanics-based approaches has gained a lot of significance for person re-identification. Biomechanics-based approaches use knee-hip angle–angle relationships and body movements for person re-identification. Generally, biomechanics of human walking and running is used for person re-identification. In fact, person re-identification is a complex and important task in academia as well as industry and remains an unsolved issue in the computer vision field. The subjects most commonly addressed regarding person re-identification include significant feature extraction that can function accurately with invariant appearance and robust classification. In this study, a significant color feature descriptor is proposed by combining dense color-SIFT and global convex hull salience region features. First convex hull boundary points are detected using the SIFT technique. Furthermore, it is extended with Grubb’s outlier test to eliminate the outlier points detected by SIFT and mark the saliency region via convex hull. Then dense-SIFT and dense-CHF methods are used to extract local and global features within the convex hull region, respectively. Finally, the pre-ranked common nearest neighbor selection technique is applied to minimize overhead of dataset and generate more robust rank classification. The proposed technique is tested using three-camera database video sequences and three publicly available datasets, namely i-LIDS, VIPeR and GRID. Performance of re-identification system is evaluated using a statistical method with CMC curves. The results show better re-identification accuracy in solving the aforementioned problems.

Improving Visual Saliency Computing With Emotion Intensity

Saliency maps that integrate individual feature maps into a global measure of visual attention are widely used to estimate human gaze density. Most of the existing methods consider low-level visual features and locations of objects, and/or emphasize the spatial position with center prior. Recent psychology research suggests that emotions strongly influence human visual attention. In this paper, we explore the influence of emotional content on visual attention. On top of the traditional bottom-up saliency map generation, our saliency map is generated in cooperation with three emotion factors, i.e., general emotional content, facial expression intensity, and emotional object locations. Experiments, carried out on National University of Singapore Eye Fixation (a public eye tracking data set), demonstrate that incorporating emotion does improve the quality of visual saliency maps computed by bottom-up approaches for the gaze density estimation. Our method increases about 0.1 on an average of area under the curve of receiver operation characteristic curve, compared with the four baseline bottom-up approaches (Itti's, attention based on information maximization, saliency using natural, and graph-based vision saliency).

Salient Object Detection: A Benchmark

We extensively compare, qualitatively and quantitatively, 41 state-of-the-art models (29 salient object detection, 10 fixation prediction, 1 objectness, and 1 baseline) over seven challenging data sets for the purpose of benchmarking salient object detection and segmentation methods. From the results obtained so far, our evaluation shows a consistent rapid progress over the last few years in terms of both accuracy and running time. The top contenders in this benchmark significantly outperform the models identified as the best in the previous benchmark conducted three years ago. We find that the models designed specifically for salient object detection generally work better than models in closely related areas, which in turn provides a precise definition and suggests an appropriate treatment of this problem that distinguishes it from other problems. In particular, we analyze the influences of center bias and scene complexity in model performance, which, along with the hard cases for the state-of-the-art models, provide useful hints toward constructing more challenging large-scale data sets and better saliency models. Finally, we propose probable solutions for tackling several open problems, such as evaluation scores and data set bias, which also suggest future research directions in the rapidly growing field of salient object detection.