Segment spatial-temporal representation and cooperative learning of convolution neural networks for multimodal-based action recognition

Memristive competitive hopfield neural network for image segmentation application

Image segmentation implementation provides simplified and effective feature information of image. Neural network algorithms have made significant progress in the application of image segmentation task. However, few studies focus on the implementation of hardware circuits with high-efficiency analog calculations and parallel operations for image segmentation problem. In this paper, a memristor-based competitive Hopfield neural network circuit is proposed to deal with the image segmentation problem. In this circuit, the memristive cross array is applied to store synaptic weights and perform matrix operations. The competition module based on the Winner-take-all mechanism is composed of the competition neurons and the competition control circuit, which simplifies the energy function of the Hopfield neural network and realizes the output function. Operational amplifiers and ABM modules are used to integrate operations and process external input information, respectively. Based on these designs, the circuit can automatically implement iteration and update of data. A series of PSPICE simulations are designed to verify the image segmentation capability of this circuit. Comparative experimental results and analysis show that this circuit has effective improvements both in processing speed and segmentation accuracy compared with other methods. Moreover, the proposed circuit shows good robustness to noise and memristive variation.

Bio-driven visual saliency detection with color factor

Most visual saliency computing methods build models based on the content of an image without considering the colorized effects. Biologically, human attention can be significantly influenced by color. This study firstly investigates the sole contribution of colors in visual saliency and then proposes a bio-driven saliency detection method with a color factor. To study the color saliency despite the contents, an eye-tracking dataset containing color images and gray-scale images of the same content is proposed, collected from 18 subjects. The CIELab color space was selected to conduct extensive analysis to identify the contribution of colors in guiding visual attention. Based on the observations that some particular colors and combinations of color blocks can attract much attention than others, the influence of colors on visual saliency is represented computationally. Incorporating the color factor, a novel saliency detection model is proposed to model the human color perception prioritization, and a deep neural network model is proposed for eye fixation prediction. Experiments validate that the proposed bio-driven saliency detection models make substantial improvements in finding informative content, and they benefit the detection of salient objects which are close to human visual attention in natural scenes.