Center–surround interaction with adaptive inhibition: A computational model for contour detection

Learning Nonclassical Receptive Field Modulation for Contour Detection

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks.

Biologically Inspired Hierarchical Contour Detection with Surround Modulation and Neural Connection

Contour is a very important feature in biological visual cognition and has been extensively investigated as a fundamental vision problem. In connection with the limitations of conventional models in detecting image contours in complex scenes, a hierarchical image contour extraction method is proposed based on the biological vision mechanism that draws on the perceptual characteristics of the early vision for features such as edges, shapes, and colours. By simulating the information processing mechanisms of the cells’ receptive fields in the early stages of the biological visual system, we put forward a computational model that combines feedforward, lateral, and feedback neural connections to decode and obtain the image contours. Our model simulations and their results show that the established hierarchical contour detection model can adequately fit the characteristics of the biological experiment, quickly and effectively detect the salient contours in complex scenes, and better suppress the unwanted textures.

A Fast Contour Detection Model Inspired by Biological Mechanisms in Primary Vision System

Compared to computer vision systems, the human visual system is more fast and accurate. It is well accepted that V1 neurons can well encode contour information. There are plenty of computational models about contour detection based on the mechanism of the V1 neurons. Multiple-cue inhibition operator is one well-known model, which is based on the mechanism of V1 neurons' non-classical receptive fields. However, this model is time-consuming and noisy. To solve these two problems, we propose an improved model which integrates some additional other mechanisms of the primary vision system. Firstly, based on the knowledge that the salient contours only occupy a small portion of the whole image, the prior filtering is introduced to decrease the running time. Secondly, based on the physiological finding that nearby neurons often have highly correlated responses and thus include redundant information, we adopt the uniform samplings to speed up the algorithm. Thirdly, sparse coding is introduced to suppress the unwanted noises. Finally, to validate the performance, we test it on Berkeley Segmentation Data Set. The results show that the improved model can decrease running time as well as keep the accuracy of the contour detection.

Contour detection improved by context-adaptive surround suppression

Recently, many image processing applications have taken advantage of a psychophysical and neurophysiological mechanism, called “surround suppression” to extract object contour from a natural scene. However, these traditional methods often adopt a single suppression model and a fixed input parameter called “inhibition level”, which needs to be manually specified. To overcome these drawbacks, we propose a novel model, called “context-adaptive surround suppression”, which can automatically control the effect of surround suppression according to image local contextual features measured by a surface estimator based on a local linear kernel. Moreover, a dynamic suppression method and its stopping mechanism are introduced to avoid manual intervention. The proposed algorithm is demonstrated and validated by a broad range of experimental results.

Orientation Histogram-Based Center-Surround Interaction: An Integration Approach for Contour Detection

Contour is a critical feature for image description and object recognition in many computer vision tasks. However, detection of object contour remains a challenging problem because of disturbances from texture edges. This letter proposes a scheme to handle texture edges by implementing contour integration. The proposed scheme integrates structural segments into contours while inhibiting texture edges with the help of the orientation histogram-based center-surround interaction model. In the model, local edges within surroundings exert a modulatory effect on central contour cues based on the co-occurrence statistics of local edges described by the divergence of orientation histograms in the local region. We evaluate the proposed scheme on two well-known challenging boundary detection data sets (RuG and BSDS500). The experiments demonstrate that our scheme achieves a high [Formula: see text]-measure of up to 0.74. Results show that our scheme achieves integrating accurate contour while eliminating most of texture edges, a novel approach to long-range feature analysis.

A biologically-inspired framework for contour detection using superpixel-based candidates and hierarchical visual cues

Contour detection has been extensively investigated as a fundamental problem in computer vision. In this study, a biologically-inspired candidate weighting framework is proposed for the challenging task of detecting meaningful contours. In contrast to previous models that detect contours from pixels, a modified superpixel generation processing is proposed to generate a contour candidate set and then weigh the candidates by extracting hierarchical visual cues. We extract the low-level visual local cues to weigh the contour intrinsic property and mid-level visual cues on the basis of Gestalt principles for weighting the contour grouping constraint. Experimental results tested on the BSDS benchmark show that the proposed framework exhibits promising performances to capture meaningful contours in complex scenes.

Limitations of short range Mexican hat connection for driving target selection in a 2D neural field: activity suppression and deviation from input stimuli

Dynamic Neural Field models (DNF) often use a kernel of connection with short range excitation and long range inhibition. This organization has been suggested as a model for brain structures or for artificial systems involved in winner-take-all processes such as saliency localization, perceptual decision or target/action selection. A good example of such a DNF is the superior colliculus (SC), a key structure for eye movements. Recent results suggest that the superficial layers of the SC (SCs) exhibit relatively short range inhibition with a longer time constant than excitation. The aim of the present study was to further examine the properties of a DNF with such an inhibition pattern in the context of target selection. First we tested the effects of stimulus size and shape on when and where self-maintained clusters of firing neurons appeared, using three variants of the model. In each model variant, small stimuli led to rapid formation of a spiking cluster, a range of medium sizes led to the suppression of any activity on the network and hence to no target selection, while larger sizes led to delayed selection of multiple loci. Second, we tested the model with two stimuli separated by a varying distance. Again single, none, or multiple spiking clusters could occur, depending on distance and relative stimulus strength. For short distances, activity attracted toward the strongest stimulus, reminiscent of well-known behavioral data for saccadic eye movements, while for larger distances repulsion away from the second stimulus occurred. All these properties predicted by the model suggest that the SCs, or any other neural structure thought to implement a short range MH, is an imperfect winner-take-all system. Although, those properties call for systematic testing, the discussion gathers neurophysiological and behavioral data suggesting that such properties are indeed present in target selection for saccadic eye movements.

Altered modulation of gamma oscillation frequency by speed of visual motion in children with autism spectrum disorders

BACKGROUND

Recent studies link autism spectrum disorders (ASD) with an altered balance between excitation and inhibition (E/I balance) in cortical networks. The brain oscillations in high gamma-band (50-120 Hz) are sensitive to the E/I balance and may appear useful biomarkers of certain ASD subtypes. The frequency of gamma oscillations is mediated by level of excitation of the fast-spiking inhibitory basket cells recruited by increasing strength of excitatory input. Therefore, the experimental manipulations affecting gamma frequency may throw light on inhibitory networks dysfunction in ASD.

METHODS

Here, we used magnetoencephalography (MEG) to investigate modulation of visual gamma oscillation frequency by speed of drifting annular gratings (1.2, 3.6, 6.0 °/s) in 21 boys with ASD and 26 typically developing boys aged 7-15 years. Multitaper method was used for analysis of spectra of gamma power change upon stimulus presentation and permutation test was applied for statistical comparisons. We also assessed in our participants visual orientation discrimination thresholds, which are thought to depend on excitability of inhibitory networks in the visual cortex.

RESULTS

Although frequency of the oscillatory gamma response increased with increasing velocity of visual motion in both groups of participants, the velocity effect was reduced in a substantial proportion of children with ASD. The range of velocity-related gamma frequency modulation correlated inversely with the ability to discriminate oblique line orientation in the ASD group, while no such correlation has been observed in the group of typically developing participants.

CONCLUSIONS

Our findings suggest that abnormal velocity-related gamma frequency modulation in ASD may constitute a potential biomarker for reduced excitability of fast-spiking inhibitory neurons in a subset of children with ASD.

Boundary detection using double-opponency and spatial sparseness constraint

Brightness and color are two basic visual features integrated by the human visual system (HVS) to gain a better understanding of color natural scenes. Aiming to combine these two cues to maximize the reliability of boundary detection in natural scenes, we propose a new framework based on the color-opponent mechanisms of a certain type of color-sensitive double-opponent (DO) cells in the primary visual cortex (V1) of HVS. This type of DO cells has oriented receptive field with both chromatically and spatially opponent structure. The proposed framework is a feedforward hierarchical model, which has direct counterpart to the color-opponent mechanisms involved in from the retina to V1. In addition, we employ the spatial sparseness constraint (SSC) of neural responses to further suppress the unwanted edges of texture elements. Experimental results show that the DO cells we modeled can flexibly capture both the structured chromatic and achromatic boundaries of salient objects in complex scenes when the cone inputs to DO cells are unbalanced. Meanwhile, the SSC operator further improves the performance by suppressing redundant texture edges. With competitive contour detection accuracy, the proposed model has the additional advantage of quite simple implementation with low computational cost.

Potential roles of the interaction between model V1 neurons with orientation-selective and non-selective surround inhibition in contour detection

Both the neurons with orientation-selective and with non-selective surround inhibition have been observed in the primary visual cortex (V1) of primates and cats. Though the inhibition coming from the surround region (named as non-classical receptive field, nCRF) has been considered playing critical role in visual perception, the specific role of orientation-selective and non-selective inhibition in the task of contour detection is less known. To clarify above question, we first carried out computational analysis of the contour detection performance of V1 neurons with different types of surround inhibition, on the basis of which we then proposed two integrated models to evaluate their role in this specific perceptual task by combining the two types of surround inhibition with two different ways. The two models were evaluated with synthetic images and a set of challenging natural images, and the results show that both of the integrated models outperform the typical models with orientation-selective or non-selective inhibition alone. The findings of this study suggest that V1 neurons with different types of center–surround interaction work in cooperative and adaptive ways at least when extracting organized structures from cluttered natural scenes. This work is expected to inspire efficient phenomenological models for engineering applications in field of computational machine-vision.

Silent suppressive surrounds and optimal spatial frequencies of single neurons in cat V1

The receptive fields of the clear majority of neurons in the primary visual cortex (V1) of cats contain silent surround regions beyond the classical receptive field (CRF). When stimulated on their own, the silent surround regions do not generate action potentials (spikes); instead, they modulate (and usually partially suppress) spike responses to stimuli presented in the CRF. In the present study, we subdivided our sample of single V1 neurons recorded from anesthetized cats into two distinct categories: surround-suppressive (SS) cells and surround-non-suppressive (SN) cells. Consistent with previous reports, we found a negative correlation between the size of the CRF and the optimal spatial frequency (SF) of circular patches of achromatic gratings presented in the cells' receptive fields. Furthermore, we found a positive correlation between the strength of the surround suppression and the optimal spatial frequency of the achromatic gratings presented in the cells' receptive fields. The correlation between the strength of surround suppression and the optimal spatial frequency was stronger in neurons with suppressive regions located in the so-called 'end' zones. The functional implications of these relationships are discussed.

Multifeature-based surround inhibition improves contour detection in natural images

To effectively perform visual tasks like detecting contours, the visual system normally needs to integrate multiple visual features. Sufficient physiological studies have revealed that for a large number of neurons in the primary visual cortex (V1) of monkeys and cats, neuronal responses elicited by the stimuli placed within the classical receptive field (CRF) are substantially modulated, normally inhibited, when difference exists between the CRF and its surround, namely, non-CRF, for various local features. The exquisite sensitivity of V1 neurons to the center-surround stimulus configuration is thought to serve important perceptual functions, including contour detection. In this paper, we propose a biologically motivated model to improve the performance of perceptually salient contour detection. The main contribution is the multifeature-based center-surround framework, in which the surround inhibition weights of individual features, including orientation, luminance, and luminance contrast, are combined according to a scale-guided strategy, and the combined weights are then used to modulate the final surround inhibition of the neurons. The performance was compared with that of single-cue-based models and other existing methods (especially other biologically motivated ones). The results show that combining multiple cues can substantially improve the performance of contour detection compared with the models using single cue. In general, luminance and luminance contrast contribute much more than orientation to the specific task of contour extraction, at least in gray-scale natural images.

Contextual modulation as de-texturizer

Contextual modulation refers to the effect of texture placed outside of a neuron's classical receptive field as well as the effect of surround texture on the perceptual properties of variegated regions within. In this minireview, we argue that one role of contextual modulation is to enhance the perception of contours at the expense of textures, in short to de-texturize the image. The evidence for this role comes mainly from three sources: psychophysical studies of shape after-effects, computational models of neurons that exhibit iso-orientation surround inhibition, and fMRI studies revealing specialized areas for contour as opposed to texture processing. The relationship between psychophysical studies that support the notion of contextual modulation as de-texturizer and those that investigate contour integration and crowding is discussed.

Weighted KPCA degree of homogeneity amended nonclassical receptive field inhibition model for salient contour extraction in low-light-level image

The stimulus response of the classical receptive field (CRF) of neuron in primary visual cortex is affected by its periphery [i.e., non-CRF (nCRF)]. This modulation exerts inhibition, which depends primarily on the correlation of both visual stimulations. The theory of periphery and center interaction with visual characteristics can be applied in night vision information processing. In this paper, a weighted kernel principal component analysis (WKPCA) degree of homogeneity (DH) amended inhibition model inspired by visual perceptual mechanisms is proposed to extract salient contour from complex natural scene in low-light-level image. The core idea is that multifeature analysis can recognize the homogeneity in modulation coverage effectively. Computationally, a novel WKPCA algorithm is presented to eliminate outliers and anomalous distribution in CRF and accomplish principal component analysis precisely. On this basis, a new concept and computational procedure for DH is defined to evaluate the dissimilarity between periphery and center comprehensively. Through amending the inhibition from nCRF to CRF by DH, our model can reduce the interference of noises, suppress details, and textures in homogeneous regions accurately. It helps to further avoid mutual suppression among inhomogeneous regions and contour elements. This paper provides an improved computational visual model with high-performance for contour detection from cluttered natural scene in night vision image.

Image segmentation using a sparse coding model of cortical area V1

Algorithms that encode images using a sparse set of basis functions have previously been shown to explain aspects of the physiology of a primary visual cortex (V1), and have been used for applications, such as image compression, restoration, and classification. Here, a sparse coding algorithm, that has previously been used to account for the response properties of orientation tuned cells in primary visual cortex, is applied to the task of perceptually salient boundary detection. The proposed algorithm is currently limited to using only intensity information at a single scale. However, it is shown to out-perform the current state-of-the-art image segmentation method (Pb) when this method is also restricted to using the same information.