The perceived contrast of texture patches embedded in natural images

Mechanisms of Surround Suppression Effect on the Contrast Sensitivity of V1 Neurons in Cats

Surround suppression (SS) is a phenomenon that a neuron’s response to visual stimuli within the classical receptive field (cRF) is suppressed by a concurrent stimulation in the surrounding receptive field (sRF) beyond the cRF. Studies show that SS affects neuronal response contrast sensitivity in the primary visual cortex (V1). However, the underlying mechanisms remain unclear. Here, we examined SS effect on the contrast sensitivity of cats’ V1 neurons with different preferred SFs using external noise-masked visual stimuli and perceptual template model (PTM) analysis at the system level. The contrast sensitivity was evaluated by the inverted threshold contrast of neurons in response to circular gratings of different contrasts in the cRF with or without an annular grating in the sRF. Our results showed that SS significantly reduced the contrast sensitivity of cats’ V1 neurons. The SS-induced reduction of contrast sensitivity was not correlated with SS strength but was dependent on neuron’s preferred SF, with a larger reduction for neurons with low preferred SFs than those with high preferred SFs. PTM analysis of threshold versus external noise contrast (TvC) functions indicated that SS decreased contrast sensitivity by increasing both the internal additive noise and impact of external noise for neurons with low preferred SFs, but improving only internal additive noise for neurons with high preferred SFs. Furthermore, the SS effect on the contrast-response function of low- and high-SF neurons also exhibited different mechanisms in contrast gain and response gain. Collectively, these results suggest that the mechanisms of SS effect on neuronal contrast sensitivity may depend on neuronal populations with different SFs.

Flexible contextual modulation of naturalistic texture perception in peripheral vision

Peripheral vision comprises most of our visual field, and is essential in guiding visual behavior. Its characteristic capabilities and limitations, which distinguish it from foveal vision, have been explained by the most influential theory of peripheral vision as the product of representing the visual input using summary statistics. Despite its success, this account may provide a limited understanding of peripheral vision, because it neglects processes of perceptual grouping and segmentation. To test this hypothesis, we studied how contextual modulation, namely the modulation of the perception of a stimulus by its surrounds, interacts with segmentation in human peripheral vision. We used naturalistic textures, which are directly related to summary-statistics representations. We show that segmentation cues affect contextual modulation, and that this is not captured by our implementation of the summary-statistics model. We then characterize the effects of different texture statistics on contextual modulation, providing guidance for extending the model, as well as for probing neural mechanisms of peripheral vision.

Human sensitivity to perturbations constrained by a model of the natural image manifold

Humans are remarkably well tuned to the statistical properties of natural images. However, quantitative characterization of processing within the domain of natural images has been difficult because most parametric manipulations of a natural image make that image appear less natural. We used generative adversarial networks (GANs) to constrain parametric manipulations to remain within an approximation of the manifold of natural images. In the first experiment, seven observers decided which one of two synthetic perturbed images matched a synthetic unperturbed comparison image. Observers were significantly more sensitive to perturbations that were constrained to an approximate manifold of natural images than they were to perturbations applied directly in pixel space. Trial-by-trial errors were consistent with the idea that these perturbations disrupt configural aspects of visual structure used in image segmentation. In a second experiment, five observers discriminated paths along the image manifold as recovered by the GAN. Observers were remarkably good at this task, confirming that observers are tuned to fairly detailed properties of an approximate manifold of natural images. We conclude that human tuning to natural images is more general than detecting deviations from natural appearance, and that humans have, to some extent, access to detailed interrelations between natural images.

The tuning of human visual cortex to variations in the 1/fα amplitude spectra and fractal properties of synthetic noise images

Natural scenes share a consistent distribution of energy across spatial frequencies (SF) known as the 1/f^α amplitude spectrum (α≈0.8-1.5, mean 1.2). This distribution is scale-invariant, which is a fractal characteristic of natural scenes with statistically similar structure at different spatial scales. While the sensitivity of the visual system to the 1/f properties of natural scenes has been studied extensively using psychophysics, relatively little is known about the tuning of cortical responses to these properties. Here, we use fMRI and retinotopic mapping techniques to measure and analyze BOLD responses in early visual cortex (V1, V2, and V3) to synthetic noise images that vary in their 1/f^α amplitude spectra (α=0.25 to 2.25, step size: 0.50) and contrast levels (10% and 30%) (Experiment 1). To compare the dependence of the BOLD response between the photometric (intensity based) and geometric (fractal) properties of our stimuli, in Experiment 2 we compared grayscale noise images to their binary (thresholded) counterparts, which contain only black and white regions. In both experiments, early visual cortex responded maximally to stimuli generated to have an input 1/f slope corresponding to natural 1/f^α amplitude spectra, and lower BOLD responses were found for steeper or shallower 1/f slopes (peak modulation: 0.59% for 1.25 vs. 0.31% for 2.25). To control for changing receptive field sizes, responses were also analyzed across multiple eccentricity bands in cortical surface space. For most eccentricity bands, BOLD responses were maximal for natural 1/f^α amplitude spectra, but importantly there was no difference in the BOLD response to grayscale stimuli and their corresponding thresholded counterparts. Since the thresholding of an image changes its measured 1/f slope (α) but not its fractal characteristics, this suggests that neuronal responses in early visual cortex are not strictly driven by spectral slope values (photometric properties) but rather their embedded geometric, fractal-like scaling properties.

Optimal Information Representation and Criticality in an Adaptive Sensory Recurrent Neuronal Network

Recurrent connections play an important role in cortical function, yet their exact contribution to the network computation remains unknown. The principles guiding the long-term evolution of these connections are poorly understood as well. Therefore, gaining insight into their computational role and into the mechanism shaping their pattern would be of great importance. To that end, we studied the learning dynamics and emergent recurrent connectivity in a sensory network model based on a first-principle information theoretic approach. As a test case, we applied this framework to a model of a hypercolumn in the visual cortex and found that the evolved connections between orientation columns have a "Mexican hat" profile, consistent with empirical data and previous modeling work. Furthermore, we found that optimal information representation is achieved when the network operates near a critical point in its dynamics. Neuronal networks working near such a phase transition are most sensitive to their inputs and are thus optimal in terms of information representation. Nevertheless, a mild change in the pattern of interactions may cause such networks to undergo a transition into a different regime of behavior in which the network activity is dominated by its internal recurrent dynamics and does not reflect the objective input. We discuss several mechanisms by which the pattern of interactions can be driven into this supercritical regime and relate them to various neurological and neuropsychiatric phenomena.

Nonblurred regions show priority for gaze direction over spatial blur

The human eye continuously forms images of our 3D environment using a finite and dynamically changing depth of focus. Since different objects in our environment reside at different depth planes, the resulting retinal images consist of both focused and spatially blurred objects concurrently. Here, we wanted to measure what effect such a mixed visual diet may have on the pattern of eye movements. For that, we have constructed composite stimuli, each containing an intact photograph and several progressively blurred versions of it, all arranged in a 3 × 3 square array and presented simultaneously as a single image. We have measured eye movements for 7 such composite stimuli as well as for their corresponding root mean square (RMS) contrast-equated versions to control for any potential contrast variations as a result of the blurring. We have found that when observers are presented with such arrays of blurred and nonblurred images they fixate significantly more frequently on the stimulus regions that had little or no blur at all ( p < .001). A similar pattern of fixations was found for the RMS contrast-equated versions of the stimuli indicating that the observed distributions of fixations is not simply the result of variations in image contrasts due to spatial blurring. Further analysis revealed that, during each 5 second presentation, the image regions containing little or no spatial blur were fixated first while other regions with larger amounts of blur were fixated later, if fixated at all. The results contribute to the increasing list of stimulus parameters that affect patterns of eye movements during scene perception.

On the effectiveness of noise masks: naturalistic vs. un-naturalistic image statistics

It has been argued that the human visual system is optimized for identification of broadband objects embedded in stimuli possessing orientation averaged power spectra fall-offs that obey the 1/f(β) relationship typically observed in natural scene imagery (i.e., β=2.0 on logarithmic axes). Here, we were interested in whether individual spatial channels leading to recognition are functionally optimized for narrowband targets when masked by noise possessing naturalistic image statistics (β=2.0). The current study therefore explores the impact of variable β noise masks on the identification of narrowband target stimuli ranging in spatial complexity, while simultaneously controlling for physical or perceived differences between the masks. The results show that β=2.0 noise masks produce the largest identification thresholds regardless of target complexity, and thus do not seem to yield functionally optimized channel processing. The differential masking effects are discussed in the context of contrast gain control.

Contextual effects in speed perception may occur at an early stage of processing

How does nearby motion affect the perceived speed of a target region? When a central drifting Gabor patch is surrounded by translating noise, its speed can be misperceived over a fourfold range. Typically, when a surround moves in the same direction, perceived centre speed is reduced; for opposite-direction surrounds it increases. Measuring this illusion for a variety of surround properties reveals that the motion context effects are a saturating function of surround speed (Experiment I) and contrast (Experiment II). Our analyses indicate that the effects are consistent with a subtractive process, rather than with speed being averaged over area. In Experiment III we exploit known properties of the motion system to ask where these surround effects impact. Using 2D plaid stimuli, we find that surround-induced shifts in perceived speed of one plaid component produce substantial shifts in perceived plaid direction. This indicates that surrounds exert their influence early in processing, before pattern motion direction is computed. These findings relate to ongoing investigations of surround suppression for direction discrimination, and are consistent with single-cell findings of direction-tuned suppressive and facilitatory interactions in primary visual cortex (V1).

Natural images dominate in binocular rivalry

Ecological approaches to perception have demonstrated that information encoding by the visual system is informed by the natural environment, both in terms of simple image attributes like luminance and contrast, and more complex relationships corresponding to Gestalt principles of perceptual organization. Here, we ask if this optimization biases perception of visual inputs that are perceptually bistable. Using the binocular rivalry paradigm, we designed stimuli that varied in either their spatiotemporal amplitude spectra or their phase spectra. We found that noise stimuli with "natural" amplitude spectra (i.e., amplitude content proportional to 1/f, where f is spatial or temporal frequency) dominate over those with any other systematic spectral slope, along both spatial and temporal dimensions. This could not be explained by perceived contrast measurements, and occurred even though all stimuli had equal energy. Calculating the effective contrast following attenuation by a model contrast sensitivity function suggested that the strong contrast dependency of rivalry provides the mechanism by which binocular vision is optimized for viewing natural images. We also compared rivalry between natural and phase-scrambled images and found a strong preference for natural phase spectra that could not be accounted for by observer biases in a control task. We propose that this phase specificity relates to contour information, and arises either from the activity of V1 complex cells, or from later visual areas, consistent with recent neuroimaging and single-cell work. Our findings demonstrate that human vision integrates information across space, time, and phase to select the input most likely to hold behavioral relevance.

Centre-surround effects on perceived orientation in complex images

Using the simultaneous tilt illusion [Gibson, J., & Radner, M. (1937). Adaptation, after-effect and contrast in the perception of tilted lines. Journal of Experimental Psychology, 12, 453-467], we investigate the perception of orientation in natural images and textures with similar statistical properties. We show that the illusion increases if observers judge the average orientation of textures rather than sinusoidal gratings. Furthermore, the illusion can be induced by surrounding textures with a broad range of orientations, even those without a clearly perceivable orientation. A robust illusion is induced by natural images, and is increased by randomising the phase spectra of those images. We present a simple model of orientation processing that can accommodate most of our observations.