From image processing to computational neuroscience: a neural model based on histogram equalization

Matching visual induction effects on screens of different size

In the film industry, the same movie is expected to be watched on displays of vastly different sizes, from cinema screens to mobile phones. But visual induction, the perceptual phenomenon by which the appearance of a scene region is affected by its surroundings, will be different for the same image shown on two displays of different dimensions. This phenomenon presents a practical challenge for the preservation of the artistic intentions of filmmakers, because it can lead to shifts in image appearance between viewing destinations. In this work, we show that a neural field model based on the efficient representation principle is able to predict induction effects and how, by regularizing its associated energy functional, the model is still able to represent induction but is now invertible. From this finding, we propose a method to preprocess an image in a screen-size dependent way so that its perception, in terms of visual induction, may remain constant across displays of different size. The potential of the method is demonstrated through psychophysical experiments on synthetic images and qualitative examples on natural images.

A Cortical-Inspired Sub-Riemannian Model for Poggendorff-Type Visual Illusions

We consider Wilson-Cowan-type models for the mathematical description of orientation-dependent Poggendorff-like illusions. Our modelling improves two previously proposed cortical-inspired approaches, embedding the sub-Riemannian heat kernel into the neuronal interaction term, in agreement with the intrinsically anisotropic functional architecture of V1 based on both local and lateral connections. For the numerical realisation of both models, we consider standard gradient descent algorithms combined with Fourier-based approaches for the efficient computation of the sub-Laplacian evolution. Our numerical results show that the use of the sub-Riemannian kernel allows us to reproduce numerically visual misperceptions and inpainting-type biases in a stronger way in comparison with the previous approaches.

Visual illusions via neural dynamics: Wilson-Cowan-type models and the efficient representation principle

We reproduce suprathreshold perception phenomena, specifically visual illusions, by Wilson-Cowan (WC)-type models of neuronal dynamics. Our findings show that the ability to replicate the illusions considered is related to how well the neural activity equations comply with the efficient representation principle. Our first contribution consists in showing that the WC equations can reproduce a number of brightness and orientation-dependent illusions. Then we formally prove that there cannot be an energy functional that the WC dynamics are minimizing. This leads us to consider an alternative, variational modeling, which has been previously employed for local histogram equalization (LHE) tasks. To adapt our model to the architecture of V1, we perform an extension that has an explicit dependence on local image orientation. Finally, we report several numerical experiments showing that LHE provides a better reproduction of visual illusions than the original WC formulation, and that its cortical extension is capable also to reproduce complex orientation-dependent illusions.NEW & NOTEWORTHY We show that the Wilson-Cowan equations can reproduce a number of brightness and orientation-dependent illusions. Then we formally prove that there cannot be an energy functional that the Wilson-Cowan equations are minimizing, making them suboptimal with respect to the efficient representation principle. We thus propose a slight modification that is consistent with such principle and show that this provides a better reproduction of visual illusions than the original Wilson-Cowan formulation. We also consider the cortical extension of both models to deal with more complex orientation-dependent illusions.

A neural field model for color perception unifying assimilation and contrast

We address the question of color-space interactions in the brain, by proposing a neural field model of color perception with spatial context for the visual area V1 of the cortex. Our framework reconciles two opposing perceptual phenomena, known as simultaneous contrast and chromatic assimilation. They have been previously shown to act synergistically, so that at some point in an image, the color seems perceptually more similar to that of adjacent neighbors, while being more dissimilar from that of remote ones. Thus, their combined effects are enhanced in the presence of a spatial pattern, and can be measured as larger shifts in color matching experiments. Our model supposes a hypercolumnar structure coding for colors in V1, and relies on the notion of color opponency introduced by Hering. The connectivity kernel of the neural field exploits the balance between attraction and repulsion in color and physical spaces, so as to reproduce the sign reversal in the influence of neighboring points. The color sensation at a point, defined from a steady state of the neural activities, is then extracted as a nonlinear percept conveyed by an assembly of neurons. It connects the cortical and perceptual levels, because we describe the search for a color match in asymmetric matching experiments as a mathematical projection on color sensations. We validate our color neural field alongside this color matching framework, by performing a multi-parameter regression to data produced by psychophysicists and ourselves. All the results show that we are able to explain the nonlinear behavior of shifts observed along one or two dimensions in color space, which cannot be done using a simple linear model.

The color perception produced by an image heavily depends on the spatial distribution of its colors. From this “color in context” phenomenon, extensively studied in psychophysics for decades, has arisen the question in neuroscience of how color and space interact in the brain. Visual signals are indeed processed in such a way that neighboring pixels make the perception at some point different from its real color, inducing a color shift. In this work, we propose to emulate perception in context by modeling the activity of color sensitive neurons with a neural field. Our framework unifies two antagonistic effects, assimilation and contrast, which have been suggested to occur simultaneously but at different scales. We use the notion of color opponency inspired by the work of Hering, so as to express these effects as a combination of attraction and repulsion in physical and color spaces. We introduce the concept of “color sensation”, and show how to rigorously link the neural field model to perceptual shifts, by considering color matching as a mathematical projection on color sensations. The results show that our model is able to reproduce some nontrivial behaviors of the color shifts observed in experiments.

In Praise of Artifice Reloaded: Caution With Natural Image Databases in Modeling Vision

Subjective image quality databases are a major source of raw data on how the visual system works in naturalistic environments. These databases describe the sensitivity of many observers to a wide range of distortions of different nature and intensity seen on top of a variety of natural images. Data of this kind seems to open a number of possibilities for the vision scientist to check the models in realistic scenarios. However, while these natural databases are great benchmarks for models developed in some other way (e.g., by using the well-controlled artificial stimuli of traditional psychophysics), they should be carefully used when trying to fit vision models. Given the high dimensionality of the image space, it is very likely that some basic phenomena are under-represented in the database. Therefore, a model fitted on these large-scale natural databases will not reproduce these under-represented basic phenomena that could otherwise be easily illustrated with well selected artificial stimuli. In this work we study a specific example of the above statement. A standard cortical model using wavelets and divisive normalization tuned to reproduce subjective opinion on a large image quality dataset fails to reproduce basic cross-masking. Here we outline a solution for this problem by using artificial stimuli and by proposing a modification that makes the model easier to tune. Then, we show that the modified model is still competitive in the large-scale database. Our simulations with these artificial stimuli show that when using steerable wavelets, the conventional unit norm Gaussian kernels in divisive normalization should be multiplied by high-pass filters to reproduce basic trends in masking. Basic visual phenomena may be misrepresented in large natural image datasets but this can be solved with model-interpretable stimuli. This is an additional argument in praise of artifice in line with Rust and Movshon (2005).

Derivatives and inverse of cascaded linear+nonlinear neural models

In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina).

Retinal Lateral Inhibition Provides the Biological Basis of Long-Range Spatial Induction

Retinal lateral inhibition is one of the conventional efficient coding mechanisms in the visual system that is produced by interneurons that pool signals over a neighborhood of presynaptic feedforward cells and send inhibitory signals back to them. Thus, the receptive-field (RF) of a retinal ganglion cell has a center-surround receptive-field (RF) profile that is classically represented as a difference-of-Gaussian (DOG) adequate for efficient spatial contrast coding. The DOG RF profile has been attributed to produce the psychophysical phenomena of brightness induction, in which the perceived brightness of an object is affected by that of its vicinity, either shifting away from it (brightness contrast) or becoming more similar to it (brightness assimilation) depending on the size of the surfaces surrounding the object. While brightness contrast can be modeled using a DOG with a narrow surround, brightness assimilation requires a wide suppressive surround. Early retinal studies determined that the suppressive surround of a retinal ganglion cell is narrow (< 100–300 μm; ‘classic RF’), which led researchers to postulate that brightness assimilation must originate at some post-retinal, possibly cortical, stage where long-range interactions are feasible. However, more recent studies have reported that the retinal interneurons also exhibit a spatially wide component (> 500–1000 μm). In the current study, we examine the effect of this wide interneuron RF component in two biophysical retinal models and show that for both of the retinal models it explains the long-range effect evidenced in simultaneous brightness induction phenomena and that the spatial extent of this long-range effect of the retinal model responses matches that of perceptual data. These results suggest that the retinal lateral inhibition mechanism alone can regulate local as well as long-range spatial induction through the narrow and wide RF components of retinal interneurons, arguing against the existing view that spatial induction is operated by two separate local vs. long-range mechanisms.