A possible explanation of the low-level brightness-contrast illusions in the light of an extended classical receptive field model of retinal ganglion cells

Photosynthetic Protein-Based Retinal Ganglion Cell Receptive Fields for Detecting Edges and Brightness Illusions

Bacteriorhodopsin, isolated from a halophilic bacterium, is a photosynthetic protein with a structure and function similar to those of the visual pigment rhodopsin. A voltaic cell with bacteriorhodopsin sandwiched between two transparent electrodes exhibits a time-differential response akin to that observed in retinal ganglion cells. It is intriguing as a means to emulate excitation and inhibition in the neural response. Here, we present a neuromorphic device emulating the retinal ganglion cell receptive field fabricated by patterning bacteriorhodopsin onto two transparent electrodes and encapsulating them with an electrolyte solution. This protein-based artificial ganglion cell receptive field is characterized as a bandpass filter that simultaneously replicates excitatory and inhibitory responses within a single element, successfully detecting image edges and phenomena of brightness illusions. The device naturally emulates the highly interacting ganglion cell receptive fields by exploiting the inherent properties of proteins without the need for electronic components, bias power supply, or an external operating circuit.

Binocular luster - A review

Binocular luster is a visual phenomenon that can be elicited by dichoptic stimuli showing an interocular difference in color or luminance contrast. For instance, when the two eyes are presented with simple center-surround stimuli in which the center patch in one eye is brighter and in the other eye darker than the common surround, the center patch in the fused percept assumes a lustrous appearance reminiscent of metal or graphite. Soon after the discovery of this phenomenon in the mid-19th century, it was intensively studied and several explanations were proposed. After this initial phase, however, research interest waned significantly. Stimulated by new insights into related phenomena and the underlying physiological mechanisms, the last 20 years have seen an increase in research activity in this field, which has considerably expanded our understanding of binocular luster. In this paper, we provide a detailed review of research on binocular luster over the past 170 years. We present and discuss the existing findings in a number of separate sections, dealing with 1) the phenomenology of binocular luster, 2) different theories that have been proposed, 3) several factors influencing the lustrous impression, 4) the relationship between binocular luster and binocular rivalry, 5) the current understanding of its neural basis, and 6) potential applications based on binocular luster.

A parsimonious model of brightness induction

We present a parsimonious model of brightness induction which can account for various brightness illusions of both brightness-contrast and brightness-assimilation types. Our model is based on a difference of difference-of-Gaussian filter and a two-pass model of attentive vision based on the parallel channels in the central visual pathway. It overcomes some of the problems that could not be addressed by the well-known oriented difference of Gaussian model like those associated with Mach band and checkerboard illusions. This model attempts to provide insight to the mechanism of attention in brightness perception through the two major complimentary visual channels, viz. the magnocellular and the parvocellular.

Bioplausible multiscale filtering in retino-cortical processing as a mechanism in perceptual grouping

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, 'Geometrical' and, in particular, 'Tilt Illusions' are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. The model is a variation of classical receptive field implementation for simple cells in early stages of vision with the scales tuned to the object/texture sizes in the pattern. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as 'Anchoring theory' and 'Perceptual grouping'.

Saliency computation via whitened frequency band selection

Many saliency computational models have been proposed to simulate bottom-up visual attention mechanism of human visual system. However, most of them only deal with certain kinds of images or aim at specific applications. In fact, human beings have the ability to correctly select attentive focuses of objects with arbitrary sizes within any scenes. This paper proposes a new bottom-up computational model from the perspective of frequency domain based on the biological discovery of non-Classical Receptive Field (nCRF) in the retina. A saliency map can be obtained according to the idea of Extended Classical Receptive Field. The model is composed of three major steps: firstly decompose the input image into several feature maps representing different frequency bands that cover the whole frequency domain by utilizing Gabor wavelet. Secondly, whiten the feature maps to highlight the embedded saliency information. Thirdly, select some optimal maps, simulating the response of receptive field especially nCRF, to generate the saliency map. Experimental results show that the proposed algorithm is able to work with stable effect and outstanding performance in a variety of situations as human beings do and is adaptive to both psychological patterns and natural images. Beyond that, biological plausibility of nCRF and Gabor wavelet transform make this approach reliable.

A DOG filter model of the occurrence of Mach bands on spatial contrast discontinuities

The present work proposes a unified model to explain two previously reported properties of the Mach band illusion. The first is the frequently referenced fact that Mach bands are prominently visible at ramps, but practically vanish at intensity steps. The second property, less studied, on the other hand may also be related to the first. It concerns the fact that the width of the illusory Mach bands appears to be a function of the slope of the ramp itself. The model proposed here combines the difference of Gaussians (DOG) model of lateral inhibition in receptive fields with the models of feature detection, based on a holistic approach. The sharpness of discontinuity (SOD) concept for Mach band stimulus has been defined and is related to the slope of the ramp. It is suggested that calculation of SOD leads to an adaptive change in inhibitory surround, a notion that has the support of physiological experiments too.

An exponential filter model predicts lightness illusions

Lightness, or perceived reflectance of a surface, is influenced by surrounding context. This is demonstrated by the Simultaneous Contrast Illusion (SCI), where a gray patch is perceived lighter against a black background and vice versa. Conversely, assimilation is where the lightness of the target patch moves toward that of the bounding areas and can be demonstrated in White's effect. Blakeslee and McCourt (1999) introduced an oriented difference-of-Gaussian (ODOG) model that is able to account for both contrast and assimilation in a number of lightness illusions and that has been subsequently improved using localized normalization techniques. We introduce a model inspired by image statistics that is based on a family of exponential filters, with kernels spanning across multiple sizes and shapes. We include an optional second stage of normalization based on contrast gain control. Our model was tested on a well-known set of lightness illusions that have previously been used to evaluate ODOG and its variants, and model lightness values were compared with typical human data. We investigate whether predictive success depends on filters of a particular size or shape and whether pooling information across filters can improve performance. The best single filter correctly predicted the direction of lightness effects for 21 out of 27 illusions. Combining two filters together increased the best performance to 23, with asymptotic performance at 24 for an arbitrarily large combination of filter outputs. While normalization improved prediction magnitudes, it only slightly improved overall scores in direction predictions. The prediction performance of 24 out of 27 illusions equals that of the best performing ODOG variant, with greater parsimony. Our model shows that V1-style orientation-selectivity is not necessary to account for lightness illusions and that a low-level model based on image statistics is able to account for a wide range of both contrast and assimilation effects.

A possible role and basis of visual pathway selection in brightness induction

It is a well-known fact that the perceived brightness of any surface depends on the brightness of the surfaces that surround it. This phenomenon is termed as brightness induction. Isotropic arrays of multi-scale DoG (Difference of Gaussians) as well as cortical Oriented DoG (ODOG) and extensions thereof, like the Frequency-specific Locally Normalized ODOG (FLODOG) functions have been employed towards prediction of the direction of brightness induction in many brightness perception effects. But the neural basis of such spatial filters is seldom obvious. For instance, the visual information from retinal ganglion cells to such spatial filters, which have been generally speculated to appear at the early stage of cortical processing, are fed by at least three parallel channels viz. Parvocellular (P), Magnocellular (M) and Koniocellular (K) in the subcortical pathway, but the role of such pathways in brightness induction is generally not implicit. In this work, three different spatial filters based on an extended classical receptive field (ECRF) model of retinal ganglion cells, have been approximately related to the spatial contrast sensitivity functions of these three parallel channels. Based on our analysis involving different brightness perception effects, we propose that the M channel, with maximum conduction velocity, may have a special role for an initial sensorial perception. As a result, brightness assimilation may be the consequence of vision at a glance through the M pathway; contrast effect may be the consequence of a subsequent vision with scrutiny through the P channel; and the K pathway response may represent an intermediate situation resulting in ambiguity in brightness perception. The present work attempts to correlate this phenomenon of pathway selection with the complementary nature of these channels in terms of spatial frequency as well as contrast.

Compact image representation model based on both nCRF and reverse control mechanisms

The aim of this paper is to construct a bio-inspired hierarchical neural network that could accurately represent visual images and facilitate follow-up processing. Our computational model adopted a ganglion cell (GC) mechanism with a receptive field that dynamically self-adjusts according to the characteristics of an input image. For each GC, a micro neural circuit and a reverse control circuit were developed to self-adaptively resize the receptive field. An array was also designed to imitate the layer of GCs that perform image representation. Results revealed that this GC array could represent images from the external environment with a low processing cost, and this nonclassical receptive field mechanism could substantially improve both segmentation and integration processing. This model enables automatic extraction of blocks from images, which makes multiscale representation feasible. Importantly, once an original pixel-level image was reorganized into a GC array, semantic-level features emerged. Because GCs, like symbols, are discrete and separable, this GC-grained compact representation is open to operations that can manipulate images partially and selectively. Thus, the GC-array model provides a basic infrastructure and allows for high-level image processing.

A possible mechanism of stochastic resonance in the light of an extra-classical receptive field model of retinal ganglion cells

Traditionally the intensity discontinuities in an image are detected as zero-crossings of the second derivative with the help of a Laplacian of Gaussian (LOG) operator that models the receptive field of retinal Ganglion cells. Such zero-crossings supposedly form a raw primal sketch edge map of the external world in the primary visual cortex of the brain. Based on a new operator which is a linear combination of the LOG and a Dirac-delta function that models the extra-classical receptive field of the ganglion cells, we find that zero-crossing points thus generated, store in presence of noise, apart from the edge information, the shading information of the image in the form of density variation of these points. We have also shown that an optimal image contrast produces best mapping of the shading information to such zero-crossing density variation for a given amount of noise contamination. Furthermore, we have observed that an optimal amount of noise contamination reproduces the minimum optimal contrast and hence gives rise to the best representation of the original image. We show that this phenomenon is similar in nature to that of stochastic resonance phenomenon observed in psychophysical experiments.

Retinomorphic image processing

The present work is aimed at understanding and explaining some of the aspects of visual signal processing at the retinal level while exploiting the same towards the development of some simple techniques in the domain of digital image processing. Classical studies on retinal physiology revealed the nature of contrast sensitivity of the receptive field of bipolar or ganglion cells, which lie in the outer and inner plexiform layers of the retina. To explain these observations, a difference of Gaussian (DOG) filter was suggested, which was subsequently modified to a Laplacian of Gaussian (LOG) filter for computational ease in handling two-dimensional retinal inputs. Till date almost all image processing algorithms, used in various branches of science and engineering had followed LOG or one of its variants. Recent observations in retinal physiology however, indicate that the retinal ganglion cells receive input from a larger area than the classical receptive fields. We have proposed an isotropic model for the non-classical receptive field of the retinal ganglion cells, corroborated from these recent observations, by introducing higher order derivatives of Gaussian expressed as linear combination of Gaussians only. In digital image processing, this provides a new mechanism of edge detection on one hand and image half-toning on the other. It has also been found that living systems may sometimes prefer to "perceive" the external scenario by adding noise to the received signals in the pre-processing level for arriving at better information on light and shade in the edge map. The proposed model also provides explanation to many brightness-contrast illusions hitherto unexplained not only by the classical isotropic model but also by some other Gestalt and Constructivist models or by non-isotropic multi-scale models. The proposed model is easy to implement both in the analog and digital domain. A scheme for implementation in the analog domain generates a new silicon retina model implemented on a hardware development platform.