Spatial filtering versus anchoring accounts of brightness/lightness perception in staircase and simultaneous brightness/lightness contrast stimuli

Fixational eye movements and edge integration in lightness perception

A neural theory of human lightness computation is described and computer-simulated. The theory proposes that lightness is derived from transient ON and OFF cell responses in the early visual pathways that have different characteristic neural gains and that are generated by fixational eye movements (FEMs) as the eyes transit luminance edges in the image. The ON and OFF responses are combined with corollary discharge signals that encode the eye movement direction to create directionally selective ON and OFF responses. Cortical neurons with large-scale receptive fields independently integrate the outputs of all of the directional ON or OFF responses whose associated eye movement directions point towards their receptive field centers, with a spatial weighting determined by the receptive field profile. Lightness is computed by subtracting the spatially integrated OFF activity from spatially integrated ON activity and normalizing the difference signal so that the maximum response in the spatial lightness map at any given time equals a fixed activation level corresponding to the percept of white. Two different mechanisms for ON and OFF cells responses are considered and simulated, and both are shown to produce an overall lightness model that explains a host of quantitative and qualitative lightness phenomena, including the Staircase Gelb and related illusions, failures of lightness constancy in the simultaneous contrast illusion, Chevreul's illusion, lightness filling-in, and perceptual fading of stabilized images. The neural plausibility of the two variants of the theory, as well as its implication for lightness constancy and failures of lightness constancy are discussed.

Neurocomputational Lightness Model Explains the Appearance of Real Surfaces Viewed Under Gelb Illumination

One of the primary functions of visual perception is to represent, estimate, and evaluate the properties of material surfaces in the visual environment. One such property is surface color, which can convey important information about ecologically relevant object characteristics such as the ripeness of fruit and the emotional reactions of humans in social interactions. This paper further develops and applies a neural model (Rudd, 2013, 2017) of how the human visual system represents the light/dark dimension of color-known as lightness-and computes the colors of achromatic material surfaces in real-world spatial contexts. Quantitative lightness judgments conducted with real surfaces viewed under Gelb (i.e., spotlight) illumination are analyzed and simulated using the model. According to the model, luminance ratios form the inputs to ON- and OFF-cells, which encode local luminance increments and decrements, respectively. The response properties of these cells are here characterized by physiologically motivated equations in which different parameters are assumed for the two cell types. Under non-saturating conditions, ON-cells respond in proportion to a compressive power law of the local incremental luminance in the image that causes them to respond, while OFF-cells respond linearly to local decremental luminance. ON- and OFF-cell responses to edges are log-transformed at a later stage of neural processing and then integrated across space to compute lightness via an edge integration process that can be viewed as a neurally elaborated version of Land's retinex model (Land & McCann, 1971). It follows from the model assumptions that the perceptual weights-interpreted as neural gain factors-that the model observer applies to steps in log luminance at edges in the edge integration process are determined by the product of a polarity-dependent factor 1-by which incremental steps in log luminance (i.e., edges) are weighted by the value <1.0 and decremental steps are weighted by 1.0-and a distance-dependent factor 2, whose edge weightings are estimated to fit perceptual data. The model accounts quantitatively (to within experimental error) for the following: lightness constancy failures observed when the illumination level on a simultaneous contrast display is changed (Zavagno, Daneyko, & Liu, 2018); the degree of dynamic range compression in the staircase-Gelb paradigm (Cataliotti & Gilchrist, 1995; Zavagno, Annan, & Caputo, 2004); partial releases from compression that occur when the staircase-Gelb papers are reordered (Zavagno, Annan, & Caputo, 2004); and the larger compression release that occurs when the display is surrounded by a white border (Gilchrist & Cataliotti, 1994).

The Oriented Difference of Gaussians (ODOG) model of brightness perception: Overview and executable Mathematica notebooks

The Oriented Difference of Gaussians (ODOG) model of brightness (perceived intensity) by Blakeslee and McCourt (Vision Research 39:4361-4377, 1999), which is based on linear spatial filtering by oriented receptive fields followed by contrast normalization, has proven highly successful in parsimoniously predicting the perceived intensity (brightness) of regions in complex visual stimuli such as White's effect, which had been believed to defy filter-based explanations. Unlike competing explanations such as anchoring theory, filling-in, edge-integration, or layer decomposition, the spatial filtering approach embodied by the ODOG model readily accounts for the often overlooked but ubiquitous gradient structure of induction which, while most striking in grating induction, also occurs within the test fields of classical simultaneous brightness contrast and the White stimulus. Also, because the ODOG model does not require defined regions of interest, it is generalizable to any stimulus, including natural images. The ODOG model has motivated other researchers to develop modified versions (LODOG and FLODOG), and has served as an important counterweight and proof of concept to constrain high-level theories which rely on less well understood or justified mechanisms such as unconscious inference, transparency, perceptual grouping, and layer decomposition. Here we provide a brief but comprehensive description of the ODOG model as it has been implemented since 1999, as well as working Mathematica (Wolfram, Inc.) notebooks which users can employ to generate ODOG model predictions for their own stimuli.

Standard deviation of luminance distribution affects lightness and pupillary response

We examined whether the standard deviation (SD) of luminance distribution serves as information of illumination. We measured the lightness of a patch presented in the center of a scrambled-dot pattern while manipulating the SD of the luminance distribution. Results showed that lightness decreased as the SD of the surround stimulus increased. We also measured pupil diameter while viewing a similar stimulus. The pupil diameter decreased as the SD of luminance distribution of the stimuli increased. We confirmed that these results were not obtained because of the increase of the highest luminance in the stimulus. Furthermore, results of field measurements revealed a correlation between the SD of luminance distribution and illuminance in natural scenes. These results indicated that the visual system refers to the SD of the luminance distribution in the visual stimulus to estimate the scene illumination.

A cortical edge-integration model of object-based lightness computation that explains effects of spatial context and individual differences

Previous work has demonstrated that perceived surface reflectance (lightness) can be modeled in simple contexts in a quantitatively exact way by assuming that the visual system first extracts information about local, directed steps in log luminance, then spatially integrates these steps along paths through the image to compute lightness (Rudd and Zemach, 2004, 2005, 2007). This method of computing lightness is called edge integration. Recent evidence (Rudd, 2013) suggests that human vision employs a default strategy to integrate luminance steps only along paths from a common background region to the targets whose lightness is computed. This implies a role for gestalt grouping in edge-based lightness computation. Rudd (2010) further showed the perceptual weights applied to edges in lightness computation can be influenced by the observer's interpretation of luminance steps as resulting from either spatial variation in surface reflectance or illumination. This implies a role for top-down factors in any edge-based model of lightness (Rudd and Zemach, 2005). Here, I show how the separate influences of grouping and attention on lightness can be modeled in tandem by a cortical mechanism that first employs top-down signals to spatially select regions of interest for lightness computation. An object-based network computation, involving neurons that code for border-ownership, then automatically sets the neural gains applied to edge signals surviving the earlier spatial selection stage. Only the borders that survive both processing stages are spatially integrated to compute lightness. The model assumptions are consistent with those of the cortical lightness model presented earlier by Rudd (2010, 2013), and with neurophysiological data indicating extraction of local edge information in V1, network computations to establish figure-ground relations and border ownership in V2, and edge integration to encode lightness and darkness signals in V4.

Reversing the Reversed Contrast

Galmonte and Agostini (1998 Investigative Ophthalmology & Visual Science, 39(4), S158), Agostini and Galmonte (2002 Psychological Science, 13, 89–93), Bressan (2001 Perception, 30, 1031–1046), and Gilchrist and Annan (2002 Perception, 31, 141–150) reported three different lightness contrast configurations in which grouping factors make a gray target totally surrounded by black appear darker than an equal gray target surrounded by white, reversing the classical contrast effect. In this paper we demonstrate that the three configurations known as ‘reversed contrast’ are based on different mechanisms. Sixteen participants judged the lightness of the gray targets of the original and modified versions of the three configurations. Our results highlight that the Agostini and Galmonte effect is reversed when the global grouping factors are removed, while in a number of variations of Bressan's and Gilchrist and Annan's displays the direction of the effect does not change, even in absence of global grouping factors. Our results indicate that the factors determining the Agostini and Galmonte effect are different from those acting on the other two configurations, in which the lightness change is also due to factors other than belongingness.

Linking appearance to neural activity through the study of the perception of lightness in naturalistic contexts

The present paper deals with the classical question how a psychological experience, in this case apparent lightness, is linked by intervening neural processing to physical variables. We address two methodological issues: (a) how does one know the appropriate physical variable (what is the right x?) to look at, and (b) how can behavioral measurements be used to probe the internal transformation that leads to psychological experience. We measured so-called lightness transfer functions (LTFs), that is the functions that describe the mapping between retinal luminance and perceived lightness for naturalistic checkerboard stimuli. The LTFs were measured for different illumination situations: plain view, a cast shadow, and an intervening transparent medium. Observers adjusted the luminance of a comparison patch such that it had the same lightness as each of the test patches. When the data were plotted in luminance-luminance space, we found qualitative differences between mapping functions in different contexts. These differences were greatly diminished when the data were plotted in terms of contrast. On contrast-contrast coordinates, the data were compatible with a single linear generative model. This result is an indication that, for the naturalistic scenes used here, lightness perception depends mostly on local contrast. We further discuss that, in addition to the mean adjustments, one may find it useful to consider also the variability of an observer's adjustments in order to infer the true luminance-to-lightness mapping function.

Orientation enhancement in early visual processing can explain time course of brightness contrast and White's illusion

Dynamics of orientation tuning in V1 indicates that computational model of V1 should not only comprise of bank of static spatially oriented filters but also include the contribution for dynamical response facilitation or suppression along orientation. Time evolution of orientation response in V1 can emerge due to time- dependent excitation and lateral inhibition in the orientation domain. Lateral inhibition in the orientation domain suggests that Ernst Mach's proposition can be applied for the enhancement of initial orientation distribution that is generated due to interaction of visual stimulus with spatially oriented filters and subcortical temporal filter. Oriented spatial filtering that appears much early (<70 ms) in the sequence of visual information processing can account for many of the brightness illusions observed at steady state. It is therefore expected that time evolution of orientation response might be reflecting in the brightness percept over time. Our numerical study suggests that only spatio-temporal filtering at early phase can explain experimentally observed temporal dynamics of brightness contrast illusion. But, enhancement of orientation response at early phase of visual processing is the key mechanism that can guide visual system to predict the brightness by "Max-rule" or "Winner Takes All" (WTA) estimation and thus producing White's illusions at any exposure.

When is spatial filtering enough? Investigation of brightness and lightness perception in stimuli containing a visible illumination component

Brightness (perceived intensity) and lightness (perceived reflectance) matching were investigated in seven well-known visual stimuli that contain a visible shadow or transparent overlay. These stimuli are frequently upheld as evidence that low-level spatial filtering is inadequate to explain brightness/lightness illusions and that additional mid- or high-level mechanisms are required. The argument in favor of rejecting low-level spatial filtering explanations has been founded on the erroneous assumption that equating test patch and near surround luminance is sufficient to control for and rule out this type of mechanism. We tested this idea by comparing the matching behavior of four observers to the predictions of the ODOG multiscale filtering model (Blakeslee & McCourt, 1999). Lightness and brightness matching differed significantly only when test patches appeared in shadow or beneath a transparency. Lightness and brightness matches were both significantly larger under these conditions; however, the lightness matches greatly exceeded the brightness matches. Lightness matches were greater for test patches in shadow or beneath a transparency because lightness matches under these conditions were based on conscious inferential (not sensory-level) judgments where observers attempted to discount the difference in illumination. The ODOG model accounted for approximately 80% of the total variance in the brightness matches (as well as in the lightness matches for targets not in shadow or beneath a transparency), and successfully predicted the relative magnitude of these matches in five of the seven stimulus sets. These results indicate that multiscale spatial filtering provides a unified and parsimonious explanation for brightness perception in these stimuli and imply that higher-level mechanisms are not required to explain them. The model was not as successful for the argyle and wall of blocks illusions in that it incorrectly rank-ordered the relative magnitude of the effects across different versions of the stimuli. It is an important question whether such model failures are due to known but corrigible limitations of the ODOG model or whether they will require other (possibly higher-level) explanations.

Spatiotemporal analysis of brightness induction

Brightness induction refers to a class of visual illusions in which the perceived intensity of a region of space is influenced by the luminance of surrounding regions. These illusions are significant because they provide insight into the neural organization of the visual system. A novel quadrature-phase motion cancelation technique was developed to measure the magnitude of the grating induction brightness illusion across a wide range of spatial frequencies, temporal frequencies and test field heights. Canceling contrast is greatest at low frequencies and declines with increasing frequency in both dimensions, and with increasing test field height. Canceling contrast scales as the product of inducing grating spatial frequency and test field height (the number of inducing grating cycles per test field height). When plotted using a spatial axis which indexes this product, the spatiotemporal induction surfaces for four test field heights can be described as four partially overlapping sections of a single larger surface. These properties of brightness induction are explained in the context of multiscale spatial filtering. The present study is the first to measure the magnitude of grating induction as a function of temporal frequency. Taken in conjunction with several other studies (Blakeslee & McCourt, 2008; Magnussen & Glad, 1975; Robinson & de Sa, 2008) the results of this study illustrate that at least one form of brightness induction is very much faster than that reported by DeValois, Webster, DeValois, and Lingelbach (1986) and Rossi and Paradiso (1996), and are inconsistent with the proposition that brightness induction results from a slow "filling in" process.

Measuring the meter: on the constancy of lightness scales seen against different backgrounds

The constancy of a 16-step achromatic Munsell scale was tested with regards to background variations in two experiments. In experiment 1 three groups of observers were asked to find lightness matches for targets in simultaneous lightness displays by using a 16-step achromatic Munsell scale placed on a white, black, or white-black checkered background. In experiment 2, a yellow-blue checkered background and a green-red checkered background replaced Munsell scales on the black and on the white backgrounds. Significant effects of scale background on matches were found only in experiment 1, suggesting that background luminance is a crucial factor in the overall appearance of the scale. The lack of significant differences in experiment 2, however, may stand for an overall robustness of the scale with respect to background luminance changes occurring within certain luminance ranges.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.