A multiscale filtering explanation of gradient induction and remote brightness induction effects: a reply to Logvinenko (2003)

A Compound Computational Model for Filling-In Processes Triggered by Edges: Watercolor Illusions

The goal of our research was to develop a compound computational model with the ability to predict different variations of the "watercolor effects" and additional filling-in effects that are triggered by edges. The model is based on a filling-in mechanism solved by a Poisson equation, which considers the different gradients as "heat sources" after the gradients modification. The biased (modified) contours (edges) are ranked and determined according to their dominancy across the different chromatic and achromatic channels. The color and intensity of the perceived surface are calculated through a diffusive filling-in process of color triggered by the enhanced and biased edges of stimulus formed as a result of oriented double-opponent receptive fields. The model can successfully predict both the assimilative and non-assimilative watercolor effects, as well as a number of "conflicting" visual effects. Furthermore, the model can also predict the classic Craik-O'Brien-Cornsweet (COC) effect. In summary, our proposed computational model is able to predict most of the "conflicting" filling-in effects that derive from edges that have been recently described in the literature, and thus supports the theory that a shared visual mechanism is responsible for the vast variety of the "conflicting" filling-in effects that derive from edges.

Remote controls illusion: strange interactions across space cannot be explained by simple contrast filters

The visual system has separable visual encoding for luminance and for contrast modulation [J. Vis.8(1), B152 (2008)1534-736210.1167/8.6.1]; the two dimensions can be represented with a luminance contrast versus luminance plane. Here we use a contrast asynchrony paradigm to explore contextual effects on luminance contrast modulation: two identical rectangular bars (0.5°×2.5°) have luminance levels that modulate at 2 Hz; when one bar is placed on a bright field and the other bar on a dark field, observers perceive the bars modulating in antiphase with each other and yet becoming light and dark at the same time. The antiphase perception corresponds to the change in contrast between the bars and their surrounds (a change along the contrast axis of the plane); the in-phase perception corresponds to the luminance modulation (a change along the luminance axis of the plane). We examine spatial interaction by adding bright rectangular (0.5°×2.5°) flankers on both sides of the dark-field bar and dark flankers on both sides of the bright-field bar. Remarkably, flankers produce an in-phase appearance when separated from the bars by between 2^' and 4^' of visual angle, and produce antiphase appearance when they directly adjoin the bars or are separated by more than 8^'. To estimate the dimensions of the spatial interaction, we parametrically adjust the size of the gap between bars and flankers and the length of the flankers. We attempt to account for the results with models based on rectified difference of Gaussian filters and with rectified oriented difference of Gaussian filters. The models can account for the results when the flankers are the same height as bars, but are unable to account for the effects of increasing the flanker length. The models therefore suggest that the spatial interaction across distances requires more complex interactions of contrast filters.

The Oriented Difference of Gaussians (ODOG) model of brightness perception: Overview and executable Mathematica notebooks

The Oriented Difference of Gaussians (ODOG) model of brightness (perceived intensity) by Blakeslee and McCourt (Vision Research 39:4361-4377, 1999), which is based on linear spatial filtering by oriented receptive fields followed by contrast normalization, has proven highly successful in parsimoniously predicting the perceived intensity (brightness) of regions in complex visual stimuli such as White's effect, which had been believed to defy filter-based explanations. Unlike competing explanations such as anchoring theory, filling-in, edge-integration, or layer decomposition, the spatial filtering approach embodied by the ODOG model readily accounts for the often overlooked but ubiquitous gradient structure of induction which, while most striking in grating induction, also occurs within the test fields of classical simultaneous brightness contrast and the White stimulus. Also, because the ODOG model does not require defined regions of interest, it is generalizable to any stimulus, including natural images. The ODOG model has motivated other researchers to develop modified versions (LODOG and FLODOG), and has served as an important counterweight and proof of concept to constrain high-level theories which rely on less well understood or justified mechanisms such as unconscious inference, transparency, perceptual grouping, and layer decomposition. Here we provide a brief but comprehensive description of the ODOG model as it has been implemented since 1999, as well as working Mathematica (Wolfram, Inc.) notebooks which users can employ to generate ODOG model predictions for their own stimuli.

Luminance gradient configuration determines perceived lightness in a simple geometric illusion

Accurate perception of surface reflectance poses a significant computational problem for the visual system. The amount of light reflected by a surface is affected by a combination of factors including the surface's reflectance properties and illumination conditions. The latter are not limited by the strength of the illuminant but also include the relative placement of the light illuminating the surface, the orientation of the surface and its 3d shape, all of which result in a pattern of luminance gradients across the surface. In this study we explore how luminance gradients contribute to lightness perception. We introduce a novel, simple lightness illusion. It consists of six separate checks, organized in rows of two. Each check has a negative luminance gradient across it. The top and the bottom rows are the same: with the darker check on the left, and the lighter check on the right. Two checks in the middle row are identical; however, the check on the right appears darker than the check on the left. As there are no shared borders between the checks, simultaneous contrast cannot explain the effect. However, there are multiple possible explanations including spatial filtering (Blakeslee and McCourt, 2004) or some higher-order mechanism such as perceptual grouping or amodal completion. Here, we explore these possibilities by manipulating the luminance configurations and the gradient slopes of the checks.

The roles of image decomposition and edge curvature in the 'snake' lightness illusion

The snake illusion is an effect in which the lightness of target patches is strongly affected by the luminance of remote patches. One explanation is that such images are decomposed into a pattern of illumination and a pattern of reflectance, involving a classification of luminance edges into illumination and reflectance edges. Based on this decomposition, perceived reflectance is determined by discounting the illumination. A problem for this account is that image decomposition is not unique, and that different decompositions may lead to different lightness predictions. One way to rule out alternative decompositions and ensure correct predictions is to postulate that the visual system tends to classify curved luminance edges as reflectance edges rather than illumination edges. We have constructed several variations of the basic snake display in order to test the proposed curvature constraint and the more general image decomposition hypothesis. Although the results from some displays have confirmed previous findings of the effect of curvature, the general pattern of data questions the relevance of the shape of luminance edges for the determination of lightness in this class of displays. The data also argue against an image decomposition mechanism as an explanation of this effect. As an alternative, a tentative neurally based account is sketched.

Brightness induction magnitude declines with increasing distance from the inducing field edge

Brightness induction refers to a class of visual illusions where the perceived intensity of a region of space is influenced by the luminance of surrounding regions. These illusions are significant because they provide insight into the neural organization and processing strategies employed by the visual system. The nature of these processing strategies, however, has long been debated. Here we investigate the spatial characteristics of grating induction as a function of the distance from the inducing field edge to evaluate the viability of various competing models. In particular multiscale spatial filtering models and homogeneous filling-in models make very different predictions in regard to the magnitude of induction as a function of this distance. Filling-in explanations predict that the brightness/lightness of the filled-in region will be homogeneous, whereas multiscale filtering predicts a fall-off in induction magnitude with distance from the inducing field edge. Induction magnitude was measured using a narrow probe version of the quadrature-phase motion-cancellation paradigm (Blakeslee & McCourt, 2011) and a point-by-point brightness matching paradigm (Blakeslee & McCourt, 1997, 1999; McCourt, 1994). Both techniques reveal a decrease in the magnitude of induction with increasing distance from the inducing edge. A homogeneous filling-in mechanism cannot explain the induced structure in the test fields of these stimuli. The results argue strongly against filling-in mechanisms as well as against any mechanism that posits that induction is homogeneous. The structure of the induction is, however, well accounted for by the multiscale filtering (ODOG) model of Blakeslee and McCourt (1999). These results support models of brightness/lightness, such as filtering models, which preserve these gradients of induction.

When is spatial filtering enough? Investigation of brightness and lightness perception in stimuli containing a visible illumination component

Brightness (perceived intensity) and lightness (perceived reflectance) matching were investigated in seven well-known visual stimuli that contain a visible shadow or transparent overlay. These stimuli are frequently upheld as evidence that low-level spatial filtering is inadequate to explain brightness/lightness illusions and that additional mid- or high-level mechanisms are required. The argument in favor of rejecting low-level spatial filtering explanations has been founded on the erroneous assumption that equating test patch and near surround luminance is sufficient to control for and rule out this type of mechanism. We tested this idea by comparing the matching behavior of four observers to the predictions of the ODOG multiscale filtering model (Blakeslee & McCourt, 1999). Lightness and brightness matching differed significantly only when test patches appeared in shadow or beneath a transparency. Lightness and brightness matches were both significantly larger under these conditions; however, the lightness matches greatly exceeded the brightness matches. Lightness matches were greater for test patches in shadow or beneath a transparency because lightness matches under these conditions were based on conscious inferential (not sensory-level) judgments where observers attempted to discount the difference in illumination. The ODOG model accounted for approximately 80% of the total variance in the brightness matches (as well as in the lightness matches for targets not in shadow or beneath a transparency), and successfully predicted the relative magnitude of these matches in five of the seven stimulus sets. These results indicate that multiscale spatial filtering provides a unified and parsimonious explanation for brightness perception in these stimuli and imply that higher-level mechanisms are not required to explain them. The model was not as successful for the argyle and wall of blocks illusions in that it incorrectly rank-ordered the relative magnitude of the effects across different versions of the stimuli. It is an important question whether such model failures are due to known but corrigible limitations of the ODOG model or whether they will require other (possibly higher-level) explanations.

Microgenesis of surface completion in visual objects: evidence for filling-out

Using metacontrast masking we examined the temporal dynamics of surface completion in object vision. By varying the stimulus onset asynchrony between the target object and the flanking mask(s), we obtained estimates of the time required for the entire surface contrast to fill out within the area delimited by the contours/edges of the target. The estimated speed of the filling-out process was 36.0 deg/s. Using existing estimates of cortical magnification, the computed filling-out speed in terms of cortical distance is .385 m/s, a value that approximates the estimated cortical filling-in speed and the speed of horizontal propagation in monkey V1. We discuss our results in relation to (1) prior findings of filling-in and filling-out phenomena, using surface completion in cortical space as the unifying principle, and (2) extant computational models of object vision.

Spatiotemporal analysis of brightness induction

Brightness induction refers to a class of visual illusions in which the perceived intensity of a region of space is influenced by the luminance of surrounding regions. These illusions are significant because they provide insight into the neural organization of the visual system. A novel quadrature-phase motion cancelation technique was developed to measure the magnitude of the grating induction brightness illusion across a wide range of spatial frequencies, temporal frequencies and test field heights. Canceling contrast is greatest at low frequencies and declines with increasing frequency in both dimensions, and with increasing test field height. Canceling contrast scales as the product of inducing grating spatial frequency and test field height (the number of inducing grating cycles per test field height). When plotted using a spatial axis which indexes this product, the spatiotemporal induction surfaces for four test field heights can be described as four partially overlapping sections of a single larger surface. These properties of brightness induction are explained in the context of multiscale spatial filtering. The present study is the first to measure the magnitude of grating induction as a function of temporal frequency. Taken in conjunction with several other studies (Blakeslee & McCourt, 2008; Magnussen & Glad, 1975; Robinson & de Sa, 2008) the results of this study illustrate that at least one form of brightness induction is very much faster than that reported by DeValois, Webster, DeValois, and Lingelbach (1986) and Rossi and Paradiso (1996), and are inconsistent with the proposition that brightness induction results from a slow "filling in" process.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.

Spatial filtering versus anchoring accounts of brightness/lightness perception in staircase and simultaneous brightness/lightness contrast stimuli

J. Cataliotti and A. Gilchrist (1995) reported that, consistent with anchoring theory, the lightness of a black step in a reflectance staircase was not altered by moving a white step from a remote to an adjacent location. Recently, E. Economou, S. Zdravkovic, and A. Gilchrist (2007) reported data supporting three additional predictions of the anchoring model (A. Gilchrist et al., 1999): 1) equiluminant incremental targets in staircase simultaneous lightness contrast stimuli appeared equally light; 2) the simultaneous lightness contrast effect was due mainly to the lightening of the target on the black surround; and 3) the strength of lightness induction was greatest for darker targets. We investigated similar stimuli using brightness/lightness matching and found, contrary to these reports, that: 1) the relative position of the steps in a luminance staircase significantly influenced their brightness/lightness; 2) equiluminant incremental targets in staircase simultaneous brightness/lightness contrast stimuli did not all appear equally bright/light; 3) an asymmetry due to a greater brightening/lightening of the target on the black surround was not general; and 4) darker targets produced larger effects only when plotted on a log scale. In addition, the ODOG model (B. Blakeslee & M. E. McCourt, 1999) did an excellent job of accounting for brightness/lightness matching in these stimuli.

Oriented multiscale spatial filtering and contrast normalization: a parsimonious model of brightness induction in a continuum of stimuli including White, Howe and simultaneous brightness contrast

The White effect [Perception 8 (1979) 413] cannot be simply explained as due to either brightness contrast or brightness assimilation because the direction of the induced brightness change does not correlate with the amount of black or white border in contact with the gray test patch. This has led some investigators to abandon spatial filtering explanations not only for the White effect but for brightness perception in general. Offered instead are explanations based on a variety of junction analyses and/or perceptual organization schemes which in the case of the White effect are usually based on T-junctions. Recently, Howe [Perception 30 (2001) 1023] challenged T-junction based explanations with a novel variation of White's effect in which the T-junctions were constant while the brightness effect was eliminated or reversed, and proposed an alternative explanation in terms of illusory contours. The present study argues that an analysis at the level of illusory contours is not necessary and that a much simpler spatial filtering based explanation is sufficient. Brightness induction was measured in a set of stimuli chosen to illustrate the relationship between the Howe stimulus [Perception 30 (2001) 1023], the White stimulus [Perception 8 (1979) 413] and the classical simultaneous brightness contrast (SBC) stimulus. The White stimulus and the SBC stimulus occupy opposite ends of a continuum of stimuli in which the Howe stimulus is the mid-point. The psychophysical measurements were compared with the predictions of the oriented difference-of-Gaussians (ODOG) computational model of Blakeslee and McCourt [Vision Research 39 (1999) 4361]. The ODOG model parsimoniously accounted for both the direction and relative magnitude of the brightness effects suggesting that more complex mechanisms are not required to explain them.