A multiscale spatial filtering account of the Wertheimer-Benary effect and the corrugated Mondrian

Empirical evaluation of computational models of lightness perception

Lightness of a surface depends not only on its physical characteristics, but also on the properties of the surrounding context. As a result, varying the context can significantly alter surface lightness, an effect exploited in many lightness illusions. Computational models can produce outcomes similar to human illusory percepts, allowing for demonstrable assessment of the applied mechanisms and principles. We tested 8 computational models on 13 typical displays used in lightness research (11 Illusions and 2 Mondrians), and compared them with results from human participants (N = 85). Results show that HighPass and MIR models predict empirical results for simultaneous lightness contrast (SLC) and its close variations. ODOG and its newer variants (ODOG-2 and L-ODOG) in addition to SLC displays were able to predict effect of White's illusion. RETINEX was able to predict effects of both SLC displays and Dungeon illusion. Dynamic decorrelation model was able to predict obtained effects for all tested stimuli except two SLC variations. Finally, FL-ODOG model was best at simulating human data, as it was able to predict empirical results for all displays, bar the Reversed contrast illusion. Finally, most models underperform on the Mondrian displays that represent most natural stimuli for the human visual system.

Layer and framework theories of lightness

Lightness (the perceived dimension running from black to white) represents a problem for vision science because the light coming to the eye from an object totally fails to specify the shade of gray of the object, due to the confounding of surface gray and illumination intensity. The two leading approaches, decomposition theories and anchoring theories, split the retinal image into overlapping layers and adjacent frameworks, respectively. Because each approach has important strengths and some weaknesses, an integration of them would mark an important step forward for the lightness theory. But the problem remains how this integration can actually be realized.

A Compound Computational Model for Filling-In Processes Triggered by Edges: Watercolor Illusions

The goal of our research was to develop a compound computational model with the ability to predict different variations of the "watercolor effects" and additional filling-in effects that are triggered by edges. The model is based on a filling-in mechanism solved by a Poisson equation, which considers the different gradients as "heat sources" after the gradients modification. The biased (modified) contours (edges) are ranked and determined according to their dominancy across the different chromatic and achromatic channels. The color and intensity of the perceived surface are calculated through a diffusive filling-in process of color triggered by the enhanced and biased edges of stimulus formed as a result of oriented double-opponent receptive fields. The model can successfully predict both the assimilative and non-assimilative watercolor effects, as well as a number of "conflicting" visual effects. Furthermore, the model can also predict the classic Craik-O'Brien-Cornsweet (COC) effect. In summary, our proposed computational model is able to predict most of the "conflicting" filling-in effects that derive from edges that have been recently described in the literature, and thus supports the theory that a shared visual mechanism is responsible for the vast variety of the "conflicting" filling-in effects that derive from edges.

Remote controls illusion: strange interactions across space cannot be explained by simple contrast filters

The visual system has separable visual encoding for luminance and for contrast modulation [J. Vis.8(1), B152 (2008)1534-736210.1167/8.6.1]; the two dimensions can be represented with a luminance contrast versus luminance plane. Here we use a contrast asynchrony paradigm to explore contextual effects on luminance contrast modulation: two identical rectangular bars (0.5°×2.5°) have luminance levels that modulate at 2 Hz; when one bar is placed on a bright field and the other bar on a dark field, observers perceive the bars modulating in antiphase with each other and yet becoming light and dark at the same time. The antiphase perception corresponds to the change in contrast between the bars and their surrounds (a change along the contrast axis of the plane); the in-phase perception corresponds to the luminance modulation (a change along the luminance axis of the plane). We examine spatial interaction by adding bright rectangular (0.5°×2.5°) flankers on both sides of the dark-field bar and dark flankers on both sides of the bright-field bar. Remarkably, flankers produce an in-phase appearance when separated from the bars by between 2^' and 4^' of visual angle, and produce antiphase appearance when they directly adjoin the bars or are separated by more than 8^'. To estimate the dimensions of the spatial interaction, we parametrically adjust the size of the gap between bars and flankers and the length of the flankers. We attempt to account for the results with models based on rectified difference of Gaussian filters and with rectified oriented difference of Gaussian filters. The models can account for the results when the flankers are the same height as bars, but are unable to account for the effects of increasing the flanker length. The models therefore suggest that the spatial interaction across distances requires more complex interactions of contrast filters.

Kitaoka's Tomato: Two Simple Explanations Based on Information in the Stimulus

Kitaoka’s Tomato is a color illusion in which a semitransparent blue-green field is placed on top of a red object (a tomato). The tomato appears red even though the pixels would appear green if viewed in isolation. We show that this phenomenon can be explained by a high-pass filter and by histogram equalization. The results suggest that this illusion does not require complex inferences about color constancy; rather, the tomato’s red is available in the physical stimulus at the appropriate spatial scale and dynamic range.

An exponential filter model predicts lightness illusions

Lightness, or perceived reflectance of a surface, is influenced by surrounding context. This is demonstrated by the Simultaneous Contrast Illusion (SCI), where a gray patch is perceived lighter against a black background and vice versa. Conversely, assimilation is where the lightness of the target patch moves toward that of the bounding areas and can be demonstrated in White's effect. Blakeslee and McCourt (1999) introduced an oriented difference-of-Gaussian (ODOG) model that is able to account for both contrast and assimilation in a number of lightness illusions and that has been subsequently improved using localized normalization techniques. We introduce a model inspired by image statistics that is based on a family of exponential filters, with kernels spanning across multiple sizes and shapes. We include an optional second stage of normalization based on contrast gain control. Our model was tested on a well-known set of lightness illusions that have previously been used to evaluate ODOG and its variants, and model lightness values were compared with typical human data. We investigate whether predictive success depends on filters of a particular size or shape and whether pooling information across filters can improve performance. The best single filter correctly predicted the direction of lightness effects for 21 out of 27 illusions. Combining two filters together increased the best performance to 23, with asymptotic performance at 24 for an arbitrarily large combination of filter outputs. While normalization improved prediction magnitudes, it only slightly improved overall scores in direction predictions. The prediction performance of 24 out of 27 illusions equals that of the best performing ODOG variant, with greater parsimony. Our model shows that V1-style orientation-selectivity is not necessary to account for lightness illusions and that a low-level model based on image statistics is able to account for a wide range of both contrast and assimilation effects.

The Oriented Difference of Gaussians (ODOG) model of brightness perception: Overview and executable Mathematica notebooks

The Oriented Difference of Gaussians (ODOG) model of brightness (perceived intensity) by Blakeslee and McCourt (Vision Research 39:4361-4377, 1999), which is based on linear spatial filtering by oriented receptive fields followed by contrast normalization, has proven highly successful in parsimoniously predicting the perceived intensity (brightness) of regions in complex visual stimuli such as White's effect, which had been believed to defy filter-based explanations. Unlike competing explanations such as anchoring theory, filling-in, edge-integration, or layer decomposition, the spatial filtering approach embodied by the ODOG model readily accounts for the often overlooked but ubiquitous gradient structure of induction which, while most striking in grating induction, also occurs within the test fields of classical simultaneous brightness contrast and the White stimulus. Also, because the ODOG model does not require defined regions of interest, it is generalizable to any stimulus, including natural images. The ODOG model has motivated other researchers to develop modified versions (LODOG and FLODOG), and has served as an important counterweight and proof of concept to constrain high-level theories which rely on less well understood or justified mechanisms such as unconscious inference, transparency, perceptual grouping, and layer decomposition. Here we provide a brief but comprehensive description of the ODOG model as it has been implemented since 1999, as well as working Mathematica (Wolfram, Inc.) notebooks which users can employ to generate ODOG model predictions for their own stimuli.

A Neurocomputational account of the role of contour facilitation in brightness perception

A new filling-in model is proposed in order to account for challenging brightness illusions, where inducing background elements are spatially separated from the gray target such as dungeon, cube and grating illusions, bullseye display and ring patterns. This model implements the simple idea that neural response to low-contrast contour is enhanced (facilitated) by the presence of collinear or parallel high-contrast contours in its wider neighborhood. Contour facilitation is achieved via dendritic inhibition, which enables the computation of maximum function among inputs to the node. Recurrent application of maximum function leads to the propagation of the neural signal along collinear or parallel contour segments. When a strong global-contour signal is accompanied with a weak local-contour signal at the same location, conditions are met to produce brightness assimilation within the Filling-in Layer. Computer simulations showed that the model correctly predicts brightness appearance in all of the aforementioned illusions as well as in White's effect, Benary's cross, Todorović's illusion, checkerboard contrast, contrast-contrast illusion and various variations of the White's effect. The proposed model offers new insights on how geometric factors (contour colinearity or parallelism), together with contrast magnitude contribute to the brightness perception.

Brightness induction and suprathreshold vision: effects of age and visual field

A variety of visual capacities show significant age-related alterations. We assessed suprathreshold contrast and brightness perception across the lifespan in a large sample of healthy participants (N=155; 142) ranging in age from 16 to 80 years. Experiment 1 used a quadrature-phase motion cancelation technique (Blakeslee & McCourt, 2008) to measure canceling contrast (in central vision) for induced gratings at two temporal frequencies (1 Hz and 4 Hz) at two test field heights (0.5° or 2°×38.7°; 0.052 c/d). There was a significant age-related reduction in canceling contrast at 4 Hz, but not at 1 Hz. We find no age-related change in induction magnitude in the 1 Hz condition. We interpret the age-related decline in grating induction magnitude at 4 Hz to reflect a diminished capacity for inhibitory processing at higher temporal frequencies. In Experiment 2 participants adjusted the contrast of a matching grating (0.5° or 2°×38.7°; 0.052 c/d) to equal that of both real (30% contrast, 0.052 c/d) and induced (McCourt, 1982) standard gratings (100% inducing grating contrast; 0.052 c/d). Matching gratings appeared in the upper visual field (UVF) and test gratings appeared in the lower visual field (LVF), and vice versa, at eccentricities of ±7.5°. Average induction magnitude was invariant with age for both test field heights. There was a significant age-related reduction in perceived contrast of stimuli in the LVF versus UVF for both real and induced gratings.

A temporal window for estimating surface brightness in the Craik-O'Brien-Cornsweet effect

The central edge of an opposing pair of luminance gradients (COC edge) makes adjoining regions with identical luminance appear to be different. This brightness illusion, called the Craik-O'Brien-Cornsweet effect (COCe), can be explained by low-level spatial filtering mechanisms (Dakin and Bex, 2003). Also, the COCe is greatly reduced when the stimulus lacks a frame element surrounding the COC edge (Purves et al., 1999). This indicates that the COCe can be modulated by extra contextual cues that are related to ideas about lighting priors. In this study, we examined whether processing for contextual modulation could be independent of the main COCe processing mediated by the filtering mechanism. We displayed the COC edge and frame element at physically different times. Then, while varying the onset asynchrony between them and changing the luminance contrast of the frame element, we measured the size of the COCe. We found that the COCe was observed in the temporal range of around 600–800 ms centered at the 0 ms (from around −400 to 400 ms in stimulus onset asynchrony), which was much larger than the range of typical visual persistency. More importantly, this temporal range did not change significantly regardless of differences in the luminance contrast of the frame element (5–100%), in the durations of COC edge and/or the frame element (50 or 200 ms), in the display condition (interocular or binocular), and in the type of lines constituting the frame element (solid or illusory lines). Results suggest that the visual system can bind the COC edge and frame element with a temporal window of ~1 s to estimate surface brightness. Information from the basic filtering mechanism and information of contextual cue are separately processed and are linked afterwards.

Visual crowding illustrates the inadequacy of local vs. global and feedforward vs. feedback distinctions in modeling visual perception

Experimentalists tend to classify models of visual perception as being either local or global, and involving either feedforward or feedback processing. We argue that these distinctions are not as helpful as they might appear, and we illustrate these issues by analyzing models of visual crowding as an example. Recent studies have argued that crowding cannot be explained by purely local processing, but that instead, global factors such as perceptual grouping are crucial. Theories of perceptual grouping, in turn, often invoke feedback connections as a way to account for their global properties. We examined three types of crowding models that are representative of global processing models, and two of which employ feedback processing: a model based on Fourier filtering, a feedback neural network, and a specific feedback neural architecture that explicitly models perceptual grouping. Simulations demonstrate that crucial empirical findings are not accounted for by any of the models. We conclude that empirical investigations that reject a local or feedforward architecture offer almost no constraints for model construction, as there are an uncountable number of global and feedback systems. We propose that the identification of a system as being local or global and feedforward or feedback is less important than the identification of a system's computational details. Only the latter information can provide constraints on model development and promote quantitative explanations of complex phenomena.

Lightness constancy and its link with cone contrast

The link between chromatic constancy (compensation for hue and saturation shifts) and lightness constancy (compensation for a change in surface reflectance) was tested theoretically by computing cone contrasts and by asymmetric matching experiments. The effect of a thin achromatic line (a frame) around the test sample was tested empirically. When the samples were outlined by the frame, lightness constancy was increased and chromatic constancy reduced (p<0.001). Changes in luminance are more likely to be compensated when the luminance contrast edge around the test stimulus is disturbed as with the addition of an achromatic frame.

The roles of image decomposition and edge curvature in the 'snake' lightness illusion

The snake illusion is an effect in which the lightness of target patches is strongly affected by the luminance of remote patches. One explanation is that such images are decomposed into a pattern of illumination and a pattern of reflectance, involving a classification of luminance edges into illumination and reflectance edges. Based on this decomposition, perceived reflectance is determined by discounting the illumination. A problem for this account is that image decomposition is not unique, and that different decompositions may lead to different lightness predictions. One way to rule out alternative decompositions and ensure correct predictions is to postulate that the visual system tends to classify curved luminance edges as reflectance edges rather than illumination edges. We have constructed several variations of the basic snake display in order to test the proposed curvature constraint and the more general image decomposition hypothesis. Although the results from some displays have confirmed previous findings of the effect of curvature, the general pattern of data questions the relevance of the shape of luminance edges for the determination of lightness in this class of displays. The data also argue against an image decomposition mechanism as an explanation of this effect. As an alternative, a tentative neurally based account is sketched.

The effect of background and illumination on color identification of real, 3D objects

For the surface reflectance of an object to be a useful cue to object identity, judgments of its color should remain stable across changes in the object's environment. In 2D scenes, there is general consensus that color judgments are much more stable across illumination changes than background changes. Here we investigate whether these findings generalize to real 3D objects. Observers made color matches to cubes as we independently varied both the illumination impinging on the cube and the 3D background of the cube. As in 2D scenes, we found relatively high but imperfect stability of color judgments under an illuminant shift. In contrast to 2D scenes, we found that background had little effect on average color judgments. In addition, variability of color judgments was increased by an illuminant shift and decreased by embedding the cube within a background. Taken together, these results suggest that in real 3D scenes with ample cues to object segregation, the addition of a background may improve stability of color identification.

Orientation enhancement in early visual processing can explain time course of brightness contrast and White's illusion

Dynamics of orientation tuning in V1 indicates that computational model of V1 should not only comprise of bank of static spatially oriented filters but also include the contribution for dynamical response facilitation or suppression along orientation. Time evolution of orientation response in V1 can emerge due to time- dependent excitation and lateral inhibition in the orientation domain. Lateral inhibition in the orientation domain suggests that Ernst Mach's proposition can be applied for the enhancement of initial orientation distribution that is generated due to interaction of visual stimulus with spatially oriented filters and subcortical temporal filter. Oriented spatial filtering that appears much early (<70 ms) in the sequence of visual information processing can account for many of the brightness illusions observed at steady state. It is therefore expected that time evolution of orientation response might be reflecting in the brightness percept over time. Our numerical study suggests that only spatio-temporal filtering at early phase can explain experimentally observed temporal dynamics of brightness contrast illusion. But, enhancement of orientation response at early phase of visual processing is the key mechanism that can guide visual system to predict the brightness by "Max-rule" or "Winner Takes All" (WTA) estimation and thus producing White's illusions at any exposure.

When is spatial filtering enough? Investigation of brightness and lightness perception in stimuli containing a visible illumination component

Brightness (perceived intensity) and lightness (perceived reflectance) matching were investigated in seven well-known visual stimuli that contain a visible shadow or transparent overlay. These stimuli are frequently upheld as evidence that low-level spatial filtering is inadequate to explain brightness/lightness illusions and that additional mid- or high-level mechanisms are required. The argument in favor of rejecting low-level spatial filtering explanations has been founded on the erroneous assumption that equating test patch and near surround luminance is sufficient to control for and rule out this type of mechanism. We tested this idea by comparing the matching behavior of four observers to the predictions of the ODOG multiscale filtering model (Blakeslee & McCourt, 1999). Lightness and brightness matching differed significantly only when test patches appeared in shadow or beneath a transparency. Lightness and brightness matches were both significantly larger under these conditions; however, the lightness matches greatly exceeded the brightness matches. Lightness matches were greater for test patches in shadow or beneath a transparency because lightness matches under these conditions were based on conscious inferential (not sensory-level) judgments where observers attempted to discount the difference in illumination. The ODOG model accounted for approximately 80% of the total variance in the brightness matches (as well as in the lightness matches for targets not in shadow or beneath a transparency), and successfully predicted the relative magnitude of these matches in five of the seven stimulus sets. These results indicate that multiscale spatial filtering provides a unified and parsimonious explanation for brightness perception in these stimuli and imply that higher-level mechanisms are not required to explain them. The model was not as successful for the argyle and wall of blocks illusions in that it incorrectly rank-ordered the relative magnitude of the effects across different versions of the stimuli. It is an important question whether such model failures are due to known but corrigible limitations of the ODOG model or whether they will require other (possibly higher-level) explanations.

Spatiotemporal analysis of brightness induction

Brightness induction refers to a class of visual illusions in which the perceived intensity of a region of space is influenced by the luminance of surrounding regions. These illusions are significant because they provide insight into the neural organization of the visual system. A novel quadrature-phase motion cancelation technique was developed to measure the magnitude of the grating induction brightness illusion across a wide range of spatial frequencies, temporal frequencies and test field heights. Canceling contrast is greatest at low frequencies and declines with increasing frequency in both dimensions, and with increasing test field height. Canceling contrast scales as the product of inducing grating spatial frequency and test field height (the number of inducing grating cycles per test field height). When plotted using a spatial axis which indexes this product, the spatiotemporal induction surfaces for four test field heights can be described as four partially overlapping sections of a single larger surface. These properties of brightness induction are explained in the context of multiscale spatial filtering. The present study is the first to measure the magnitude of grating induction as a function of temporal frequency. Taken in conjunction with several other studies (Blakeslee & McCourt, 2008; Magnussen & Glad, 1975; Robinson & de Sa, 2008) the results of this study illustrate that at least one form of brightness induction is very much faster than that reported by DeValois, Webster, DeValois, and Lingelbach (1986) and Rossi and Paradiso (1996), and are inconsistent with the proposition that brightness induction results from a slow "filling in" process.

The effect of figural manipulations on brightness differences in the Benary cross

The Benary cross is a classical demonstration showing that the perceived brightness f an area is not solely determined by its luminance, but also by the context in which it is embedded. Despite the fact that two identical grey triangles are flanked by an equal amount of black and white, one of the triangles is perceived as being lighter than the other. It has been argued that the junctions surrounding a test area are crucial in determining brightness. Here, we explored how different aspects influencing perceptual organisation influence perceived figure-background relations in the Benary cross and, with that, the perceived brightness of the triangular patches in our stimuli. The results of a cancellation task confirm that the alignment of contours at junctions indeed has a strong influence on an area's brightness. At the same time, however, the Benary effect is also influenced by the overall symmetry of the cross and its orientation.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.

Spatial filtering versus anchoring accounts of brightness/lightness perception in staircase and simultaneous brightness/lightness contrast stimuli

J. Cataliotti and A. Gilchrist (1995) reported that, consistent with anchoring theory, the lightness of a black step in a reflectance staircase was not altered by moving a white step from a remote to an adjacent location. Recently, E. Economou, S. Zdravkovic, and A. Gilchrist (2007) reported data supporting three additional predictions of the anchoring model (A. Gilchrist et al., 1999): 1) equiluminant incremental targets in staircase simultaneous lightness contrast stimuli appeared equally light; 2) the simultaneous lightness contrast effect was due mainly to the lightening of the target on the black surround; and 3) the strength of lightness induction was greatest for darker targets. We investigated similar stimuli using brightness/lightness matching and found, contrary to these reports, that: 1) the relative position of the steps in a luminance staircase significantly influenced their brightness/lightness; 2) equiluminant incremental targets in staircase simultaneous brightness/lightness contrast stimuli did not all appear equally bright/light; 3) an asymmetry due to a greater brightening/lightening of the target on the black surround was not general; and 4) darker targets produced larger effects only when plotted on a log scale. In addition, the ODOG model (B. Blakeslee & M. E. McCourt, 1999) did an excellent job of accounting for brightness/lightness matching in these stimuli.

Multiplicative and additive Adelson's snake illusions

Two different versions of Adelson's snake lightness illusion are quantitatively investigated. In one experiment an additive version of the illusion is investigated by varying the additive component of the atmosphere transfer function (ATF) introduced by Adelson [2000, in The New Cognitive Neuroscience Ed. M Gazzaniga (Cambridge, MA: MIT Press) pp 339-351]. In the other, a multiplicative version of the illusion is examined by varying the multiplicative component of the ATE In both experiments four observers matched the targets' lightness of the snake patterns with Munsell samples. Increasing the additive or the multiplicative component elicited an approximately equal increase in the magnitude of the lightness illusion. The results show that both components, in the absence of other kinds of information, can be used as heuristics by our visual system to anchor luminance of the object when converting it into lightness.

Low-level features determine brightness in White's and Benary's illusions

We masked White's and Benary's brightness illusions and simultaneous contrast with narrowband visual noise and measured detection thresholds and brightness. The noise was either isotropic or orientation filtered. A narrow spatial frequency tuning was found for detection and brightness for every stimulus. A narrow orientation tuning was also found: the strength of the illusions decreased (White and Benary) or increased (White) depending on the orientation of the mask. The critical borders were always of the same contrast polarity. The results suggest that the brightness in figure-ground scenes is determined by mechanisms integrating incremental and decremental borders in early visual cortices.

Nearly instantaneous brightness induction

Brightness induction is the modulation of the perceived intensity of a region by the luminance of surrounding regions and reveals fundamental properties of neural organization in the visual system. Grating induction affords a unique opportunity to precisely measure the temporal properties of induction using a quadrature motion technique. Contrary to previous reports that induction is a sluggish process with temporal frequency cutoffs of 2-5 Hz (R. L. DeValois, M. A. Webster, K. K. DeValois, & B. Lingelbach, 1986; A. F. Rossi & M. A. Paradiso, 1996), we find that induction is nearly instantaneous. The temporal response of induced brightness differs from that of luminance gratings by a small time lag (<1 ms), or by a small temporal phase lag (<0.016 cycle), and remains relatively constant across wide variations in test field height. These data are not easily explained by an edge-dependent, homogeneous filling-in process (A. F. Rossi & M. A. Paradiso, 1996); however, they are consistent with an explanation of brightness induction based on spatial filtering by cortical simple cells (B. Blakeslee & M. E. McCourt, 1999).

Multiresolution wavelet framework models brightness induction effects

A new multiresolution wavelet model is presented here, which accounts for brightness assimilation and contrast effects in a unified framework, and includes known psychophysical and physiological attributes of the primate visual system (such as spatial frequency channels, oriented receptive fields, contrast sensitivity function, contrast non-linearities, and a unified set of parameters). Like other low-level models, such as the ODOG model [Blakeslee, B., & McCourt, M. E. (1999). A multiscale spatial filtering account of the white effect, simultaneous brightness contrast and grating induction. Vision Research, 39, 4361-4377], this formulation reproduces visual effects such as simultaneous contrast, the White effect, grating induction, the Todorović effect, Mach bands, the Chevreul effect and the Adelson-Logvinenko tile effects, but it also reproduces other previously unexplained effects such as the dungeon illusion, all using a single set of parameters.

Responses to lightness variations in early human visual cortex

Lightness is the apparent reflectance of a surface, and it depends not only on the actual luminance of the surface but also on the context in which the surface is viewed [1-10]. The cortical mechanisms of lightness processing are largely unknown, and the role of early cortical areas is still a matter of debate [11-17]. We studied the cortical responses to lightness variations in early stages of the human visual system with functional magnetic resonance imaging (fMRI) while observers were performing a demanding fixation task. The set of dynamically presented visual stimuli included the rectangular version of the classic Craik-O'Brien stimulus [3, 18, 19] and a variant that led to a weaker lightness effect, as well as a pattern with actual luminance variations. We found that the cortical activity in retinotopic areas, including the primary visual cortex (V1), is correlated with context-dependent lightness variations.

Explaining brightness illusions using spatial filtering and local response normalization

We introduce two new low-level computational models of brightness perception that account for a wide range of brightness illusions, including many variations on White's Effect [Perception, 8, 1979, 413]. Our models extend Blakeslee and McCourt's ODOG model [Vision Research, 39, 1999, 4361], which combines multiscale oriented difference-of-Gaussian filters and response normalization. We extend the response normalization to be more neurally plausible by constraining normalization to nearby receptive fields (models 1 and 2) and spatial frequencies (model 2), and show that both of these changes increase the effectiveness of the models at predicting brightness illusions.

Gradient representation and perception in the early visual system--a novel account of Mach band formation

Recent evidence suggests that object surfaces and their properties are represented at early stages in the visual system of primates. Most likely invariant surface properties are extracted to endow primates with robust object recognition capabilities. In real-world scenes, luminance gradients are often superimposed on surfaces. We argue that gradients should also be represented in the visual system, since they encode highly variable information, such as shading, focal blur, and penumbral blur. We present a neuronal architecture which was designed and optimized for segregating and representing luminance gradients in real-world images. Our architecture in addition provides a novel theory for Mach bands, whereby corresponding psychophysical data are predicted consistently.

Smooth Gradient Representations as a Unifying Account of Chevreul's Illusion, Mach Bands, and a Variant of the Ehrenstein Disk

Recent evidence suggests that the primate visual system generates representations for object surfaces (where we consider representations for the surface attribute brightness). Object recognition can be expected to perform robustly if those representations are invariant despite environmental changes (e.g., in illumination). In real-world scenes, it happens, however, that surfaces are often overlaid by luminance gradients, which we define as smooth variations in intensity. Luminance gradients encode highly variable information, which may represent surface properties (curvature), nonsurface properties (e.g., specular highlights, cast shadows, illumination inhomogeneities), or information about depth relationships (cast shadows, blur). We argue, on grounds of the unpredictable nature of luminance gradients, that the visual system should establish corresponding representations, in addition to surface representations. We accordingly present a neuronal architecture, the so-called gradient system, which clarifies how spatially accurate gradient representations can be obtained by relying on only high-resolution retinal responses. Although the gradient system was designed and optimized for segregating, and generating, representations of luminance gradients with real-world luminance images, it is capable of quantitatively predicting psychophysical data on both Mach bands and Chevreul's illusion. It furthermore accounts qualitatively for a modified Ehrenstein disk.

A multiscale filtering explanation of gradient induction and remote brightness induction effects: a reply to Logvinenko (2003)

Grating induction is a brightness effect in which a counterphase spatial brightness variation (a grating) is induced in a homogeneous test strip that is surrounded by an inducing luminance grating (McCourt, 1982 Vision Research 22 119-134). Moulden and Kingdom (1991 Vision Research 31 1999-2008) introduced an interesting variant of grating induction (sometimes referred to as gradient induction) in which multiple strips of either a linear luminance ramp or a sine-wave grating were interlaced with strips of homogeneous luminance. We (Blakeslee and McCourt, 1999 Vision Research 39 4361-4377) demonstrated that a simple multiscale filtering explanation could account for grating induction. Recently, however, Logvinenko (2003 Perception 32 621-626) presented several arguments impugning the adequacy of spatial filtering approaches to understanding brightness induction in gradient induction stimuli. We propose that Logvinenko's arguments apply only to a limited class of filtering models, specifically those which employ only a single spatial filter. To test this hypothesis we modeled gradient induction stimuli as a function of inducing contrast, as well as Logvinenko's (2003) remote induction stimulus, using our multiscale oriented difference-of-Gaussians (ODOG) model (Blakeslee and McCourt 1999). The ODOG model successfully predicts the appearance of the inducing strips and the homogeneous test strips in the gradient induction stimuli and the appearance of the test patches in the remote induction stimuli. These results refute Logvinenko's (2003) claims, and we interpret them as providing strong evidence for a multiscale filtering approach to understanding both gradient induction and remote brightness induction effects.

Comparing the spatial-frequency response of first-order and second-order lateral visual interactions: grating induction and contrast-contrast

The magnitudes of two suprathreshold lateral spatial-interaction effects--grating induction and contrast--contrast--were compared with regard to their dependence upon inducing-grating spatial frequency. Both effects cause the contrast of target stimuli embedded in surrounding patterns to be matched nonveridically. The magnitudes of each effect were measured in a common unit that indexed the degree of nonveridical contrast matching across a large range of target-grating contrasts (+/- 0.80). Grating induction was a low-pass effect with respect to spatial frequency, whereas contrast-contrast was bandpass, peaking at approximately 4.0 cycles deg(-1). The magnitude of grating induction exceeded that of contrast--contrast, both overall and at their optimal frequencies (0.03125 and 4.0 cycles deg(-1), respectively); the two effects are equipotent at an inducing-grating spatial frequency of 1.0 cycle deg(-1). A significant negative correlation between the magnitudes of the two effects suggests a link whereby activation of second-order normalization mechanisms may inhibit first-order mechanisms.

Oriented multiscale spatial filtering and contrast normalization: a parsimonious model of brightness induction in a continuum of stimuli including White, Howe and simultaneous brightness contrast

The White effect [Perception 8 (1979) 413] cannot be simply explained as due to either brightness contrast or brightness assimilation because the direction of the induced brightness change does not correlate with the amount of black or white border in contact with the gray test patch. This has led some investigators to abandon spatial filtering explanations not only for the White effect but for brightness perception in general. Offered instead are explanations based on a variety of junction analyses and/or perceptual organization schemes which in the case of the White effect are usually based on T-junctions. Recently, Howe [Perception 30 (2001) 1023] challenged T-junction based explanations with a novel variation of White's effect in which the T-junctions were constant while the brightness effect was eliminated or reversed, and proposed an alternative explanation in terms of illusory contours. The present study argues that an analysis at the level of illusory contours is not necessary and that a much simpler spatial filtering based explanation is sufficient. Brightness induction was measured in a set of stimuli chosen to illustrate the relationship between the Howe stimulus [Perception 30 (2001) 1023], the White stimulus [Perception 8 (1979) 413] and the classical simultaneous brightness contrast (SBC) stimulus. The White stimulus and the SBC stimulus occupy opposite ends of a continuum of stimuli in which the Howe stimulus is the mid-point. The psychophysical measurements were compared with the predictions of the oriented difference-of-Gaussians (ODOG) computational model of Blakeslee and McCourt [Vision Research 39 (1999) 4361]. The ODOG model parsimoniously accounted for both the direction and relative magnitude of the brightness effects suggesting that more complex mechanisms are not required to explain them.

A unified theory of brightness contrast and assimilation incorporating oriented multiscale spatial filtering and contrast normalization

Brightness induction includes both contrast and assimilations effects. Brightness contrast occurs when the brightness of a test region shifts away from the brightness of adjacent regions. Brightness assimilation refers to the opposite situation in which the brightness of the test region shifts toward that of the surrounding regions. Interestingly, in the White effect [Perception 8 (1979) 413] the direction of the induced brightness change does not correlate with the amount of black or white border in contact with the gray test patch. This has led some investigators to reject spatial filtering explanations not only for the White effect but for brightness perception in general. Instead, these investigators have offered explanations based on a variety of junction analyses and/or perceptual organization schemes. Here, these approaches are challenged with a critical set of new psychophysical measurements that determined the magnitude of the White effect, the shifted White effect [Perception 10 (1981) 215] and the checkerboard illusion [R.L. DeValois, K.K. DeValois, Spatial Vision, Oxford University Press, NY, 1988] as a function of inducing pattern spatial frequency and test patch height. The oriented difference-of-Gaussians (ODOG) computational model of Blakeslee and McCourt [Vision Res. 39 (1999) 4361] parsimoniously accounts for the psychophysical data, and illustrates that mechanisms based on junction analysis or perceptual inference are not required to explain them. According to the ODOG model, brightness induction results from linear spatial filtering with an incomplete basis set (the finite array of spatial filters in the human visual system). In addition, orientation selectivity of the filters and contrast normalization across orientation channels are critical for explaining some brightness effects, such as the White effect.

Mechanisms of contrast induction in heterogeneous displays

This study examines how judgments of a region's contrast are influenced by components of a heterogeneous surround. Each stimulus comprised a 5x5 grid of squares in a homogeneous background of fixed mean luminance, with the central square the target. On a given trial, the task was to judge (with feedback) whether the (Weber) contrast of the target was 0.04 or -0.04 (relative to the background); the contrasts assigned (in random order) to the 24 surrounding squares were drawn from the values -0.98, -0.33, 0.33, 0.98 in conformity to one of nine pre-chosen histograms. Presentations were brief (80 ms) in one condition and long (800 ms) in another. A novel psychophysical method was used to estimate the impact exerted on judged target contrast (JTC) by a given contrast in a given grid position. Results were similar for four observers. For both display durations, the four squares sharing an edge with the target influenced JTC 2.4-9 times more than any other surrounding squares. In long presentations, abutting squares of extreme contrast repelled target contrast: squares of contrast -0.98 (0.98) increased (decreased) JTC. However, lower contrast abutting squares attracted target contrast: squares of contrast -0.33 (0.33) decreased (increased) JTC. This central finding can be explained by supposing that: (a) JTC is strongly correlated with the average boundary contrast from surround to target, as registered by linear, edge-selective neurons, and, crucially, (b) the responses of these neurons are themselves subject to lateral inhibition from the rectified responses of other similarly tuned neurons. Finally, in brief presentations, a polarity-specific asymmetry was observed: the two positive abutting-square contrasts continued to influence JTC as they did in long presentations, but contrasts -0.33 and -0.98 ceased to exert much impact, suggesting that lateral influences on target appearance propagate more quickly from positive than from negative contrast abutting regions.

Quantitative properties of achromatic color induction: an edge integration analysis

Edge integration refers to a hypothetical process by which the visual system combines information about the local contrast, or luminance ratios, at luminance borders within an image to compute a scale of relative reflectances for the regions between the borders. The results of three achromatic color matching experiments, in which a test and matching ring were surrounded by one or more rings of varying luminance, were analyzed in terms of three alternative quantitative edge integration models: (1) a generalized Retinex algorithm, in which achromatic color is computed from a weighted sum of log luminance ratios, with weights free to vary as a function of distance from the test (Weighted Log Luminance Ratio model); (2) an elaboration of the first model, in which the weights given to distant edges are reduced by a percentage that depends on the log luminance ratios of borders lying between the distant edges and the target (Weighted Log Luminance Ratio model with Blockage); and (3) an alternative modification of the first model, in which Michelson contrasts are substituted for log luminance ratios in the achromatic color computation (Weighted Michelson Contrast model). The experimental results support the Weighted Log Luminance Ratio model over the other two edge integration models. The Weighted Log Luminance Ratio model is also shown to provide a better fit to the achromatic color matching data than does Wallach's Ratio Rule, which states that the two disks will match in achromatic color when their respective disk/ring luminance ratios are equal.

Brightening prospects for early cortical coding of perceived luminance: a high-density electrical mapping study

Establishing the computational rules and neural substrates of brightness coding is a topic of both historical and contemporary interest. Two major classes of explanations for brightness illusions, such as brightness contrast, can be traced to Hering and Helmholtz. Hering's legacy is a low-level account in which brightness contrast results from obligatory lateral inhibitory interactions occurring at some level(s) in the visual system. Helmholtz offered a high-level account, positing a causal role for factors such as perceptual grouping, inferred illumination, and the extraction of surface properties such as orientation and reflectance. The tension between these theoretical viewpoints persists unabated to date. Intracranial electrophysiological recordings have revealed that brightness is represented in the firing rates of striate neurons, a fact consistent with low-level explanations. However, since the time-course of brightness-related responses relative to the onset of striate activity is undisclosed, it remains possible that striate activation might be temporally and causally secondary to higher-level computational processes. Knowledge of the timing of brightness-related neural activity is thus crucial to both constrain and adjudicate between these competing theories. We utilize high-density electrophysiological recording and a tachistoscopic brightness discrimination task to measure the time-course and scalp topography of brightness-related electrical potentials in human observers. Brightness perception is correlated with electrical activity at the earliest stages of visual cortical processing. These findings are interpreted to support Hering's low-level account of brightness for White's effect, and the results are discussed in the context of current theories of brightness perception.

Perceiving the intensity of light

The relationship between luminance (i.e., the photometric intensity of light) and its perception (i.e., sensations of lightness or brightness) has long been a puzzle. In addition to the mystery of why these perceptual qualities do not scale with luminance in any simple way, "illusions" such as simultaneous brightness contrast, Mach bands, Craik-O'Brien-Cornsweet edge effects, and the Chubb-Sperling-Solomon illusion have all generated much interest but no generally accepted explanation. The authors review evidence that the full range of this perceptual phenomenology can be rationalized in terms of an empirical theory of vision. The implication of these observations is that perceptions of lightness and brightness are generated according to the probability distributions of the possible sources of luminance values in stimuli that are inevitably ambiguous.

BOLD fMRI and psychophysical measurements of contrast response to broadband images

We have measured the relationship between image contrast, perceived contrast, and BOLD fMRI activity in human early visual areas, for natural, whitened, pink noise, and white noise images. As root-mean-square contrast increases, BOLD response to natural images is stronger and saturates more rapidly than response to the whitened images. Perceived contrast and BOLD fMRI responses are higher for pink noise than for white noise patterns, by the same ratio as between natural and whitened images. Spatial phase structure has no measurable effect on perceived contrast or BOLD fMRI response. The fMRI and perceived contrast response results can be described by models of spatial frequency response in V1, that match the contrast sensitivity function at low contrasts, and have more uniform spatial frequency response at high contrasts.

Natural image statistics mediate brightness 'filling in'

Although the human visual system can accurately estimate the reflectance (or lightness) of surfaces under enormous variations in illumination, two equiluminant grey regions can be induced to appear quite different simply by placing a light-dark luminance transition between them. This illusion, the Craik-Cornsweet-O'Brien (CCOB) effect, has been taken as evidence for a low-level 'filling-in' mechanism subserving lightness perception. Here, we present evidence that the mechanism responsible for the CCOB effect operates not via propagation of a neural signal across space but by amplification of the low spatial frequency (SF) structure of the image. We develop a simple computational model that relies on the statistics of natural scenes actively to reconstruct the image that is most likely to have caused an observed series of responses across SF channels. This principle is tested psychophysically by deriving classification images (CIs) for subjects' discrimination of the contrast polarity of CCOB stimuli masked with noise. CIs resemble 'filled-in' stimuli; i.e. observers rely on portions of the stimuli that contain no information per se but that correspond closely to the reported perceptual completion. As predicted by the model, the filling-in process is contingent on the presence of appropriate low SF structure.

Influences of motion and depth on brightness induction: an illusory transparency effect?

To experiments were performed to investigate whether motion and binocular disparity influence brightness induction, and whether the effects of motion and binocular disparity, if any, interact with each other. In order to introduce motion, textured backgrounds were used as the inducing field. The results showed that motion and/or crossed disparity reduce brightness induction, whereas uncrossed disparity increases it. The effect of motion and the effect of disparity are independent of each other and additive, which suggests that, to the extent that brightness induction reflects segmentation of objects, motion and binocular disparity serve independently to segment objects from their background. The difference between the effects of crossed and uncrossed disparity can be explained by what we call 'illusory transparency'.

Balancing visual weights

An experiment is described which is based on a peculiar response index, the visual balance, which involves the match of visual weight. The stimulus consists of a pair of grey fields, differing in their spectral reflectance factor, and the experiment is designed to determine how the balance condition depends on their relative sizes. This dependence is found to be linear when the test object is considered globally; non-linear when considering its fine intra-pair geometrical characteristics.