Straightness as a cue for luminance edge interpretation

Lightness Perception in Complex Scenes

Lightness perception is the perception of achromatic surface colors: black, white, and shades of grey. Lightness has long been a central research topic in experimental psychology, as perceiving surface color is an important visual task but also a difficult one due to the deep ambiguity of retinal images. In this article, I review psychophysical work on lightness perception in complex scenes over the past 20 years, with an emphasis on work that supports the development of computational models. I discuss Bayesian models, equivalent illumination models, multidimensional scaling, anchoring theory, spatial filtering models, natural scene statistics, and related work in computer vision. I review open topics in lightness perception that seem ready for progress, including the relationship between lightness and brightness, and developing more sophisticated computational models of lightness in complex scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

A model of lightness perception guided by probabilistic assumptions about lighting and reflectance

Lightness perception is the ability to perceive black, white, and gray surface colors in a wide range of lighting conditions and contexts. This ability is fundamental for any biological or artificial visual system, but it poses a difficult computational problem, and how the human visual system computes lightness is not well understood. Here I show that several key phenomena in lightness perception can be explained by a probabilistic graphical model that makes a few simple assumptions about local patterns of lighting and reflectance, and infers globally optimal interpretations of stimulus images. Like human observers, the model exhibits partial lightness constancy, codetermination, contrast, glow, and articulation effects. It also arrives at human-like interpretations of strong lightness illusions that have challenged previous models. The model's assumptions are reasonable and generic, including, for example, that lighting intensity spans a much wider range than surface reflectance and that shadow boundaries tend to be straighter than reflectance edges. Thus, a probabilistic model based on simple assumptions about lighting and reflectance gives a good computational account of lightness perception over a wide range of conditions. This work also shows how graphical models can be extended to develop more powerful models of constancy that incorporate features such color and depth.

The Influence of Physical Illumination on Lightness Perception in Simultaneous Contrast Displays

Three experiments investigated the role of physical illumination on lightness perception in simultaneous lightness contrast (SLC). Four configurations were employed: the classic textbook version of the illusion and three configurations that produced either enhanced or reduced SLC. Experiment 1 tested the effect of ambient illumination on lightness perception. It simulated very dark environmental conditions that nevertheless still allowed perception of different shades of gray. Experiment 2 tested the effect of the intensity of Gelb lighting on lightness perception. Experiment 3 presented two conditions that integrated illumination conditions from Experiments 1 and 2. Our results demonstrated an illumination effect on both lightness matching and perceived SLC contrast: As the intensity of illumination increased, the target on the black background appeared lighter, while the target on the white background was little affected. We hypothesize the existence of two illumination ranges that affect lightness perception differently: low and normal. In the low range, the SLC contrast was reduced and targets appeared darker. In the normal range, the SLC contrast and lightness matchings for each background were little changed across illumination intensities.

The roles of image decomposition and edge curvature in the 'snake' lightness illusion

The snake illusion is an effect in which the lightness of target patches is strongly affected by the luminance of remote patches. One explanation is that such images are decomposed into a pattern of illumination and a pattern of reflectance, involving a classification of luminance edges into illumination and reflectance edges. Based on this decomposition, perceived reflectance is determined by discounting the illumination. A problem for this account is that image decomposition is not unique, and that different decompositions may lead to different lightness predictions. One way to rule out alternative decompositions and ensure correct predictions is to postulate that the visual system tends to classify curved luminance edges as reflectance edges rather than illumination edges. We have constructed several variations of the basic snake display in order to test the proposed curvature constraint and the more general image decomposition hypothesis. Although the results from some displays have confirmed previous findings of the effect of curvature, the general pattern of data questions the relevance of the shape of luminance edges for the determination of lightness in this class of displays. The data also argue against an image decomposition mechanism as an explanation of this effect. As an alternative, a tentative neurally based account is sketched.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.