A model of lightness perception guided by probabilistic assumptions about lighting and reflectance

What Fechner could not do: Separating perceptual encoding and decoding with difference scaling

A key question in perception research is how stimulus variations translate into perceptual magnitudes, that is, the perceptual encoding process. As experimenters, we cannot probe perceptual magnitudes directly, but infer the encoding process from responses obtained in a psychophysical experiment. The most prominent experimental technique to measure perceptual appearance is matching, where observers adjust a probe stimulus to match a target in its appearance along the dimension of interest. The resulting data quantify the perceived magnitude of the target in physical units of the probe, and are thus an indirect expression of the underlying encoding process. In this paper, we show analytically and in simulation that data from matching tasks do not sufficiently constrain perceptual encoding functions, because there exist an infinite number of pairs of encoding functions that generate the same matching data. We use simulation to demonstrate that maximum likelihood conjoint measurement (Ho, Landy, & Maloney, 2008; Knoblauch & Maloney, 2012) does an excellent job of recovering the shape of ground truth encoding functions from data that were generated with these very functions. Finally, we measure perceptual scales and matching data for White's effect (White, 1979) and show that the matching data can be predicted from the estimated encoding functions, down to individual differences.

Vertical anisotropy in lightness perception not caused by lighting assumption

Our recent study found an illusory effect whereby an image of an upward-facing gray panel appears darker than its 180-degree rotated image. We attributed this inversion effect to the observer's implicit assumption that light from above is more intense than light from below. This paper aims to explore the possibility that low-level visual anisotropy also contributes to the effect. In Experiment 1, we investigated whether the effect could be observed even when the position, the contrast polarity, and the existence of the edge were manipulated. In Experiments 2 and 3, the effect was further examined using stimuli that contained no depth cues. Experiment 4 confirmed the effect using stimuli of even simpler configuration. The results of all the experiments demonstrated that brighter edges on the upper side of the target make it appear lighter, indicating that low-level anisotropy contributes to the inversion effect, even without depth orientation information. However, darker edges on the upper side of the target produced ambiguous results. We speculate that the perceived lightness of the target might be affected by two kinds of vertical anisotropy, one of which is dependent on contrast polarity while the other is independent of it. Moreover, the results also replicated the previous finding that the lighting assumption contributes to perceived lightness. Overall, the present study demonstrates that both low-level vertical anisotropy and mid-level lighting assumption influence lightness.

Texture statistics involved in specular highlight exclusion for object lightness perception

The human visual system estimates the physical properties of objects, such as their lightness. Previous studies on the lightness perception of glossy three-dimensional objects have suggested that specular highlights are detected and excluded in lightness perception. However, only a few studies have attempted to elucidate the mechanisms underlying this exclusion. This study aimed to elucidate the image features that contribute to the highlight exclusion of lightness perception. We used Portilla-Simoncelli texture statistics (PS statistics), an image feature set similar to the representation in the early visual cortex, to explore their relationships with highlight exclusion for lightness perception. In experiment 1, computer graphics images of bumpy plastic plates with various physical parameters were used as stimuli, and the lightness perception on them was measured using a lightness matching task. We then calculated the highlight exclusion index, which represented the degree of highlight exclusion. Finally, we evaluated the correlation between the highlight exclusion index and the four PS statistic subsets. In experiment 2, an image synthesis algorithm was used to create images in which either the PS statistic subset was manipulated. The highlight exclusion indexes of the synthesized images were then measured. The results revealed that the PS statistic subset consisting of lowest-order image features, such as moment statistics of luminance, acts as a necessary condition for highlight exclusion, whereas the other three subsets consisting of higher order features are not crucial. These results suggest that the low-order image features are the most important among the features in PS statistics for highlight exclusion, even though image features higher order than those in PS statistics must be directly involved.

The effects of distractors on brightness perception based on a spiking network

Visual perception can be modified by the surrounding context. Particularly, experimental observations have demonstrated that visual perception and primary visual cortical responses could be modified by properties of surrounding distractors. However, the underlying mechanism remains unclear. To simulate primary visual cortical activities in this paper, we design a k-winner-take-all (k-WTA) spiking network whose responses are generated through probabilistic inference. In simulations, images with the same target and various surrounding distractors perform as stimuli. Distractors are designed with multiple varying properties, including the luminance, the sizes and the distances to the target. Simulations for each varying property are performed with other properties fixed. Each property could modify second-layer neural responses and interactions in the network. To the same target in the designed images, the modified network responses could simulate distinguishing brightness perception consistent with experimental observations. Our model provides a possible explanation of how the surrounding distractors modify primary visual cortical responses to induce various brightness perception of the given target.

Empirical evaluation of computational models of lightness perception

Lightness of a surface depends not only on its physical characteristics, but also on the properties of the surrounding context. As a result, varying the context can significantly alter surface lightness, an effect exploited in many lightness illusions. Computational models can produce outcomes similar to human illusory percepts, allowing for demonstrable assessment of the applied mechanisms and principles. We tested 8 computational models on 13 typical displays used in lightness research (11 Illusions and 2 Mondrians), and compared them with results from human participants (N = 85). Results show that HighPass and MIR models predict empirical results for simultaneous lightness contrast (SLC) and its close variations. ODOG and its newer variants (ODOG-2 and L-ODOG) in addition to SLC displays were able to predict effect of White's illusion. RETINEX was able to predict effects of both SLC displays and Dungeon illusion. Dynamic decorrelation model was able to predict obtained effects for all tested stimuli except two SLC variations. Finally, FL-ODOG model was best at simulating human data, as it was able to predict empirical results for all displays, bar the Reversed contrast illusion. Finally, most models underperform on the Mondrian displays that represent most natural stimuli for the human visual system.

Simple Assumptions to Improve Markov Illuminance and Reflectance

Murray recently introduced a novel computational lightness model, Markov illuminance and reflectance (MIR). MIR is a promising new approach that simulates human lightness processing using a conditional random field (CRF) where natural-scene statistics of reflectance and illumination are implemented. Although MIR can account for various lightness illusions and phenomena, it has limitations, such as the inability to predict reverse-contrast phenomena. In this study, we improved MIR performance by modifying its inference process, the prior on X-junctions, and that on general illumination changes. Our modified model improved predictions for Checkerboard assimilation, the simplified Checkershadow and its control figure, the influence of luminance noise, and White's effect and its several variants. In particular, White's effect is a partial reverse contrast that is challenging for computational models, so this improvement is a significant advance for the MIR framework. This study showed the high extensibility and potential of MIR, which shows the promise for further sophistication.

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers' internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest - within a factor of two of the equivalent noise level set by internal noise.

Lightness Perception in Complex Scenes

Lightness perception is the perception of achromatic surface colors: black, white, and shades of grey. Lightness has long been a central research topic in experimental psychology, as perceiving surface color is an important visual task but also a difficult one due to the deep ambiguity of retinal images. In this article, I review psychophysical work on lightness perception in complex scenes over the past 20 years, with an emphasis on work that supports the development of computational models. I discuss Bayesian models, equivalent illumination models, multidimensional scaling, anchoring theory, spatial filtering models, natural scene statistics, and related work in computer vision. I review open topics in lightness perception that seem ready for progress, including the relationship between lightness and brightness, and developing more sophisticated computational models of lightness in complex scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.