Coming to terms with lightness and brightness: effects of stimulus configuration and instructions on brightness and lightness judgments

Lightness constancy in reality, in virtual reality, and on flat-panel displays

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

A Novel Smartphone-Based Color Test for Detection of Color Vision Defects in Age Related Macular Degeneration

Purpose

To evaluate the efficacy of the smartphone-based K-color test to detect color defects in patients with Age-related Macular Degeneration (AMD).

Methods

88 patients (n = 135 eyes) with AMD and 28 controls (n = 53 eyes) underwent color testing with the Hardy–Rand–Rittler (H-R-R), the K-color test, and the Ishihara test. The K-color test presents randomized colored shapes in decreasing steps of intensity, providing also a record system for result tele-transmission. Sensitivity, specificity, and reliability were examined to investigate the validity of the novel test. 26 participants with AMD also completed a questionnaire regarding the feasibility of the test.

Results

Linear mixed-effects models indicated a significant difference (p < 0.001) between AMD and normal eyes. The areas under the curve (AUC) were estimated to be 0.897 [95% CI: 0.841–0.952], 0.943 [95% CI: 0.901–0.984], and 0.931 [95% CI: 0.886–0.977] for the red, green, and blue color, respectively. Based on the H-R-R, the sensitivity of the test was 0.79, 0.90, and 0.95 for the red, green, and blue colors, respectively, and specificity was 0.88 for all colors. The new test recognized more abnormal cases than the Ishihara (sensitivity of 0.98 and 1.0 and specificity of 0.48 and 0.38 for red and green colors, respectively). Test-retest reliability was found to be high for the red [ICC = 0.996 (0.990–0.999)], green [ICC = 0.974 (0.929–0.990)], and blue [ICC = 0.992 (0.981–0.997)] colors. The majority of the asked participants stated that they could easily perform the test.

Conclusion

The K-color test was found to be sensitive and specific in detecting color defects in AMD patients. The K-color test may serve as a useful tool both for patients and their physicians.

Lightness Perception in Complex Scenes

Lightness perception is the perception of achromatic surface colors: black, white, and shades of grey. Lightness has long been a central research topic in experimental psychology, as perceiving surface color is an important visual task but also a difficult one due to the deep ambiguity of retinal images. In this article, I review psychophysical work on lightness perception in complex scenes over the past 20 years, with an emphasis on work that supports the development of computational models. I discuss Bayesian models, equivalent illumination models, multidimensional scaling, anchoring theory, spatial filtering models, natural scene statistics, and related work in computer vision. I review open topics in lightness perception that seem ready for progress, including the relationship between lightness and brightness, and developing more sophisticated computational models of lightness in complex scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Lateral effects in pattern vision

There is a large literature on lateral effects in pattern vision but no consensus about them or comprehensive model of them. This paper reviews the literature with a focus on the effects of parallel context in the central fovea. It describes seven experiments that measure detection and discrimination thresholds in annular and Gabor-pattern contexts at different separations. It presents a model of these effects, which is an elaboration of Foley's (1994) model. The model describes the results well, and it shows that lateral context affects the response to the target by both multiplicative excitation and additive inhibition. Both lateral effects extend for several wavelengths beyond the target. They vary in relative strength, producing near suppression and far enhancement of the response to the target. The model describes the detection and discrimination results well, and it also describes the results of experiments on lateral effects on perceived contrast. The model is consistent with the physiology of V1 cells.

The perception of colour and material in naturalistic tasks

Perceived object colour and material help us to select and interact with objects. Because there is no simple mapping between the pattern of an object's image on the retina and its physical reflectance, our perceptions of colour and material are the result of sophisticated visual computations. A long-standing goal in vision science is to describe how these computations work, particularly as they act to stabilize perceived colour and material against variation in scene factors extrinsic to object surface properties, such as the illumination. If we take seriously the notion that perceived colour and material are useful because they help guide behaviour in natural tasks, then we need experiments that measure and models that describe how they are used in such tasks. To this end, we have developed selection-based methods and accompanying perceptual models for studying perceived object colour and material. This focused review highlights key aspects of our work. It includes a discussion of future directions and challenges, as well as an outline of a computational observer model that incorporates early, known, stages of visual processing and that clarifies how early vision shapes selection performance.

The Influence of Physical Illumination on Lightness Perception in Simultaneous Contrast Displays

Three experiments investigated the role of physical illumination on lightness perception in simultaneous lightness contrast (SLC). Four configurations were employed: the classic textbook version of the illusion and three configurations that produced either enhanced or reduced SLC. Experiment 1 tested the effect of ambient illumination on lightness perception. It simulated very dark environmental conditions that nevertheless still allowed perception of different shades of gray. Experiment 2 tested the effect of the intensity of Gelb lighting on lightness perception. Experiment 3 presented two conditions that integrated illumination conditions from Experiments 1 and 2. Our results demonstrated an illumination effect on both lightness matching and perceived SLC contrast: As the intensity of illumination increased, the target on the black background appeared lighter, while the target on the white background was little affected. We hypothesize the existence of two illumination ranges that affect lightness perception differently: low and normal. In the low range, the SLC contrast was reduced and targets appeared darker. In the normal range, the SLC contrast and lightness matchings for each background were little changed across illumination intensities.

The nature of instructional effects in color constancy

The instructions subjects receive can have a large effect on experimentally measured color constancy, but the nature of these effects and how their existence should inform our understanding of color perception remains unclear. We used a factorial design to measure how instructional effects on constancy vary with experimental task and stimulus set. In each of 2 experiments, we employed both a classic adjustment-based asymmetric matching task and a novel color selection task. Four groups of naive subjects were instructed to make adjustments/selections based on (a) color (neutral instructions); (b) the light reaching the eye (physical spectrum instructions); (c) the actual surface reflectance of an object (objective reflectance instructions); or (d) the apparent surface reflectance of an object (apparent reflectance instructions). Across the 2 experiments we varied the naturalness of the stimuli. We find clear interactions between instructions, task, and stimuli. With simplified stimuli (Experiment 1), instructional effects were large and the data revealed 2 instruction-dependent patterns. In 1 (neutral and physical spectrum instructions) constancy was low, intersubject variability was also low, and adjustment-based and selection-based constancy were in agreement. In the other (reflectance instructions) constancy was high, intersubject variability was large, adjustment-based constancy deviated from selection-based constancy and for some subjects selection-based constancy increased across sessions. Similar patterns held for naturalistic stimuli (Experiment 2), although instructional effects were smaller. We interpret these 2 patterns as signatures of distinct task strategies-1 is perceptual, with judgments based primarily on the perceptual representation of color; the other involves explicit instruction-driven reasoning. (PsycINFO Database Record

What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception

Research in lightness perception centers on understanding the prior assumptions and processing strategies the visual system uses to parse the retinal intensity distribution (the proximal stimulus) into the surface reflectance and illumination components of the scene (the distal stimulus—ground truth). It is agreed that the visual system must compare different regions of the visual image to solve this inverse problem; however, the nature of the comparisons and the mechanisms underlying them are topics of intense debate. Perceptual illusions are of value because they reveal important information about these visual processing mechanisms. We propose a framework for lightness research that resolves confusions and paradoxes in the literature, and provides insight into the mechanisms the visual system employs to tackle the inverse problem. The main idea is that much of the debate and confusion in the literature stems from the fact that lightness, defined as apparent reflectance, is underspecified and refers to three different types of judgments that are not comparable. Under stimulus conditions containing a visible illumination component, such as a shadow boundary, observers can distinguish and match three independent dimensions of achromatic experience: apparent intensity (brightness), apparent local intensity ratio (brightness-contrast), and apparent reflectance (lightness). In the absence of a visible illumination boundary, however, achromatic vision reduces to two dimensions and, depending on stimulus conditions and observer instructions, judgments of lightness are identical to judgments of brightness or brightness-contrast. Furthermore, because lightness judgments are based on different information under different conditions, they can differ greatly in their degree of difficulty and in their accuracy. This may, in part, explain the large variability in lightness constancy across studies.

The Oriented Difference of Gaussians (ODOG) model of brightness perception: Overview and executable Mathematica notebooks

The Oriented Difference of Gaussians (ODOG) model of brightness (perceived intensity) by Blakeslee and McCourt (Vision Research 39:4361-4377, 1999), which is based on linear spatial filtering by oriented receptive fields followed by contrast normalization, has proven highly successful in parsimoniously predicting the perceived intensity (brightness) of regions in complex visual stimuli such as White's effect, which had been believed to defy filter-based explanations. Unlike competing explanations such as anchoring theory, filling-in, edge-integration, or layer decomposition, the spatial filtering approach embodied by the ODOG model readily accounts for the often overlooked but ubiquitous gradient structure of induction which, while most striking in grating induction, also occurs within the test fields of classical simultaneous brightness contrast and the White stimulus. Also, because the ODOG model does not require defined regions of interest, it is generalizable to any stimulus, including natural images. The ODOG model has motivated other researchers to develop modified versions (LODOG and FLODOG), and has served as an important counterweight and proof of concept to constrain high-level theories which rely on less well understood or justified mechanisms such as unconscious inference, transparency, perceptual grouping, and layer decomposition. Here we provide a brief but comprehensive description of the ODOG model as it has been implemented since 1999, as well as working Mathematica (Wolfram, Inc.) notebooks which users can employ to generate ODOG model predictions for their own stimuli.

A unified account of perceptual layering and surface appearance in terms of gamut relativity

When we look at the world--or a graphical depiction of the world--we perceive surface materials (e.g. a ceramic black and white checkerboard) independently of variations in illumination (e.g. shading or shadow) and atmospheric media (e.g. clouds or smoke). Such percepts are partly based on the way physical surfaces and media reflect and transmit light and partly on the way the human visual system processes the complex patterns of light reaching the eye. One way to understand how these percepts arise is to assume that the visual system parses patterns of light into layered perceptual representations of surfaces, illumination and atmospheric media, one seen through another. Despite a great deal of previous experimental and modelling work on layered representation, however, a unified computational model of key perceptual demonstrations is still lacking. Here we present the first general computational model of perceptual layering and surface appearance--based on a boarder theoretical framework called gamut relativity--that is consistent with these demonstrations. The model (a) qualitatively explains striking effects of perceptual transparency, figure-ground separation and lightness, (b) quantitatively accounts for the role of stimulus- and task-driven constraints on perceptual matching performance, and (c) unifies two prominent theoretical frameworks for understanding surface appearance. The model thereby provides novel insights into the remarkable capacity of the human visual system to represent and identify surface materials, illumination and atmospheric media, which can be exploited in computer graphics applications.

A temporal window for estimating surface brightness in the Craik-O'Brien-Cornsweet effect

The central edge of an opposing pair of luminance gradients (COC edge) makes adjoining regions with identical luminance appear to be different. This brightness illusion, called the Craik-O'Brien-Cornsweet effect (COCe), can be explained by low-level spatial filtering mechanisms (Dakin and Bex, 2003). Also, the COCe is greatly reduced when the stimulus lacks a frame element surrounding the COC edge (Purves et al., 1999). This indicates that the COCe can be modulated by extra contextual cues that are related to ideas about lighting priors. In this study, we examined whether processing for contextual modulation could be independent of the main COCe processing mediated by the filtering mechanism. We displayed the COC edge and frame element at physically different times. Then, while varying the onset asynchrony between them and changing the luminance contrast of the frame element, we measured the size of the COCe. We found that the COCe was observed in the temporal range of around 600–800 ms centered at the 0 ms (from around −400 to 400 ms in stimulus onset asynchrony), which was much larger than the range of typical visual persistency. More importantly, this temporal range did not change significantly regardless of differences in the luminance contrast of the frame element (5–100%), in the durations of COC edge and/or the frame element (50 or 200 ms), in the display condition (interocular or binocular), and in the type of lines constituting the frame element (solid or illusory lines). Results suggest that the visual system can bind the COC edge and frame element with a temporal window of ~1 s to estimate surface brightness. Information from the basic filtering mechanism and information of contextual cue are separately processed and are linked afterwards.

No Measured Effect of a Familiar Contextual Object on Color Constancy

Some familiar objects have a typical color, such as the yellow of a banana. The presence of such objects in a scene is a potential cue to the scene illumination, since the light reflected from them should on average be consistent with their typical surface reflectance. Although there are many studies on how the identity of an object affects how its color is perceived, little is known about whether the presence of a familiar object in a scene helps the visual system stabilize the color appearance of other objects with respect to changes in illumination. We used a successive color matching procedure in three experiments designed to address this question. Across the experiments we studied a total of 6 subjects (2 in Experiment 1, 3 in Experiment 2, and 4 in Experiment 3) with partial overlap of subjects between experiments. We compared measured color constancy across conditions in which a familiar object cue to the illuminant was available with conditions in which such a cue was not present. Overall, our results do not reveal a reliable improvement in color constancy with the addition of a familiar object to a scene. An analysis of the experimental power of our data suggests that if there is such an effect, it is small: less than approximately a change of 0.09 in a constancy index where an absence of constancy corresponds to an index value of 0 and perfect constancy corresponds to an index value of 1.

A theory divided: current representations of the anchoring theory of lightness contradict the original's core claims

The anchoring theory of lightness perception (Gilchrist et al., Psychological Review 106 (1999) 795-834) has been described as one of the most successful approaches to lightness perception. Yet, not only does the original proposal contain serious gaps and inconsistencies, later expressions of the theory, which was never formally revised, seem to contradict the original claims while leaving the gaps unresolved. These problems call into question the theory's viability.

Lighting direction and visual field modulate perceived intensity of illumination

When interpreting object shape from shading the visual system exhibits a strong bias that illumination comes from above and slightly from the left. We asked whether such biases in the perceived direction of illumination might also influence its perceived intensity. Arrays of nine cubes were stereoscopically rendered where individual cubes varied in their 3D pose, but possessed identical triplets of visible faces. Arrays were virtually illuminated from one of four directions: Above-Left, Above-Right, Below-Left, and Below-Right (±24.4° azimuth; ±90° elevation). Illumination intensity possessed 15 levels, resulting in mean cube array luminances ranging from 1.31-3.45 cd/m(2). A "reference" array was consistently illuminated from Above-Left at mid-intensity (mean array luminance = 2.38 cd/m(2)). The reference array's illumination was compared to that of matching arrays which were illuminated from all four directions at all intensities. Reference and matching arrays appeared in the left and right visual field, respectively, or vice versa. Subjects judged which cube array appeared to be under more intense illumination. Using the method of constant stimuli we determined the illumination level of matching arrays required to establish subjective equality with the reference array as a function of matching cube visual field, illumination elevation, and illumination azimuth. Cube arrays appeared significantly more intensely illuminated when they were situated in the left visual field (p = 0.017), and when they were illuminated from below (p = 0.001), and from the left (p = 0.001). An interaction of modest strength was that the effect of illumination azimuth was greater for matching arrays situated in the left visual field (p = 0.042). We propose that objects lit from below appear more intensely illuminated than identical objects lit from above due to long-term adaptation to downward lighting. The amplification of perceived intensity of illumination for stimuli situated in the left visual field and lit from the left is best explained by tonic egocentric and allocentric leftward attentional biases, respectively.

We infer light in space

In studies of lightness and color constancy, the terms lightness and brightness refer to the qualia corresponding to perceived surface reflectance and perceived luminance, respectively. However, what has rarely been considered is the fact that the volume of space containing surfaces appears neither empty, void, nor black, but filled with light. Helmholtz (1866/1962) came closest to describing this phenomenon when discussing inferred illumination, but previous theoretical treatments have fallen short by restricting their considerations to the surfaces of objects. The present work is among the first to explore how we infer the light present in empty space. It concludes with several research examples supporting the theory that humans can infer the differential levels and chromaticities of illumination in three-dimensional space.

When is spatial filtering enough? Investigation of brightness and lightness perception in stimuli containing a visible illumination component

Brightness (perceived intensity) and lightness (perceived reflectance) matching were investigated in seven well-known visual stimuli that contain a visible shadow or transparent overlay. These stimuli are frequently upheld as evidence that low-level spatial filtering is inadequate to explain brightness/lightness illusions and that additional mid- or high-level mechanisms are required. The argument in favor of rejecting low-level spatial filtering explanations has been founded on the erroneous assumption that equating test patch and near surround luminance is sufficient to control for and rule out this type of mechanism. We tested this idea by comparing the matching behavior of four observers to the predictions of the ODOG multiscale filtering model (Blakeslee & McCourt, 1999). Lightness and brightness matching differed significantly only when test patches appeared in shadow or beneath a transparency. Lightness and brightness matches were both significantly larger under these conditions; however, the lightness matches greatly exceeded the brightness matches. Lightness matches were greater for test patches in shadow or beneath a transparency because lightness matches under these conditions were based on conscious inferential (not sensory-level) judgments where observers attempted to discount the difference in illumination. The ODOG model accounted for approximately 80% of the total variance in the brightness matches (as well as in the lightness matches for targets not in shadow or beneath a transparency), and successfully predicted the relative magnitude of these matches in five of the seven stimulus sets. These results indicate that multiscale spatial filtering provides a unified and parsimonious explanation for brightness perception in these stimuli and imply that higher-level mechanisms are not required to explain them. The model was not as successful for the argyle and wall of blocks illusions in that it incorrectly rank-ordered the relative magnitude of the effects across different versions of the stimuli. It is an important question whether such model failures are due to known but corrigible limitations of the ODOG model or whether they will require other (possibly higher-level) explanations.

Melanopsin contributions to irradiance coding in the thalamo-cortical visual system

Photoreception in the mammalian retina is not restricted to rods and cones but extends to a subset of retinal ganglion cells expressing the photopigment melanopsin (mRGCs). These mRGCs are known to drive such reflex light responses as circadian photoentrainment and pupillomotor movements. By contrast, until now there has been no direct assessment of their contribution to conventional visual pathways. Here, we address this deficit. Using new reporter lines, we show that mRGC projections are much more extensive than previously thought and extend across the dorsal lateral geniculate nucleus (dLGN), origin of thalamo-cortical projection neurons. We continue to show that this input supports extensive physiological light responses in the dLGN and visual cortex in mice lacking rods+cones (a model of advanced retinal degeneration). Moreover, using chromatic stimuli to isolate melanopsin-derived responses in mice with an intact visual system, we reveal strong melanopsin input to the ∼40% of neurons in the LGN that show sustained activation to a light step. We demonstrate that this melanopsin input supports irradiance-dependent increases in the firing rate of these neurons. The implication that melanopsin is required to accurately encode stimulus irradiance is confirmed using melanopsin knockout mice. Our data establish melanopsin-based photoreception as a significant source of sensory input to the thalamo-cortical visual system, providing unique irradiance information and allowing visual responses to be retained even in the absence of rods+cones. These findings identify mRGCs as a potential origin for aspects of visual perception and indicate that they may support vision in people suffering retinal degeneration.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.

Contrast, constancy, and measurements of perceived lightness under parametric manipulation of surface slant and surface reflectance

Across many scenes, local contrast provides a valid cue to surface reflectance, but it is not the only such cue. To generalize beyond theories of lightness that rely exclusively on local contrast, we need to know which other potential cues matter. We had observers make lightness matches between two scene locations, and varied the surface slant and local surround reflectance of one of the locations. When local contrast was a valid cue to reflectance, all observers were approximately lightness constant. When it was not, observers' lightness matches were intermediate between contrast matching and lightness constancy. For most observers, surface slant exerted an effect on perceived lightness beyond that explainable by local contrast.

Spatial filtering versus anchoring accounts of brightness/lightness perception in staircase and simultaneous brightness/lightness contrast stimuli

J. Cataliotti and A. Gilchrist (1995) reported that, consistent with anchoring theory, the lightness of a black step in a reflectance staircase was not altered by moving a white step from a remote to an adjacent location. Recently, E. Economou, S. Zdravkovic, and A. Gilchrist (2007) reported data supporting three additional predictions of the anchoring model (A. Gilchrist et al., 1999): 1) equiluminant incremental targets in staircase simultaneous lightness contrast stimuli appeared equally light; 2) the simultaneous lightness contrast effect was due mainly to the lightening of the target on the black surround; and 3) the strength of lightness induction was greatest for darker targets. We investigated similar stimuli using brightness/lightness matching and found, contrary to these reports, that: 1) the relative position of the steps in a luminance staircase significantly influenced their brightness/lightness; 2) equiluminant incremental targets in staircase simultaneous brightness/lightness contrast stimuli did not all appear equally bright/light; 3) an asymmetry due to a greater brightening/lightening of the target on the black surround was not general; and 4) darker targets produced larger effects only when plotted on a log scale. In addition, the ODOG model (B. Blakeslee & M. E. McCourt, 1999) did an excellent job of accounting for brightness/lightness matching in these stimuli.