Challenges to color constancy in a contemporary light

Colour expectations across illumination changes

This study investigates human expectations towards naturalistic colour changes under varying illuminations. Understanding colour expectations is key to both scientific research on colour constancy and applications of colour and lighting in art and industry. We reanalysed data from asymmetric colour matches of a previous study and found that colour adjustments tended to align with illuminant-induced colour shifts predicted by naturalistic, rather than artificial, illuminants and reflectances. We conducted three experiments using hyperspectral images of naturalistic scenes to test if participants judged colour changes based on naturalistic illuminant and reflectance spectra as more plausible than artificial ones, which contradicted their expectations. When we consistently manipulated the illuminant (Experiment 1) and reflectance (Experiment 2) spectra across the whole scene, observers chose the naturalistic renderings significantly above the chance level (>25 %) but barely more often than any of the three artificial ones, collectively (>50 %). However, when we manipulated only one object/area's reflectance (Experiment 3), observers more reliably identified the version in which the object had a naturalistic reflectance like the rest of the scene. Results from Experiments 2-3 and additional analyses suggested that relational colour constancy strongly contributed to observer expectations, and stable cone-excitation ratios are not limited to naturalistic illuminants and reflectances but also occur for our artificial renderings. Our findings indicate that relational colour constancy and prior knowledge about surface colour shifts help to disambiguate surface colour identity under illumination changes, enabling human observers to recognise surface colours reliably in naturalistic conditions. Additionally, relational colour constancy may even be effective in many artificial conditions.

Are ipRGCs involved in human color vision? Hints from physiology, psychophysics, and natural image statistics

Human photoreceptors consist of cones, rods, and melanopsin-expressing intrinsically photosensitive retinal ganglion cells (ipRGCs). First studied in circadian regulation and pupillary control, ipRGCs project to a variety of brain centers suggesting a broader involvement beyond non-visual functions. IpRGC responses are stable, long-lasting, and with a particular codification of photoreceptor signals. In comparison with the transient and adaptive nature of cone and rod signals, ipRGCs' signaling might provide an ecological advantage to different attributes of color vision. Previous studies have indicated melanopsin's influence on visual responses yet its contribution to color perception in humans remains debated. We summarized evidence and hypotheses (from physiology, psychophysics, and natural image statistics) about direct and indirect involvement of ipRGCs in human color vision, by first briefly assessing the current knowledge about the role of melanopsin and ipRGCs in vision and codification of spectral signals. We then approached the question about melanopsin activation eliciting a color percept, discussing studies using the silent substitution method. Finally, we explore various avenues through which ipRGCs might impact color perception indirectly, such as through involvement in peripheral color matching, post-receptoral pathways, color constancy, long-term chromatic adaptation, and chromatic induction. While there is consensus about the role of ipRGCs in brightness perception, confirming its direct contribution to human color perception requires further investigation. We proposed potential approaches for future research, emphasizing the need for empirical validation and methodological thoroughness to elucidate the exact role of ipRGCs in human color vision.

Misidentifying illuminant changes in natural scenes due to failures in relational colour constancy

The colours of surfaces in a scene may not appear constant with a change in the colour of the illumination. Yet even when colour constancy fails, human observers can usually discriminate changes in lighting from changes in surface reflecting properties. This operational ability has been attributed to the constancy of perceived colour relations between surfaces under illuminant changes, in turn based on approximately invariant spatial ratios of cone photoreceptor excitations. Natural deviations in these ratios may, however, lead to illuminant changes being misidentified. The aim of this work was to test whether such misidentifications occur with natural scenes and whether they are due to failures in relational colour constancy. Pairs of scene images from hyperspectral data were presented side-by-side on a computer-controlled display. On one side, the scene underwent illuminant changes and on the other side, it underwent the same changes but with images corrected for any residual deviations in spatial ratios. Observers systematically misidentified the corrected images as being due to illuminant changes. The frequency of errors increased with the size of the deviations, which were closely correlated with the estimated failures in relational colour constancy.

Object-based color constancy in a deep neural network

Color constancy refers to our capacity to see consistent colors under different illuminations. In computer vision and image processing, color constancy is often approached by explicit estimation of the scene's illumination, followed by an image correction. In contrast, color constancy in human vision is typically measured as the capacity to extract color information about objects and materials in a scene consistently throughout various illuminations, which goes beyond illumination estimation and might require some degree of scene and color understanding. Here, we pursue an approach with deep neural networks that tries to assign reflectances to individual objects in the scene. To circumvent the lack of massive ground truth datasets labeled with reflectances, we used computer graphics to render images. This study presents a model that recognizes colors in an image pixel by pixel under different illumination conditions.

Color constancy for daylight illumination changes in anomalous trichromats and dichromats

Color constancy is the perceptual stability of surface colors under temporal changes in the illumination spectrum. The illumination discrimination task (IDT) reveals worse discrimination for "bluer" illumination changes in normal-trichromatic observers (changes towards cooler color temperatures on the daylight chromaticity locus), indicating greater stability of scene colors or better color constancy, compared with illumination changes in other chromatic directions. Here, we compare the performance of individuals with X-linked color-vision deficiencies (CVDs) to normal trichromats on the IDT performed in an immersive setting with a real scene illuminated by spectrally tunable LED lamps. We determine discrimination thresholds for illumination changes relative to a reference illumination (D65) in four chromatic directions, roughly parallel and orthogonal to the daylight locus. We find, using both a standard CIELUV metric and a cone-contrast metric tailored to distinct CVD types, that discrimination thresholds for daylight changes do not differ between normal trichromats and CVD types, including dichromats and anomalous trichromats, but thresholds for atypical illuminations do differ. This result extends a previous report of illumination discrimination ability in dichromats for simulated daylight changes in images. In addition, using the cone-contrast metric to compare thresholds for bluer and yellower daylight changes with those for unnatural redder and greener changes, we suggest that reduced sensitivity to daylight changes is weakly preserved in X-linked CVDs.

Sensory representation of surface reflectances: assessments with hyperspectral images

Specifying surface reflectances in a simple and perceptually informative way would be beneficial for many areas of research and application. We assessed whether a 3×3 matrix may be used to approximate how a surface reflectance modulates the sensory color signal across illuminants. We tested whether observers could discriminate between the model's approximate and accurate spectral renderings of hyperspectral images under narrowband and naturalistic, broadband illuminants for eight hue directions. Discriminating the approximate from the spectral rendering was possible with narrowband, but almost never with broadband illuminants. These results suggest that our model specifies the sensory information of reflectances across naturalistic illuminants with high fidelity, and with lower computational cost than spectral rendering.

Invariant categorical color regions across illuminant change coincide with focal colors

Are there regions in a color space where color categories are invariant across illuminant changes? If so, what characteristics make them more stable than other regions? To address these questions, we asked observers to give a color name to 424 colored surfaces, presented one at a time, under various chromatic illuminants. Results showed a high degree of categorical color constancy, especially under illuminants that occur in the natural environment. It was also shown that surfaces selected as a focal color (the best example of a color category) are more resistant to illuminant change than nonfocal color samples. We additionally ran an asymmetric color matching experiment to quantify the shift of color appearance induced by illuminant changes using surfaces that were all named gray, thereby disentangling the appearance-based color constancy from the categorical color constancy (which are often confounded). Results suggested that the appearance of color samples largely shifted owing to illuminant changes, even though all samples were named gray; showing that the constancy of a color category is substantially more robust than the constancy of color appearance.

Minimal theory of 3D vision: new approach to visual scale and visual shape

Since Kepler and Descartes in the early-1600s, vision science has been committed to a triangulation model of stereo vision. But in the early-1800s, we realized that disparities are responsible for stereo vision. And we have spent the past 200 years trying to shoe-horn disparities back into the triangulation account. The first part of this article argues that this is a mistake, and that stereo vision is a solution to a different problem: the eradication of rivalry between the two retinal images, rather than the triangulation of objects in space. This leads to a 'minimal theory of 3D vision', where 3D vision is no longer tied to estimating the scale, shape, and direction of objects in the world. The second part of this article then asks whether the other aspects of 3D vision, which go beyond stereo vision, really operate at the same level of visual experience as stereo vision? I argue they do not. Whilst we want a theory of real-world 3D vision, the literature risks giving us a theory of picture perception instead. And I argue for a two-stage theory, where our purely internal 'minimal' 3D percept (from stereo vision) is linked to the world through cognition. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Do You See What I See? Diversity in Human Color Perception

In our tendency to discuss the objective properties of the external world, we may fail to notice that our subjective perceptions of those properties differ between individuals. Variability at all levels of the color vision system creates diversity in color perception, from discrimination to color matching, appearance, and subjective experience, such that each of us lives in a unique perceptual world. In this review, I discuss what is known about individual differences in color perception and its determinants, particularly considering genetically mediated variability in cone photopigments and the paradoxical effects of visual environments in both contributing to and counteracting individual differences. I make the case that, as well as being of interest in their own right and crucial for a complete account of color vision, individual differences can be used as a methodological tool in color science for the insights that they offer about the underlying mechanisms of perception. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers' internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest - within a factor of two of the equivalent noise level set by internal noise.

Colour constancy failures expected in colourful environments

Colour constancy refers to the constant perceived or apparent colour of a surface despite changes in illumination spectrum. Laboratory measurements have often found it imperfect. The aim here was to estimate the frequency of constancy failures in natural outdoor environments and relate them to colorimetric surface properties. A computational analysis was performed with 50 hyperspectral reflectance images of outdoor scenes undergoing simulated daylight changes. For a chromatically adapted observer, estimated colour appearance changed noticeably for at least 5% of the surface area in 60% of scenes, and at least 10% of the surface area in 44% of scenes. Somewhat higher frequencies were found for estimated changes in perceived colour relations represented by spatial ratios of cone-photoreceptor excitations. These estimated changes correlated with surface chroma and saturation. Outdoors, the colour constancy of some individual surfaces seems likely to fail, particularly if those surfaces are colourful.

Color appearance model incorporating contrast adaptation - implications for individual differences in color vision

Color appearance models use standard color matching functions to derive colorimetric information from spectral radiometric measurements of a visual environment, and they process that information to predict color perceptual attributes such as hue, chroma and lightness. That processing is usually done by equations with fixed numerical coefficients that were predetermined to yield optimal agreement for a given standard observer. Here we address the well-known fact that, among color-normal observers, there are significant differences of color matching functions. These cause disagreements between individuals as to whether certain colors match, an important effect that is often called observer metamerism. Yet how these individual sensitivity differences translate into differences in perceptual metrics is not fully addressed by many appearance models. It might seem that appearance could be predicted by substituting an individual's color matching functions into an otherwise-unchanged color appearance model, but this is problematic because the model's coefficients were not optimized for the new observer. Here we explore a solution guided by the idea that processes of adaptation in the visual system tend to compensate color perception for differences in cone responses and consequent color matching functions. For this purpose, we developed a simple color appearance model that uses only a few numerical coefficients, yet accurately predicts the perceptual attributes of Munsell samples under a selected standard lighting condition. We then added a feedback loop to automatically adjust the model coefficients, in response to switching between cone fundamentals simulating different observers and color matching functions. This adjustment is intended to model long term contrast adaptation in the vision system by maintaining average overall color contrast levels. Incorporating this adaptation principle into color appearance models could allow better assessments of displays and illumination systems, to help improve color appearances for most observers.

Influence of Main Ocular Variables in #TheDress Perception: An Ophthalmic Survey

The objective of this study is to study the influence of ocular variables in the perception of #thedress and to develop a logistic regression model that could help predict it. This is a cross-sectional study on 1,100 subjects. People who did not report one of the two main perceptions were excluded from the study. Dress perception was codified as 0 (white&gold) or 1 (black&blue). The association between dress perception and demographic and main ocular variables (age, gender, binocular visual acuity, grade of nuclear cataract, crystalline lens status [phakic/pseudophakic], spherical equivalent, and ocular health status) was tested using logistic regression. Receiver operation curves were used to test the predictive value of the model. Several variables were found to be related with dress perception. The best model included three variables-Age: adjusted odds ratio (OR) = 1.02 (1.01-1.03), p = 0.08; ocular refraction: adjusted OR = 1.07 (1.02-1.12), p = 0.009; and nuclear cataract grade: adjusted OR = 1.45 (1.05-1.99), p = 0.026. The predictive value of the model was low (area under the curve = 0.62). Older age, nuclear cataract grade, and hyperopia were associated with black&blue perception. The predictive capacity of the developed model was poor. Only a small proportion of the variability in the #thedress perception can be explained by ocular examination.

Fluctuating environmental light limits number of surfaces visually recognizable by colour

Small changes in daylight in the environment can produce large changes in reflected light, even over short intervals of time. Do these changes limit the visual recognition of surfaces by their colour? To address this question, information-theoretic methods were used to estimate computationally the maximum number of surfaces in a sample that can be identified as the same after an interval. Scene data were taken from successive hyperspectral radiance images. With no illumination change, the average number of surfaces distinguishable by colour was of the order of 10,000. But with an illumination change, the average number still identifiable declined rapidly with change duration. In one condition, the number after two minutes was around 600, after 10 min around 200, and after an hour around 70. These limits on identification are much lower than with spectral changes in daylight. No recoding of the colour signal is likely to recover surface identity lost in this uncertain environment.

Temporal dynamics of daylight perception: Detection thresholds

Temporal changes in illumination are ubiquitous; natural light, for example, varies in color temperature and irradiance throughout the day. Yet little is known about human sensitivity to temporal changes in illumination spectra. Here, we aimed to determine the minimum detectable velocity of chromaticity change of daylight metamers in an immersive environment. The main stimulus was a continuous, monotonic change in global illumination chromaticity along the daylight locus in warmer (toward lower correlated color temperatures [CCTs]) or cooler directions, away from an adapting base light (CCT: 13,000 K, 6500 K, 4160 K, or 2000 K). All lights were generated by spectrally tunable overhead lamps as smoothest-possible metamers of the desired chromaticities. Mean detection thresholds (for 22 participants) for a fixed duration of 10 seconds ranged from 15 to 2 CIELUV ΔE units, depending significantly on base light CCT and with a significant interaction between CCT and direction of change. Cool changes become less noticeable for progressively warmer base lights and vice versa. For the two extreme base lights, sensitivity to changes toward neutral is significantly lower than for the opposite direction. The results suggest a “neutral bias” in illumination change discriminability, and that typical temporal changes in daylight chromaticity are likely to be below threshold detectability, at least where there are no concomitant overall illuminance changes. These factors may contribute to perceptual stability of natural scenes and color constancy.

Exploring the Determinants of Color Perception Using #Thedress and Its Variants: The Role of Spatio-Chromatic Context, Chromatic Illumination, and Material-Light Interaction

The colors that people see depend not only on the surface properties of objects but also on how these properties interact with light as well as on how light reflected from objects interacts with an individual's visual system. Because individual visual systems vary, the same visual stimulus may elicit different perceptions from different individuals. #thedress phenomenon drove home this point: different individuals viewed the same image and reported it to be widely different colors: blue and black versus white and gold. This phenomenon inspired a collection of demonstrations presented at the Vision Sciences Society 2015 Meeting which showed how spatial and temporal manipulations of light spectra affect people's perceptions of material colors and illustrated the variability in individual color perception. The demonstrations also explored the effects of temporal alterations in metameric lights, including Maxwell's Spot, an entoptic phenomenon. Crucially, the demonstrations established that #thedress phenomenon occurs not only for images of the dress but also for the real dress under real light sources of different spectral composition and spatial configurations.

The Verriest Lecture: Adventures in blue and yellow

Conventional models of color vision assume that blue and yellow (along with red and green) are the fundamental building blocks of color appearance, yet how these hues are represented in the brain and whether and why they might be special are questions that remain shrouded in mystery. Many studies have explored the visual encoding of color categories, from the statistics of the environment to neural processing to perceptual experience. Blue and yellow are tied to salient features of the natural color world, and these features have likely shaped several important aspects of color vision. However, it remains less certain that these dimensions are encoded as primary or "unique" in the visual representation of color. There are also striking differences between blue and yellow percepts that may reflect high-level inferences about the world, specifically about the colors of light and surfaces. Moreover, while the stimuli labeled as blue or yellow or other basic categories show a remarkable degree of constancy within the observer, they all vary independently of one another across observers. This pattern of variation again suggests that blue and yellow and red and green are not a primary or unitary dimension of color appearance, and instead suggests a representation in which different hues reflect qualitatively different categories rather than quantitative differences within an underlying low-dimensional "color space."