Context-dependent judgments of color that might allow color constancy in scenes with multiple regions of illumination

Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks

We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.

Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks

We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.

Human color constancy based on the geometry of color distributions

The physical inputs to our visual system are dictated by the interplay between lights and surfaces; thus, for surface color to be stably perceived, the influence of the illuminant must be discounted. To reveal our strategy to infer the illuminant color, we conducted three psychophysical experiments designed to test our optimal color hypothesis that we internalize the physical color gamut under various illuminants and apply the prior to estimate the illuminant color. In each experiment, we presented 61 hexagons arranged without spatial gaps, where the surrounding 60 hexagons were set to have a specific shape in their color distribution. We asked participants to adjust the color of a center test field so that it appeared to be a full-white surface placed under a test illuminant. Results and computational modeling suggested that, although our proposed model is limited in accounting for estimation of illuminant intensity by human observers, it agrees fairly well with the estimates of illuminant chromaticity in most tested conditions. The accuracy of estimation generally outperformed other tested conventional color constancy models. These results support the hypothesis that our visual system can utilize the geometry of scene color distribution to achieve color constancy.

Hyperspectral environmental illumination maps: characterizing directional spectral variation in natural environments

Objects placed in real-world scenes receive incident light from every direction, and the spectral content of this light may vary from one direction to another. In computer graphics, environmental illumination is approximated using maps that specify illumination at a point as a function of incident angle. However, to-date, existing public databases of environmental illumination specify only three colour channels (RGB). We have captured a new set of 12 environmental illumination maps (eight outdoor scenes; four indoor scenes) using a hyperspectral imaging system with 33 spectral channels. The data reveal a striking directional variation of spectral distribution of lighting in natural environments. We discuss limitations of using daylight models to describe natural environmental illumination.

Discrimination of spectral reflectance under environmental illumination

Color constancy is the ability to recover a stable perceptual estimate of surface reflectance, regardless of the lighting environment. However, we know little about how observers make judgments of the surface color of glossy objects, particularly in complex lighting environments that introduce complex spatial patterns of chromatic variation across an object's surface. To address this question, we measured thresholds for reflectance discrimination using computer-rendered stimuli under environmental illumination. In Experiment 1, we found that glossiness and shape had small effects on discrimination thresholds. Importantly, discrimination ellipses extended along the direction in which the chromaticities in the environmental illumination spread. In Experiment 2, we also found that the observers' abilities to judge surface colors were worse in lighting environments with an atypical chromatic distribution.

Low levels of specularity support operational color constancy, particularly when surface and illumination geometry can be inferred

We tested whether surface specularity alone supports operational color constancy-the ability to discriminate changes in illumination or reflectance. Observers viewed short animations of illuminant or reflectance changes in rendered scenes containing a single spherical surface and were asked to classify the change. Performance improved with increasing specularity, as predicted from regularities in chromatic statistics. Peak performance was impaired by spatial rearrangements of image pixels that disrupted the perception of illuminated surfaces but was maintained with increased surface complexity. The characteristic chromatic transformations that are available with nonzero specularity are useful for operational color constancy, particularly if accompanied by appropriate perceptual organization.

Luminance-dependent long-term chromatic adaptation

There is theoretical and empirical support for long-term adaptation of human vision to chromatic regularities in the environment. The current study investigates whether relationships of luminance and chromaticity in the natural environment could drive chromatic adaptation independently and differently for bright and dark colors. This is motivated by psychophysical evidence of systematic difference shifts in red-green chromatic sensitivities between contextually bright- versus dark-colored stimuli. For some broad classes of scene content, consistent shifts in chromaticity are found between high and low light levels within images. Especially in those images in which sky and terrain are juxtaposed, this shift has direction and magnitude consistent with the observed psychophysical shifts in the red-green balance between bright and dark colors. Taken together, these findings suggest that relative weighting of M- and L-cone signals could be adapted, in a luminance-dependent fashion, to regularities in the natural environment.

A Bayesian model of lightness perception that incorporates spatial variation in the illumination

The lightness of a test stimulus depends in a complex manner on the context in which it is viewed. To predict lightness, it is necessary to leverage measurements of a feasible number of contextual configurations into predictions for a wider range of configurations. Here we pursue this goal, using the idea that lightness results from the visual system's attempt to provide stable information about object surface reflectance. We develop a Bayesian algorithm that estimates both illumination and reflectance from image luminance, and link perceived lightness to the algorithm's estimates of surface reflectance. The algorithm resolves ambiguity in the image through the application of priors that specify what illumination and surface reflectances are likely to occur in viewed scenes. The prior distributions were chosen to allow spatial variation in both illumination and surface reflectance. To evaluate our model, we compared its predictions to a data set of judgments of perceived lightness of test patches embedded in achromatic checkerboards (Allred, Radonjić, Gilchrist, & Brainard, 2012). The checkerboard stimuli incorporated the large variation in luminance that is a pervasive feature of natural scenes. In addition, the luminance profile of the checks both near to and remote from the central test patches was systematically manipulated. The manipulations provided a simplified version of spatial variation in illumination. The model can account for effects of overall changes in image luminance and the dependence of such changes on spatial location as well as some but not all of the more detailed features of the data.

Slow updating of the achromatic point after a change in illumination

For a color constant observer, the color appearance of a surface is independent of the spectral composition of the light illuminating it. We ask how rapidly color appearance judgments are updated following a change in illumination. We obtained repeated binary color classifications for a set of stimuli defined by their reflectance functions and rendered under either sunlight or skylight. We used these classifications to derive boundaries in color space that identify the observer's achromatic point. In steady-state conditions of illumination, the achromatic point lay close to the illuminant chromaticity. In our experiment, the illuminant changed abruptly every 21 s (at the onset of every 10th trial), allowing us to track changes in the achromatic point that were caused by the cycle of illuminant changes. In one condition, the test reflectance was embedded in a spatial pattern of reflectance samples under consistent illumination. The achromatic point migrated across color space between the chromaticities of the steady-state achromatic points. This update took several trials rather than being immediate. To identify the factors that governed perceptual updating of appearance judgments, we used two further conditions, one in which the test reflectance was presented in isolation and one in which the surrounding reflectances were rendered under an inconsistent and unchanging illumination. Achromatic settings were not well predicted by the information available from scenes at a single time point. Instead, the achromatic points showed a strong dependence on the history of chromatic samples. The strength of this dependence differed between observers and was modulated by the spatial context.