Bayesian color constancy

Spectral Reconstruction from RGB Imagery: A Potential Option for Infinite Spectral Data?

Spectral imaging has revolutionisedvarious fields by capturing detailed spatial and spectral information. However, its high cost and complexity limit the acquisition of a large amount of data to generalise processes and methods, thus limiting widespread adoption. To overcome this issue, a body of the literature investigates how to reconstruct spectral information from RGB images, with recent methods reaching a fairly low error of reconstruction, as demonstrated in the recent literature. This article explores the modification of information in the case of RGB-to-spectral reconstruction beyond reconstruction metrics, with a focus on assessing the accuracy of the reconstruction process and its ability to replicate full spectral information. In addition to this, we conduct a colorimetric relighting analysis based on the reconstructed spectra. We investigate the information representation by principal component analysis and demonstrate that, while the reconstruction error of the state-of-the-art reconstruction method is low, the nature of the reconstructed information is different. While it appears that the use in colour imaging comes with very good performance to handle illumination, the distribution of information difference between the measured and estimated spectra suggests that caution should be exercised before generalising the use of this approach.

Reliability and stability of tactile perception in the whisker somatosensory system

Rodents rely on their whiskers as vital sensory tools for tactile perception, enabling them to distinguish textures and shapes. Ensuring the reliability and constancy of tactile perception under varying stimulus conditions remains a fascinating and fundamental inquiry. This study explores the impact of stimulus configurations, including whisker movement velocity and object spatial proximity, on texture discrimination and stability in rats. To address this issue, we employed three distinct approaches for our investigation. Stimulus configurations notably affected tactile inputs, altering whisker vibration's kinetic and kinematic aspects with consistent effects across various textures. Through a texture discrimination task, rats exhibited consistent discrimination performance irrespective of changes in stimulus configuration. However, alterations in stimulus configuration significantly affected the rats' ability to maintain stability in texture perception. Additionally, we investigated the influence of stimulus configurations on cortical neuronal responses by manipulating them experimentally. Notably, cortical neurons demonstrated substantial and intricate changes in firing rates without compromising the ability to discriminate between textures. Nevertheless, these changes resulted in a reduction in texture neuronal response stability. Stimulating multiple whiskers led to improved neuronal texture discrimination and maintained coding stability. These findings emphasize the importance of considering numerous factors and their interactions when studying the impact of stimulus configuration on neuronal responses and behavior.

Nighttime color constancy using robust gray pixels

Color constancy is a basic step for achieving stable color perception in both biological visual systems and the image signal processing (ISP) pipeline of cameras. So far, there have been numerous computational models of color constancy that focus on scenes under normal light conditions but are less concerned with nighttime scenes. Compared with daytime scenes, nighttime scenes usually suffer from relatively higher-level noise and insufficient lighting, which usually degrade the performance of color constancy methods designed for scenes under normal light. In addition, there is a lack of nighttime color constancy datasets, limiting the development of relevant methods. In this paper, based on the gray-pixel-based color constancy methods, we propose a robust gray pixel (RGP) detection method by carefully designing the computation of illuminant-invariant measures (IIMs) from a given color-biased nighttime image. In addition, to evaluate the proposed method, a new dataset that contains 513 nighttime images and corresponding ground-truth illuminants was collected. We believe this dataset is a useful supplement to the field of color constancy. Finally, experimental results show that the proposed method achieves superior performance to statistics-based methods. In addition, the proposed method was also compared with recent deep-learning methods for nighttime color constancy, and the results show the method's advantages in cross-validation among different datasets.

Statistical approach for radioactivity detection: A brief review

Radioactivity detection is a major research and development priority for many practical applications. Amongst the various technical challenges in this field is the need to carry out accurate low-level radioactivity measurements in the presence of a large fluctuations in the natural radiation background, while reducing the false alarm rates. The task becomes even more harder with high detection limits under low signal-to-background ratios. A detection method based on the statistical inference, following either a frequentist or a Bayesian paradigm, adopted to overcome these challenges as well as to ensure a reliable and accurate diagnosis with a competitive tradeoff between sensitivity, specificity and response time. With this respect, several research studies, addressing a range of applications from decommissioning and dismantling to homeland security, have been proposed. Our main goal in this paper is to present a succinct survey of these studies based on a frequentist and Bayesian approaches used to decision-making, uncertainty and risk evaluation, in the context of radioactive detection. In this prospect, a theoretical background of statistical frequentist and Bayesian inferences was presented. Then, a comparative study of both approaches was performed to determine the optimal approach in regards to accuracy and pros/cons. A case of study for low-level radioactivity detection in nuclear decommissioning operations was provided to validate the optimal approach. Results proved the efficiency and usefulness of Bayesian approach against frequentist one with respect to the most challenging scenarios in radiation detection applications.

Ranking-Based Color Constancy With Limited Training Samples

Computational color constancy is an important component of Image Signal Processors (ISP) for white balancing in many imaging devices. Recently, deep convolutional neural networks (CNN) have been introduced for color constancy. They achieve prominent performance improvements comparing with those statistics or shallow learning-based methods. However, the need for a large number of training samples, a high computational cost and a huge model size make CNN-based methods unsuitable for deployment on low-resource ISPs for real-time applications. In order to overcome these limitations and to achieve comparable performance to CNN-based methods, an efficient method is defined for selecting the optimal simple statistics-based method (SM) for each image. To this end, we propose a novel ranking-based color constancy method (RCC) that formulates the selection of the optimal SM method as a label ranking problem. RCC designs a specific ranking loss function, and uses a low rank constraint to control the model complexity and a grouped sparse constraint for feature selection. Finally, we apply the RCC model to predict the order of the candidate SM methods for a test image, and then estimate its illumination using the predicted optimal SM method (or fusing the results estimated by the top k SM methods). Comprehensive experiment results show that the proposed RCC outperforms nearly all the shallow learning-based methods and achieves comparable performance to (sometimes even better performance than) deep CNN-based methods with only 1/2000 of the model size and training time. RCC also shows good robustness to limited training samples and good generalization crossing cameras. Furthermore, to remove the dependence on the ground truth illumination, we extend RCC to obtain a novel ranking-based method without ground truth illumination (RCC_NO) that learns the ranking model using simple partial binary preference annotations provided by untrained annotators rather than experts. RCC_NO also achieves better performance than the SM methods and most shallow learning-based methods with low costs of sample collection and illumination measurement.

A model of colour appearance based on efficient coding of natural images

An object's colour, brightness and pattern are all influenced by its surroundings, and a number of visual phenomena and "illusions" have been discovered that highlight these often dramatic effects. Explanations for these phenomena range from low-level neural mechanisms to high-level processes that incorporate contextual information or prior knowledge. Importantly, few of these phenomena can currently be accounted for in quantitative models of colour appearance. Here we ask to what extent colour appearance is predicted by a model based on the principle of coding efficiency. The model assumes that the image is encoded by noisy spatio-chromatic filters at one octave separations, which are either circularly symmetrical or oriented. Each spatial band's lower threshold is set by the contrast sensitivity function, and the dynamic range of the band is a fixed multiple of this threshold, above which the response saturates. Filter outputs are then reweighted to give equal power in each channel for natural images. We demonstrate that the model fits human behavioural performance in psychophysics experiments, and also primate retinal ganglion responses. Next, we systematically test the model's ability to qualitatively predict over 50 brightness and colour phenomena, with almost complete success. This implies that much of colour appearance is potentially attributable to simple mechanisms evolved for efficient coding of natural images, and is a well-founded basis for modelling the vision of humans and other animals.

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.

Perceiving the representative surface color of real-world materials

Natural surfaces such as soil, grass, and skin usually involve far more complex and heterogenous structures than the perfectly uniform surfaces assumed in studies on color and material perception. Despite this, we can easily perceive the representative color of these surfaces. Here, we investigated the visual mechanisms underlying the perception of representative surface color using 120 natural images of diverse materials and their statistically synthesized images. Our matching experiments indicated that the perceived representative color revealed was not significantly different from the Portilla-Simoncelli-synthesized images or phase-randomized images except for one sample, even though the perceived shape and material properties were greatly impaired in the synthetic stimuli. The results also showed that the matched representative colors were predictable from the saturation-enhanced color of the brightest point in the image, excluding the high-intensity outliers. The results support the notion that humans judge the representative color and lightness of real-world surfaces depending on simple image measurements.

Developmental changes in colour constancy in a naturalistic object selection task

When the illumination falling on a surface change, so does the reflected light. Despite this, adult observers are good at perceiving surfaces as relatively unchanging-an ability termed colour constancy. Very few studies have investigated colour constancy in infants, and even fewer in children. Here we asked whether there is a difference in colour constancy between children and adults; what the developmental trajectory is between six and 11 years; and whether the pattern of constancy across illuminations and reflectances differs between adults and children. To this end, we developed a novel, child-friendly computer-based object selection task. In this, observers saw a dragon's favourite sweet under a neutral illumination and picked the matching sweet from an array of eight seen under a different illumination (blue, yellow, red, or green). This set contained a reflectance match (colour constant; perfect performance) and a tristimulus match (colour inconstant). We ran two experiments, with two-dimensional scenes in one and three-dimensional renderings in the other. Twenty-six adults and 33 children took part in the first experiment; 26 adults and 40 children took part in the second. Children performed better than adults on this task, and their performance decreased with age in both experiments. We found differences across illuminations and sweets, but a similar pattern across both age groups. This unexpected finding might reflect a real decrease in colour constancy from childhood to adulthood, explained by developmental changes in the perceptual and cognitive mechanisms underpinning colour constancy, or differences in task strategies between children and adults. HIGHLIGHTS: Six- to 11-year-old children demonstrated better performance than adults on a colour constancy object selection task. Performance decreased with age over childhood. These findings may indicate development of cognitive strategies used to overcome automatic colour constancy mechanisms.

Intrinsic Image Decomposition Using Paradigms

Intrinsic image decomposition is the task of mapping image to albedo and shading. Classical approaches derive methods from spatial models. The modern literature stresses evaluation, by comparing predictions to human judgements ("lighter", "same as", "darker"). The best modern intrinsic image methods train a map from image to albedo using images rendered from computer graphics models and example human judgements. This approach yields practical methods, but obtaining rendered images can be inconvenient. Furthermore, the approach cannot explain how a one could learn to recover intrinsic images without geometric, surface and illumination models, as people and animals appear to do. This paper describes a method that learns intrinsic image decomposition without seeing human annotations, rendered data, or ground truth data. Instead, the method relies on paradigms - spatial models of albedo and of shading. Rather than finding the "best" albedo and shading for an image via optimization, our approach trains a neural network on synthetic images. The synthetic images are constructed by multiplying albedos and shading fields sampled from our models. The network is subject to a novel smoothing procedure that ensures good behavior at short scales on real images. An averaging procedure ensures that reported albedo and shading are largely equivariant - different crops and scalings of an image will report the same albedo and shading at shared points. This averaging procedure controls long scale error. The standard evaluation for an intrinsic image method is a WHDR score. Our method achieves WHDR scores competitive with those of strong recent methods allowed to see training WHDR annotations, rendered data, and ground truth data. Our method produces albedo and shading maps with attractive qualitative properties - for example, albedo fields do not suppress wood grain and represent narrow grooves in surfaces well. Because our method is unsupervised, we can compute estimates of the test/train variance of WHDR scores; these are quite large, and suggest is unsafe to rely small differences in reported WHDR.

Distinct response properties between the FFA to faces and the PPA to houses

INTRODUCTION

The object recognition system involves both selectivity to specific object category and invariance to changes in low-level visual features. Mounting neuroimaging evidence supports that brain areas in the ventral temporal cortex, such as the FFA and PPA, respond preferentially to faces and houses, respectively. However, how regions in human ventral temporal cortex partitioned and functionally organized to selectively and invariantly respond to different object categories remains unclear. What are the changes of response properties at the intersection of adjacent but distinctively-selective regions?

METHOD

Here, we conducted an fMRI study and three-pronged analyses to compare the brain mapping relationships between the FFA to faces and the PPA to houses. Specifically, we examined: 1) the response properties of object selectivity to the preferred category; 2) the response properties of invariance to contrast and a concurrently presented non-preferred category; 3) whether there are asymmetrical changes of response properties across the boundary from the FFA to PPA versus from the PPA to FFA.

RESULTS

We found that the response properties of FFA are highly selective and reliably invariant, whereas the responses of PPA vary with the image contrast and concurrently presented face. Moreover, the response properties across the boundary between the FFA and PPA are asymmetrical from face-selective to house-selective relative to from house-selective to face-selective.

CONCLUSIONS

These results convergently revealed distinct response properties between the FFA to faces and the PPA to houses, implying a combination of spatially discrete domain-specific and relatively distributed domain-general organization mapping in human ventral temporal cortex.

Statistical Learning in Vision

Vision and learning have long been considered to be two areas of research linked only distantly. However, recent developments in vision research have changed the conceptual definition of vision from a signal-evaluating process to a goal-oriented interpreting process, and this shift binds learning, together with the resulting internal representations, intimately to vision. In this review, we consider various types of learning (perceptual, statistical, and rule/abstract) associated with vision in the past decades and argue that they represent differently specialized versions of the fundamental learning process, which must be captured in its entirety when applied to complex visual processes. We show why the generalized version of statistical learning can provide the appropriate setup for such a unified treatment of learning in vision, what computational framework best accommodates this kind of statistical learning, and what plausible neural scheme could feasibly implement this framework. Finally, we list the challenges that the field of statistical learning faces in fulfilling the promise of being the right vehicle for advancing our understanding of vision in its entirety. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Toward non-metameric reflectance recovery by emulating the spectral neighborhood using corresponding color information

In learning-based reflectance reconstruction methods, usually localized training samples are used to reconstruct spectral curves. The state-of-the-art methods localize the training samples based on their colorimetric color differences with the test sample. This approach is dependent on the working color space, color difference equation, and/or illuminant used, and it may result in a metameric match. This issue can be resolved by localizing the training samples based on their spectral difference with the test sample; however, this would require an already unknown spectral curve of the test sample. In this paper, use of corresponding color information to emulate the spectral neighborhood of the test color for non-metameric reflectance recovery is proposed. The Wiener estimation method was extended by (1) using two thresholds, (i) on the color difference between the test sample and the training samples under the reference illuminant and (ii) on the color difference between the corresponding color of the test sample and the training samples under another illuminant, to mimic the spectral neighborhood of the test sample within the gamut of the training data, and (2) also using the tristimulus values of the corresponding color in the regression. Results showed that the proposed extension of the Wiener estimation method improved the reflectance recovery and hence reduced the metamerism.

Autoencoder-based training for multi-illuminant color constancy

Color constancy is an essential component of the human visual system. It enables us to discern the color of objects invariant to the illumination that is present. This ability is difficult to reproduce in software, as the underlying problem is ill posed, i.e., for each pixel in the image, we know only the RGB values, which are a product of the spectral characteristics of the illumination and the reflectance of objects, as well as the sensitivity of the sensor. To combat this, additional assumptions about the scene have to be made. These assumptions can be either handcrafted or learned using some deep learning technique. Nonetheless, they mostly work only for single illuminant images. In this work, we propose a method for learning these assumptions for multi-illuminant scenes using an autoencoder trained to reconstruct the original image by splitting it into its illumination and reflectance components. We then show that the estimation can be used as is or can be used alongside a clustering method to create a segmentation map of illuminations. We show that our method performs the best out of all tested methods in multi-illuminant scenes while being completely invariant to the number of illuminants.

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers' internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest - within a factor of two of the equivalent noise level set by internal noise.

Human color constancy based on the geometry of color distributions

The physical inputs to our visual system are dictated by the interplay between lights and surfaces; thus, for surface color to be stably perceived, the influence of the illuminant must be discounted. To reveal our strategy to infer the illuminant color, we conducted three psychophysical experiments designed to test our optimal color hypothesis that we internalize the physical color gamut under various illuminants and apply the prior to estimate the illuminant color. In each experiment, we presented 61 hexagons arranged without spatial gaps, where the surrounding 60 hexagons were set to have a specific shape in their color distribution. We asked participants to adjust the color of a center test field so that it appeared to be a full-white surface placed under a test illuminant. Results and computational modeling suggested that, although our proposed model is limited in accounting for estimation of illuminant intensity by human observers, it agrees fairly well with the estimates of illuminant chromaticity in most tested conditions. The accuracy of estimation generally outperformed other tested conventional color constancy models. These results support the hypothesis that our visual system can utilize the geometry of scene color distribution to achieve color constancy.

Physically Plausible Spectral Reconstruction

Spectral reconstruction algorithms recover spectra from RGB sensor responses. Recent methods—with the very best algorithms using deep learning—can already solve this problem with good spectral accuracy. However, the recovered spectra are physically incorrect in that they do not induce the RGBs from which they are recovered. Moreover, if the exposure of the RGB image changes then the recovery performance often degrades significantly—i.e., most contemporary methods only work for a fixed exposure. In this paper, we develop a physically accurate recovery method: the spectra we recover provably induce the same RGBs. Key to our approach is the idea that the set of spectra that integrate to the same RGB can be expressed as the sum of a unique fundamental metamer (spanned by the camera’s spectral sensitivities and linearly related to the RGB) and a linear combination of a vector space of metameric blacks (orthogonal to the spectral sensitivities). Physically plausible spectral recovery resorts to finding a spectrum that adheres to the fundamental metamer plus metameric black decomposition. To further ensure spectral recovery that is robust to changes in exposure, we incorporate exposure changes in the training stage of the developed method. In experiments we evaluate how well the methods recover spectra and predict the actual RGBs and RGBs under different viewing conditions (changing illuminations and/or cameras). The results show that our method generally improves the state-of-the-art spectral recovery (with more stabilized performance when exposure varies) and provides zero colorimetric error. Moreover, our method significantly improves the color fidelity under different viewing conditions, with up to a 60% reduction in some cases.

Color Constancy by Reweighting Image Feature Maps

In this study, a novel illuminant color estimation framework is proposed for computational color constancy, which incorporates the high representational capacity of deep-learningbased models and the great interpretability of assumptionbased models. The well-designed building block, feature map reweight unit (ReWU), helps to achieve comparative accuracy on benchmark datasets with respect to prior state-of-the-art deep learning based models while requiring more compact model size and cheaper computational cost. In addition to local color estimation, a confidence estimation branch is also included such that the model is able to simultaneously produce point estimate and its uncertainty estimate, which provides useful clues for local estimates aggregation and multiple illumination estimation. The source code and the dataset have been made available1.

Bayesian Decision Models: A Primer

To understand decision-making behavior in simple, controlled environments, Bayesian models are often useful. First, optimal behavior is always Bayesian. Second, even when behavior deviates from optimality, the Bayesian approach offers candidate models to account for suboptimalities. Third, a realist interpretation of Bayesian models opens the door to studying the neural representation of uncertainty. In this tutorial, we review the principles of Bayesian models of decision making and then focus on five case studies with exercises. We conclude with reflections and future directions.

Lightness Discrimination Depends More on Bright Rather Than Shaded Regions of Three-Dimensional Objects

The brighter portions of a shaded complex object are in principle more informative about its lightness and are preferentially fixated during lightness judgments. In this study, we investigate whether preventing this strategy also has measurable detrimental effects on performance. Observers were presented with a reference and a comparison three-dimensional rendered object and had to choose which one was "painted with a lighter gray." The comparison was rendered with different diffuse reflectance values. We compared precision between three different conditions: full image, 20% of the lightest pixels removed, or 20% of the darkest pixels removed. Removing the bright pixels maximally impaired performance. The results confirm that the strategy of relying on the brightest areas of a complex object in order to estimate lightness is functionally optimal, yielding more precise representations.

Hyperspectral environmental illumination maps: characterizing directional spectral variation in natural environments

Objects placed in real-world scenes receive incident light from every direction, and the spectral content of this light may vary from one direction to another. In computer graphics, environmental illumination is approximated using maps that specify illumination at a point as a function of incident angle. However, to-date, existing public databases of environmental illumination specify only three colour channels (RGB). We have captured a new set of 12 environmental illumination maps (eight outdoor scenes; four indoor scenes) using a hyperspectral imaging system with 33 spectral channels. The data reveal a striking directional variation of spectral distribution of lighting in natural environments. We discuss limitations of using daylight models to describe natural environmental illumination.

Illumination discrimination for chromatically biased illuminations: Implications for color constancy

We measured discrimination thresholds for illumination changes along different chromatic directions starting from chromatically biased reference illuminations. Participants viewed a Mondrian-papered scene illuminated by LED lamps. The scene was first illuminated by a reference illumination, followed by two comparisons. One comparison matched the reference (the target); the other (the test) varied from the reference, nominally either bluer, yellower, redder, or greener. The participant's task was to correctly select the target. A staircase procedure found thresholds for discrimination of an illumination change along each axis of chromatic change. Nine participants completed the task for five different reference illumination conditions (neutral, blue, yellow, red, and green). We find that relative discrimination thresholds for different chromatic directions of illumination change vary with the reference illumination. For the neutral reference, there is a trend for thresholds to be highest in the bluer illumination-change direction, replicating our previous reports of a "blue bias" for neutral reference illuminations. For the four chromatic references (blue, yellow, red, and green), the change in illumination toward the neutral reference is less well discriminated than changes in the other directions: a "neutral bias." The results have implications for color constancy: In considering the stability of surface appearance under changes in illumination, both the starting chromaticity of the illumination and direction of change must be considered, as well as the chromatic characteristics of the surface reflectance ensemble. They also suggest it will be worthwhile to explore whether and how the human visual system has internalized the statistics of natural illumination changes.

Optimality and heuristics in perceptual neuroscience

The foundation for modern understanding of how we make perceptual decisions about what we see or where to look comes from considering the optimal way to perform these behaviors. While statistical computation is useful for deriving the optimal solution to a perceptual problem, optimality requires perfect knowledge of priors and often complex computation. Accumulating evidence, however, suggests that optimal perceptual goals can be achieved or approximated more simply by human observers using heuristic approaches. Perceptual neuroscientists captivated by optimal explanations of sensory behaviors will fail in their search for the neural circuits and cortical processes that implement an optimal computation whenever that behavior is actually achieved through heuristics. This article provides a cross-disciplinary review of decision-making with the aim of building perceptual theory that uses optimality to set the computational goals for perceptual behavior but, through consideration of ecological, computational, and energetic constraints, incorporates how these optimal goals can be achieved through heuristic approximation.

Computational luminance constancy from naturalistic images

The human visual system supports stable percepts of object color even though the light that reflects from object surfaces varies significantly with the scene illumination. To understand the computations that support stable color perception, we study how estimating a target object's luminous reflectance factor (LRF; a measure of the light reflected from the object under a standard illuminant) depends on variation in key properties of naturalistic scenes. Specifically, we study how variation in target object reflectance, illumination spectra, and the reflectance of background objects in a scene impact estimation of a target object's LRF. To do this, we applied supervised statistical learning methods to the simulated excitations of human cone photoreceptors, obtained from labeled naturalistic images. The naturalistic images were rendered with computer graphics. The illumination spectra of the light sources and the reflectance spectra of the surfaces in the scene were generated using statistical models of natural spectral variation. Optimally decoding target object LRF from the responses of a small learned set of task-specific linear receptive fields that operate on a contrast representation of the cone excitations yields estimates that are within 13% of the correct LRF. Our work provides a framework for evaluating how different sources of scene variability limit performance on luminance constancy.

Colour Constancy Beyond the Classical Receptive Field

The problem of removing illuminant variations to preserve the colours of objects (colour constancy) has already been solved by the human brain using mechanisms that rely largely on centre-surround computations of local contrast. In this paper we adopt some of these biological solutions described by long known physiological findings into a simple, fully automatic, functional model (termed Adaptive Surround Modulation or ASM). In ASM, the size of a visual neuron's receptive field (RF) as well as the relationship with its surround varies according to the local contrast within the stimulus, which in turn determines the nature of the centre-surround normalisation of cortical neurons higher up in the processing chain. We modelled colour constancy by means of two overlapping asymmetric Gaussian kernels whose sizes are adapted based on the contrast of the surround pixels, resembling the change of RF size. We simulated the contrast-dependent surround modulation by weighting the contribution of each Gaussian according to the centre-surround contrast. In the end, we obtained an estimation of the illuminant from the set of the most activated RFs' outputs. Our results on three single-illuminant and one multi-illuminant benchmark datasets show that ASM is highly competitive against the state-of-the-art and it even outperforms learning-based algorithms in one case. Moreover, the robustness of our model is more tangible if we consider that our results were obtained using the same parameters for all datasets, that is, mimicking how the human visual system operates. These results suggest a dynamical adaptation mechanisms contribute to achieving higher accuracy in computational colour constancy.

Improving Color Constancy in an Ambient Light Environment Using the Phong Reflection Model

We present a physics-based illumination estimation approach explicitly designed to handle natural images under ambient light. Existing physics-based color constancy methods are theoretically perfect but do not handle real-world images well because the majority of these methods assume a single illuminant. Therefore, specular pixels selected using existing methods produce estimated dichromatic lines that are thick or curvilinear in the presence of ambient light, thus generating significant errors. Based on the Phong reflection model, we show that a group of specular pixels on a uniformly colored object, although they are subject to intensity thresholding, produce a unique dichromatic line length depending on the geometry of each image path. Assuming that the longest dichromatic line is the most desirable when estimating the chromaticity of an illuminant, ambient-robust specular pixels are also found on the same path on which the longest dichromatic line segment is generated. Therefore, we propose a method to find the optimal image path in which the specular pixels produce the longest dichromatic line. Even though the number of collected specular pixels is reduced using the proposed method, they are proven to be more accurate when determining the illuminant chromaticity even in the existing methods. Experiments with an established benchmark data set and a self-produced image set find that the proposed method is better able to locate the illuminant chromaticity compared with the state-of-the-art color constancy methods.

Color Constancy in Two-Dimensional and Three-Dimensional Scenes: Effects of Viewing Methods and Surface Texture

There has been debate about how and why color constancy may be better in three-dimensional (3-D) scenes than in two-dimensional (2-D) scenes. Although some studies have shown better color constancy for 3-D conditions, the role of specific cues remains unclear. In this study, we compared color constancy for a 3-D miniature room (a real scene consisting of actual objects) and 2-D still images of that room presented on a monitor using three viewing methods: binocular viewing, monocular viewing, and head movement. We found that color constancy was better for the 3-D room; however, color constancy for the 2-D image improved when the viewing method caused the scene to be perceived more like a 3-D scene. Separate measurements of the perceptual 3-D effect of each viewing method also supported these results. An additional experiment comparing a miniature room and its image with and without texture suggested that surface texture of scene objects contributes to color constancy.

Auto white balance by surface reflection decomposition

Auto white balance (AWB) is an important operation in color imaging applications. Most existing AWB algorithms rely on some physical features and statistical properties of natural scenes. However, the AWB algorithms using statistical properties are sensitive to the statistics of the scene contents. Therefore, it is highly desirable to find physical features that are more robust and relatively insensitive to scene contents. In this paper, we propose such physical features based on surface reflection decomposition. Light reflection from most object surfaces can be decomposed into a specular component and a diffuse component. Instead of trying to find the common axis of the color planes as in past algorithms, we estimate the illuminant chromaticity by searching through the light source candidates to find the one that will best cancel the specular components. We provide two formulations: the minimum projected area algorithm and the minimum total variation algorithm for estimation of the scene-illuminant chromaticity. Both show very favorable results compared with other published algorithms.

What #theDress reveals about the role of illumination priors in color perception and color constancy

The disagreement between people who named #theDress (the Internet phenomenon of 2015) "blue and black" versus "white and gold" is thought to be caused by individual differences in color constancy. It is hypothesized that observers infer different incident illuminations, relying on illumination "priors" to overcome the ambiguity of the image. Different experiences may drive the formation of different illumination priors, and these may be indicated by differences in chronotype. We assess this hypothesis, asking whether matches to perceived illumination in the image and/or perceived dress colors relate to scores on the morningness-eveningness questionnaire (a measure of chronotype). We find moderate correlations between chronotype and illumination matches (morning types giving bluer illumination matches than evening types) and chronotype and dress body matches, but these are significant only at the 10% level. Further, although inferred illumination chromaticity in the image explains variation in the color matches to the dress (confirming the color constancy hypothesis), color constancy thresholds obtained using an established illumination discrimination task are not related to dress color perception. We also find achromatic settings depend on luminance, suggesting that subjective white point differences may explain the variation in dress color perception only if settings are made at individually tailored luminance levels. The results of such achromatic settings are inconsistent with their assumed correspondence to perceived illumination. Finally, our results suggest that perception and naming are disconnected, with observers reporting different color names for the dress photograph and their isolated color matches, the latter best capturing the variation in the matches.

Improving color constancy by discounting the variation of camera spectral sensitivity

It is an ill-posed problem to recover the true scene colors from a color biased image by discounting the effects of scene illuminant and camera spectral sensitivity (CSS) at the same time. Most color constancy (CC) models have been designed to first estimate the illuminant color, which is then removed from the color biased image to obtain an image taken under white light, without the explicit consideration of CSS effect on CC. This paper first studies the CSS effect on illuminant estimation arising in the inter-dataset-based CC (inter-CC), i.e., training a CC model on one dataset and then testing on another dataset captured by a distinct CSS. We show the clear degradation of existing CC models for inter-CC application. Then a simple way is proposed to overcome such degradation by first learning quickly a transform matrix between the two distinct CSSs (CSS-1 and CSS-2). The learned matrix is then used to convert the data (including the illuminant ground truth and the color-biased images) rendered under CSS-1 into CSS-2, and then train and apply the CC model on the color-biased images under CSS-2 without the need of burdensome acquiring of the training set under CSS-2. Extensive experiments on synthetic and real images show that our method can clearly improve the inter-CC performance for traditional CC algorithms. We suggest that, by taking the CSS effect into account, it is more likely to obtain the truly color constant images invariant to the changes of both illuminant and camera sensors.

Illuminant estimation in multispectral imaging

With the advancement in sensor technology, the use of multispectral imaging is gaining wide popularity for computer vision applications. Multispectral imaging is used to achieve better discrimination between the radiance spectra, as compared to the color images. However, it is still sensitive to illumination changes. This study evaluates the potential evolution of illuminant estimation models from color to multispectral imaging. We first present a state of the art on computational color constancy and then extend a set of algorithms to use them in multispectral imaging. We investigate the influence of camera spectral sensitivities and the number of channels. Experiments are performed on simulations over hyperspectral data. The outcomes indicate that extension of computational color constancy algorithms from color to spectral gives promising results and may have the potential to lead towards efficient and stable representation across illuminants. However, this is highly dependent on spectral sensitivities and noise. We believe that the development of illuminant invariant multispectral imaging systems will be a key enabler for further use of this technology.

Multiple Illuminant Color Estimation via Statistical Inference on Factor Graphs

This paper presents a method to recover a spatially varying illuminant color estimate from scenes lit by multiple light sources. Starting with the image formation process, we formulate the illuminant recovery problem in a statistically data-driven setting. To do this, we use a factor graph defined across the scale space of the input image. In the graph, we utilize a set of illuminant prototypes computed using a data driven approach. As a result, our method delivers a pixelwise illuminant color estimate being devoid of libraries or user input. The use of a factor graph also allows for the illuminant estimates to be recovered making use of a maximum a posteriori inference process. Moreover, we compute the probability marginals by performing a Delaunay triangulation on our factor graph. We illustrate the utility of our method for pixelwise illuminant color recovery on widely available data sets and compare against a number of alternatives. We also show sample color correction results on real-world images.

Lightness Constancy in Surface Visualization

Color is a common channel for displaying data in surface visualization, but is affected by the shadows and shading used to convey surface depth and shape. Understanding encoded data in the context of surface structure is critical for effective analysis in a variety of domains, such as in molecular biology. In the physical world, lightness constancy allows people to accurately perceive shadowed colors; however, its effectiveness in complex synthetic environments such as surface visualizations is not well understood. We report a series of crowdsourced and laboratory studies that confirm the existence of lightness constancy effects for molecular surface visualizations using ambient occlusion. We provide empirical evidence of how common visualization design decisions can impact viewers' abilities to accurately identify encoded surface colors. These findings suggest that lightness constancy aids in understanding color encodings in surface visualization and reveal a correlation between visualization techniques that improve color interpretation in shadow and those that enhance perceptions of surface depth. These results collectively suggest that understanding constancy in practice can inform effective visualization design.

The Interface Theory of Perception

Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natural selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and allow them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that veridical perceptions--strategies tuned to the true structure of the world--are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a window on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our perceptions of space-time and objects do not entail (by the Invention of Space-Time Theorem) that objective reality has the structure of space-time and objects. An interface serves to guide useful actions, not to resemble truth. Indeed, an interface hides the truth; for someone editing a paper or photo, seeing transistors and firmware is an irrelevant hindrance. For the perceptions of H. sapiens, space-time is the desktop and physical objects are the icons. Our perceptions of space-time and objects have been shaped by natural selection to hide the truth and guide adaptive behaviors. Perception is an adaptive interface.

Color Restoration of RGBN Multispectral Filter Array Sensor Images Based on Spectral Decomposition

A multispectral filter array (MSFA) image sensor with red, green, blue and near-infrared (NIR) filters is useful for various imaging applications with the advantages that it obtains color information and NIR information simultaneously. Because the MSFA image sensor needs to acquire invisible band information, it is necessary to remove the IR cut-offfilter (IRCF). However, without the IRCF, the color of the image is desaturated by the interference of the additional NIR component of each RGB color channel. To overcome color degradation, a signal processing approach is required to restore natural color by removing the unwanted NIR contribution to the RGB color channels while the additional NIR information remains in the N channel. Thus, in this paper, we propose a color restoration method for an imaging system based on the MSFA image sensor with RGBN filters. To remove the unnecessary NIR component in each RGB color channel, spectral estimation and spectral decomposition are performed based on the spectral characteristics of the MSFA sensor. The proposed color restoration method estimates the spectral intensity in NIR band and recovers hue and color saturation by decomposing the visible band component and the NIR band component in each RGB color channel. The experimental results show that the proposed method effectively restores natural color and minimizes angular errors.

Part Boundaries Alter the Perception of Transparency

The perception of transparency is a remarkable feat of human vision: A single stimulation at the retina is interpreted as arising from two (or more) distinct surfaces, separated in depth, in the same visual direction. This feat is intriguing because physical transparency is neither necessary nor sufficient for phenomenal transparency. Many conditions for phenomenal transparency have been studied, including luminance, chromaticity, stereo depth, apparent motion, and structure from motion. Figural conditions have also been studied, primarily by Gestalt psychologists, resulting in descriptive laws. Here we extend, and make precise, these laws using the genericity principle and the minima rule for part boundaries. We report experiments that support the psychological plausibility of these refinements. The results suggest that the formation of visual objects and their parts is an early process in human vision that can precede the representation of transparency.

Modelling colour constancy in fish: implications for vision and signalling in water

Colour vision and colour signals are important to aquatic animals, but light scattering and absorption by water distorts spectral stimuli. To investigate the performance of colour vision in water, and to suggest how photoreceptor spectral sensitivities and body colours might evolve for visual communication, we model the effects of changes in viewing distance and depth on the appearance of fish colours for three teleosts: a barracuda, Sphyraena helleri, which is dichromatic and two damselfishes, Chromis verater and Chromis hanui, which are trichromatic. We assume that photoreceptors light-adapt to the background, thereby implementing the von Kries transformation, which can largely account for observed colour constancy in humans and other animals, including fish. This transformation does not, however, compensate for light scattering over variable viewing distances, which in less than a metre seriously impairs dichromatic colour vision, and makes judgement of colour saturation unreliable for trichromats. The von Kries transformation does substantially offset colour shifts caused by changing depth, so that from depths of 0 to 30 m modelled colour changes (i.e. failures of colour constancy) are sometimes negligible. However, the magnitudes and directions of remaining changes are complex, depending upon the specific spectral sensitivities of the receptors and the reflectance spectra. This predicts that when judgement of colour is important, the spectra of signalling colours and photoreceptor spectral sensitivities should be evolutionarily linked, with the colours dependent on photoreceptor spectral sensitivities, and vice versa.

Color and the Cone Mosaic

Vision is limited by the measurements taken by the cone photoreceptors. To provide useful perceptual representations, the brain must go beyond the measurements and make inferences about the scene being viewed. This article considers the first stages of spatiochromatic vision. We show how spatial and chromatic information become intertwined by the optics of the eye and because of the structure of the retinal cone mosaic, and we consider the consequent implications for perception. Because there is at most one cone at each retinal location, the standard treatment of human trichromacy does not apply at fine spatial scales. Rather, trichromacy results from a perceptual inference based on measurements from cones of different classes at different locations. Our treatment emphasizes linking physics, biology, and computation with the goal of providing a framework for a larger understanding of how the brain interprets photoreceptor excitations to see objects and their properties.

Color Constancy Using Double-Opponency

The double-opponent (DO) color-sensitive cells in the primary visual cortex (V1) of the human visual system (HVS) have long been recognized as the physiological basis of color constancy. In this work we propose a new color constancy model by imitating the functional properties of the HVS from the single-opponent (SO) cells in the retina to the DO cells in V1 and the possible neurons in the higher visual cortexes. The idea behind the proposed double-opponency based color constancy (DOCC) model originates from the substantial observation that the color distribution of the responses of DO cells to the color-biased images coincides well with the vector denoting the light source color. Then the illuminant color is easily estimated by pooling the responses of DO cells in separate channels in LMS space with the pooling mechanism of sum or max. Extensive evaluations on three commonly used datasets, including the test with the dataset dependent optimal parameters, as well as the intra- and inter-dataset cross validation, show that our physiologically inspired DOCC model can produce quite competitive results in comparison to the state-of-the-art approaches, but with a relative simple implementation and without requiring fine-tuning of the method for each different dataset.

Shape, Illumination, and Reflectance from Shading

A fundamental problem in computer vision is that of inferring the intrinsic, 3D structure of the world from flat, 2D images of that world. Traditional methods for recovering scene properties such as shape, reflectance, or illumination rely on multiple observations of the same scene to overconstrain the problem. Recovering these same properties from a single image seems almost impossible in comparison-there are an infinite number of shapes, paint, and lights that exactly reproduce a single image. However, certain explanations are more likely than others: surfaces tend to be smooth, paint tends to be uniform, and illumination tends to be natural. We therefore pose this problem as one of statistical inference, and define an optimization problem that searches for the most likely explanation of a single image. Our technique can be viewed as a superset of several classic computer vision problems (shape-from-shading, intrinsic images, color constancy, illumination estimation, etc) and outperforms all previous solutions to those constituent problems.

Color correction using root-polynomial regression

Cameras record three color responses (RGB) which are device dependent. Camera coordinates are mapped to a standard color space, such as XYZ-useful for color measurement-by a mapping function, e.g., the simple 3×3 linear transform (usually derived through regression). This mapping, which we will refer to as linear color correction (LCC), has been demonstrated to work well in the number of studies. However, it can map [Formula: see text] to XYZs with high error. The advantage of the LCC is that it is independent of camera exposure. An alternative and potentially more powerful method for color correction is polynomial color correction (PCC). Here, the R, G, and B values at a pixel are extended by the polynomial terms. For a given calibration training set PCC can significantly reduce the colorimetric error. However, the PCC fit depends on exposure, i.e., as exposure changes the vector of polynomial components is altered in a nonlinear way which results in hue and saturation shifts. This paper proposes a new polynomial-type regression loosely related to the idea of fractional polynomials which we call root-PCC (RPCC). Our idea is to take each term in a polynomial expansion and take its k th root of each k -degree term. It is easy to show terms defined in this way scale with exposure. RPCC is a simple (low complexity) extension of LCC. The experiments presented in this paper demonstrate that RPCC enhances color correction performance on real and synthetic data.

Color constancy in a naturalistic, goal-directed task

In daily life, we use color information to select objects that will best serve a particular goal (e.g., pick the best-tasting fruit or avoid spoiled food). This is challenging when judgments must be made across changes in illumination as the spectrum reflected from an object to the eye varies with the illumination. Color constancy mechanisms serve to partially stabilize object color appearance across illumination changes, but whether and to what degree constancy supports accurate cross-illumination object selection is not well understood. To get closer to understanding how constancy operates in real-life tasks, we developed a paradigm in which subjects engage in a goal-directed task for which color is instrumental. Specifically, in each trial, subjects re-created an arrangement of colored blocks (the model) across a change in illumination. By analyzing the re-creations, we were able to infer and quantify the degree of color constancy that mediated subjects' performance. In Experiments 1 and 2, we used our paradigm to characterize constancy for two different sets of block reflectances, two different illuminant changes, and two different groups of subjects. On average, constancy was good in our naturalistic task, but it varied considerably across subjects. In Experiment 3, we tested whether varying scene complexity and the validity of local contrast as a cue to the illumination change modulated constancy. Increasing complexity did not lead to improved constancy; silencing local contrast significantly reduced constancy. Our results establish a novel goal-directed task that enables us to approach color constancy as it emerges in real life.

Camera processing with chromatic aberration

Since the refractive index of materials commonly used for lens depends on the wavelengths of light, practical camera optics fail to converge light to a single point on an image plane. Known as chromatic aberration, this phenomenon distorts image details by introducing magnification error, defocus blur, and color fringes. Though achromatic and apochromatic lens designs reduce chromatic aberration to a degree, they are complex and expensive and they do not offer a perfect correction. In this paper, we propose a new postcapture processing scheme designed to overcome these problems computationally. Specifically, the proposed solution is comprised of chromatic aberration-tolerant demosaicking algorithm and post-demosaicking chromatic aberration correction. Experiments with simulated and real sensor data verify that the chromatic aberration is effectively corrected.

Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution

Color constancy is a well-studied topic in color vision. Methods are generally categorized as (1) low-level statistical methods, (2) gamut-based methods, and (3) learning-based methods. In this work, we distinguish methods depending on whether they work directly from color values (i.e., color domain) or from values obtained from the image's spatial information (e.g., image gradients/frequencies). We show that spatial information does not provide any additional information that cannot be obtained directly from the color distribution and that the indirect aim of spatial-domain methods is to obtain large color differences for estimating the illumination direction. This finding allows us to develop a simple and efficient illumination estimation method that chooses bright and dark pixels using a projection distance in the color distribution and then applies principal component analysis to estimate the illumination direction. Our method gives state-of-the-art results on existing public color constancy datasets as well as on our newly collected dataset (NUS dataset) containing 1736 images from eight different high-end consumer cameras.

Color constancy influenced by unnatural spatial structure

The recognition of spatial structures is important for color constancy because we cannot identify an object's color under different illuminations without knowing which space it is in and how that space is illuminated. To show the importance of the natural structure of environments on color constancy, we investigated the way in which color appearance was affected by unnatural viewing conditions in which a spatial structure was distorted. Observers judged the color of a test patch placed in the center of a small room illuminated by white or reddish lights, as well as two rooms illuminated by white and reddish light, respectively. In the natural viewing condition, an observer saw the room(s) through a viewing window, whereas in an unnatural viewing condition, the scene structure was scrambled by a kaleidoscope-type viewing box. Results of single room condition with one illuminant color showed little difference in color constancy between the two viewing conditions. However, it decreased in the two-rooms condition with a more complex arrangement of space and illumination. The patch's appearance under the unnatural viewing condition was more influenced by simultaneous contrast than its appearance under the natural viewing condition. It also appears that color appearance under white illumination is more stable compared to that under reddish illumination. These findings suggest that natural spatial structure plays an important role for color constancy in a complex environment.

Evaluating combinational illumination estimation methods on real-world images

Illumination estimation is an important component of color constancy and automatic white balancing. A number of methods of combining illumination estimates obtained from multiple subordinate illumination estimation methods now appear in the literature. These combinational methods aim to provide better illumination estimates by fusing the information embedded in the subordinate solutions. The existing combinational methods are surveyed and analyzed here with the goals of determining: 1) the effectiveness of fusing illumination estimates from multiple subordinate methods; 2) the best method of combination; 3) the underlying factors that affect the performance of a combinational method; and 4) the effectiveness of combination for illumination estimation in multiple-illuminant scenes. The various combinational methods are categorized in terms of whether or not they require supervised training and whether or not they rely on high-level scene content cues (e.g., indoor versus outdoor). Extensive tests and enhanced analyzes using three data sets of real-world images are conducted. For consistency in testing, the images were labeled according to their high-level features (3D stages, indoor/outdoor) and this label data is made available on-line. The tests reveal that the trained combinational methods (direct combination by support vector regression in particular) clearly outperform both the non-combinational methods and those combinational methods based on scene content cues.

One and done? Optimal decisions from very few samples

In many learning or inference tasks human behavior approximates that of a Bayesian ideal observer, suggesting that, at some level, cognition can be described as Bayesian inference. However, a number of findings have highlighted an intriguing mismatch between human behavior and standard assumptions about optimality: People often appear to make decisions based on just one or a few samples from the appropriate posterior probability distribution, rather than using the full distribution. Although sampling-based approximations are a common way to implement Bayesian inference, the very limited numbers of samples often used by humans seem insufficient to approximate the required probability distributions very accurately. Here, we consider this discrepancy in the broader framework of statistical decision theory, and ask: If people are making decisions based on samples--but as samples are costly--how many samples should people use to optimize their total expected or worst-case reward over a large number of decisions? We find that under reasonable assumptions about the time costs of sampling, making many quick but locally suboptimal decisions based on very few samples may be the globally optimal strategy over long periods. These results help to reconcile a large body of work showing sampling-based or probability matching behavior with the hypothesis that human cognition can be understood in Bayesian terms, and they suggest promising future directions for studies of resource-constrained cognition.

Multi-illuminant estimation with conditional random fields

Most existing color constancy algorithms assume uniform illumination. However, in real-world scenes, this is not often the case. Thus, we propose a novel framework for estimating the colors of multiple illuminants and their spatial distribution in the scene. We formulate this problem as an energy minimization task within a conditional random field over a set of local illuminant estimates. In order to quantitatively evaluate the proposed method, we created a novel data set of two-dominant-illuminant images comprised of laboratory, indoor, and outdoor scenes. Unlike prior work, our database includes accurate pixel-wise ground truth illuminant information. The performance of our method is evaluated on multiple data sets. Experimental results show that our framework clearly outperforms single illuminant estimators as well as a recently proposed multi-illuminant estimation approach.