Black light: how sensors filter spectral variation of the illuminant

Ways of coloring: Comparative color vision as a case study for cognitive science

Different explanations of color vision favor different philosophical positions: Computational vision is more compatible with objectivism (the color is in the object), psychophysics and neurophysiology with subjectivism (the color is in the head). Comparative research suggests that an explanation of color must be both experientialist (unlike objectivism) and ecological (unlike subjectivism). Computational vision's emphasis on optimally “recovering” prespecified features of the environment (i.e., distal properties, independent of the sensory-motor capacities of the animal) is unsatisfactory. Conceiving of visual perception instead as the visual guidance of activity in an environment that is determined largely by that very activity suggests new directions for research.

Analytic solution for separating spectra into illumination and surface reflectance components

The measured light spectrum is the result of an illuminant interacting with a surface. The illuminant spectral power distribution multiplies the surface spectral reflectance function to form a color signal--the light spectrum that gives rise to our perception. Disambiguation of the two factors, illuminant and surface, is difficult without prior knowledge. Previously [IEEE Trans. Pattern Anal. Mach. Intell.12, 966 (1990); J. Opt. Soc. Am. A21, 1825 (2004)], one approach to this problem applied a finite-dimensional basis function model to recover the separate illuminant and surface reflectance components that make up the color signal, using principal component bases for lights and for reflectances. We introduce the idea of making use of finite-dimensional models of logarithms of spectra for this problem. Recognizing that multiplications turn into additions in such a formulation, we can replace the original iterative method with a direct, analytic algorithm with no iteration, resulting in a speedup of several orders of magnitude. Moreover, in the new, logarithm-based approach, it is straightforward to further design new basis functions, for both illuminant and reflectance simultaneously, such that the initial basis function coefficients derived from the input color signal are optimally mapped onto separate coefficients that produce spectra that more closely approximate the illuminant and the surface reflectance for any given dimensionality. This is accomplished by using an extra bias correction step that maps the analytically determined basis function coefficients onto the optimal coefficient set, separately for lights and surfaces, for the training set. The analytic equation plus the bias correction is then used for unknown input color signals.

On ratio-based color indexing

The color ratio approach to indexing has been found to be robust and effective in indexing image and video databases, in different color spaces, and when using transformed color features, such as those from the Karhunen-Loeve transform (KLT) or the discrete cosine transform (DCT). However, the reason for the superior performance of the color ratio model, especially on different color spaces or with transformed color features has, at best, been speculative. This paper develops a generalized form for the color ratio model, based on which we characterize the general distribution of the color ratios. From the distribution, we present a theory that explains and supports the performance of the color ratio approach in image and video indexing. It is shown that the same theory accounts for its effectiveness in different color spaces and in the transform domain. Some general problems encountered in using the original retinex lightness algorithm, and some other issues specific to ratio-based color indexing are discussed in the light of the theory. Results are presented which show that the proposed theory is supported by empirical evidence.

Characterization of natural illuminants in forests and the use of digital video data to reconstruct illuminant spectra

We describe illumination spectra in forests and show that they can be accurately recovered from recorded digital video images. Natural illuminant spectra of 238 samples measured in temperate forests were characterized by principal-component analysis. The spectra can be accurately approximated by the mean and the first two principal components. Compared with illumination under open skies, the loci of forest illuminants are displaced toward the green region in the chromaticity plots, and unlike open sky illumination they cannot be characterized by correlated color temperature. We show that it is possible to recover illuminant spectra accurately from digital video images by a linear least-squares-fit estimation technique. The use of digital video data in spectral analysis provides a promising new approach to the studies of the spatial and temporal variation of illumination in natural scenes and the understanding of color vision in natural environments.

Bayesian color constancy

The problem of color constancy may be solved if we can recover the physical properties of illuminants and surfaces from photosensor responses. We consider this problem within the framework of Bayesian decision theory. First, we model the relation among illuminants, surfaces, and photosensor responses. Second, we construct prior distributions that describe the probability that particular illuminants and surfaces exist in the world. Given a set of photosensor responses, we can then use Bayes's rule to compute the posterior distribution for the illuminants and the surfaces in the scene. There are two widely used methods for obtaining a single best estimate from a posterior distribution. These are maximum a posteriori (MAP) and minimum mean-square-error (MMSE) estimation. We argue that neither is appropriate for perception problems. We describe a new estimator, which we call the maximum local mass (MLM) estimate, that integrates local probability density. The new method uses an optimality criterion that is appropriate for perception tasks: It finds the most probable approximately correct answer. For the case of low observation noise, we provide an efficient approximation. We develop the MLM estimator for the color-constancy problem in which flat matte surfaces are uniformly illuminated. In simulations we show that the MLM method performs better than the MAP estimator and better than a number of standard color-constancy algorithms. We note conditions under which even the optimal estimator produces poor estimates: when the spectral properties of the surfaces in the scene are biased.

RGB calibration for color image analysis in machine vision

A color calibration method for correcting the variations in RGB color values caused by vision system components was developed and tested in this study. The calibration scheme concentrated on comprehensively estimating and removing the RGB errors without specifying error sources and their effects. The algorithm for color calibration was based upon the use of a standardized color chart and developed as a preprocessing tool for color image analysis. According to the theory of image formation, RGB errors in color images were categorized into multiplicative and additive errors. Multiplicative and additive errors contained various error sources-gray-level shift, a variation in amplification and quantization in camera electronics or frame grabber, the change of color temperature of illumination with time, and related factors. The RGB errors of arbitrary colors in an image were estimated from the RGB errors of standard colors contained in the image. The color calibration method also contained an algorithm for correcting the nonuniformity of illumination in the scene. The algorithm was tested under two different conditions-uniform and nonuniform illumination in the scene. The RGB errors of arbitrary colors in test images were almost completely removed after color calibration. The maximum residual error was seven gray levels under uniform illumination and 12 gray levels under nonuniform illumination. Most residual RGB errors were caused by residual nonuniformity of illumination in images, The test results showed that the developed method was effective in correcting the variations in RGB color values caused by vision system components.

Color constancy. III. General linear recovery of spectral descriptions for lights and surfaces

We present a color-constancy algorithm that uses quantum-catch data from reflected lights to recover surface reflectance functions and illuminant spectral power distributions. The algorithm recovers both surface and light-source spectral properties simultaneously. The method works in all situations that were handled by the earlier two-stage algorithms of Maloney and Wandell [J. Opt. Soc. Am. A 3, 29 (1986)] and D'Zmura and Iverson [J. Opt. Soc. Am. A 9, 490 (1992); 10, 2148, 2166 (1993); 11, 1970 (1994)]. In addition, the method handles problems that lie outside the scope of earlier algorithms. Using this method, a trichromatic visual system can recover, when provided adequate information, spectral descriptions of arbitrarily high accuracy for lights and surfaces. We determine conditions under which bilinear models can be used to recover color properties uniquely with the new procedure, and we formulate an algorithm for checking whether a particular bilinear model provides perfect color constancy. This research extends our analysis of linear methods for color constancy begun earlier [J. Opt. Soc. Am. A 10, 2148, 2166 (1993)].