The Müller-Lyer illusion explained by the statistics of image-source relationships

The Müller-Lyer effect, the apparent difference in the length of a line as the result of its adornment with arrowheads or arrow tails, is the best known and most controversial of the classical geometrical illusions. By sampling a range-image database of natural scenes, we show that the perceptual effects elicited by the Müller-Lyer stimulus and its major variants are correctly predicted by the probability distributions of the possible physical sources underlying the relevant retinal images. These results support the conclusion that the Müller-Lyer illusion is a manifestation of the probabilistic strategy of visual processing that has evolved to contend with the uncertain provenance of retinal stimuli.

Natural-scene geometry predicts the perception of angles and line orientation

Visual stimuli that entail the intersection of two or more straight lines elicit a variety of well known perceptual anomalies. Preeminent among these anomalies are the systematic overestimation of acute angles, the underestimation of obtuse angles, and the misperceptions of line orientation exemplified in the classical tilt, Zollner, and Hering illusions. Here we show that the probability distributions of the possible real-world sources of projected lines and angles derived from a range-image database of natural scenes accurately predict each of these perceptual peculiarities. These findings imply that the perception of angles and oriented lines is determined by the statistical relationship between geometrical stimuli and their physical sources in typical visual environments.

The statistical structure of natural light patterns determines perceived light intensity

The same target luminance in different contexts can elicit markedly different perceptions of brightness, a fact that has long puzzled vision scientists. Here we test the proposal that the visual system encodes not luminance as such but rather the statistical relationship of a particular luminance to all possible luminance values experienced in natural contexts during evolution. This statistical conception of vision was validated by using a database of natural scenes in which we could determine the probability distribution functions of co-occurring target and contextual luminance values. The distribution functions obtained in this way predict target brightness in response to a variety of challenging stimuli, thus explaining these otherwise puzzling percepts. That brightness is determined by the statistics of natural light patterns implies that the relevant neural circuitry is specifically organized to generate these probabilistic responses.

Size contrast and assimilation explained by the statistics of natural scene geometry

The term "size contrast and assimilation" refers to a large class of geometrical illusions in which the apparent sizes of identical visual targets in various contexts are different. Here we have examined whether these intriguing discrepancies between physical and perceived size can be explained by a visual process in which percepts are determined by the probability distribution of the possible real-world sources of retinal stimuli. To test this idea, we acquired a range image database of natural scenes that specified the location of every image point in 3-D space. By sampling the possible physical sources of various size contrast or assimilation stimuli in the database, we determined the probability distributions of the size of the target in the images generated by these sources. For each of the various stimuli tested, these probability distributions of target size in different contexts accurately predicted the perceptual effects reported in psychophysical studies. We conclude that size contrast and assimilation effects are a further manifestation of a fundamentally probabilistic process of visual perception.

Perceiving the intensity of light

The relationship between luminance (i.e., the photometric intensity of light) and its perception (i.e., sensations of lightness or brightness) has long been a puzzle. In addition to the mystery of why these perceptual qualities do not scale with luminance in any simple way, "illusions" such as simultaneous brightness contrast, Mach bands, Craik-O'Brien-Cornsweet edge effects, and the Chubb-Sperling-Solomon illusion have all generated much interest but no generally accepted explanation. The authors review evidence that the full range of this perceptual phenomenology can be rationalized in terms of an empirical theory of vision. The implication of these observations is that perceptions of lightness and brightness are generated according to the probability distributions of the possible sources of luminance values in stimuli that are inevitably ambiguous.

Natural scene statistics as the universal basis of color context effects

The color context effects referred to as color contrast, constancy, and assimilation underscore the fact that color percepts do not correspond to the spectral characteristics of the generative stimuli. Despite a variety of proposed theories, these phenomena have resisted explanation in a single principled framework. Using a hyperspectral image database of natural scenes, we here show that color contrast, constancy, and assimilation are all predicted by the statistical organization of spectral returns from natural visual environments.

The statistical structure of human speech sounds predicts musical universals

The similarity of musical scales and consonance judgments across human populations has no generally accepted explanation. Here we present evidence that these aspects of auditory perception arise from the statistical structure of naturally occurring periodic sound stimuli. An analysis of speech sounds, the principal source of periodic sound stimuli in the human acoustical environment, shows that the probability distribution of amplitude-frequency combinations in human utterances predicts both the structure of the chromatic scale and consonance ordering. These observations suggest that what we hear is determined by the statistical relationship between acoustical stimuli and their naturally occurring sources, rather than by the physical parameters of the stimulus per se.

Image/source statistics of surfaces in natural scenes

Perceiving surfaces in a manner that accords with their physical properties is essential for successful behaviour. Since, however, a given retinal image can have been generated by an infinite variety of natural surfaces with different geometrical and/or physical qualities, the corresponding percepts cannot be determined by the stimulus per se. Rather, resolution of this quandary requires a strategy of vision that incorporates the statistical relationship of the information in retinal images to its sources in representative environments. To examine this probabilistic relationship with respect to the features of object surfaces, we analysed a database of range images in which the distances of all the objects in a series of natural scenes were measured with respect to the image plane by a laser range scanner. By taking any particular scene obtained in this way to be made up of a set of concatenated surface patches, we were able to explore the statistics of scene roughness, size-distance relationships, surface orientation and local curvature, as well as the independent components of natural surfaces. The relevance of these statistics to both perception and the neuronal organization of the underlying visual circuitry is discussed.

A statistical explanation of visual space

The subjective visual space perceived by humans does not reflect a simple transformation of objective physical space; rather, perceived space has an idiosyncratic relationship with the real world. To date, there is no consensus about either the genesis of perceived visual space or the implications of its peculiar characteristics for visually guided behavior. Here we used laser range scanning to measure the actual distances from the image plane of all unoccluded points in a series of natural scenes. We then asked whether the differences between real and apparent distances could be explained by the statistical relationship of scene geometry and the observer. We were able to predict perceived distances in a variety of circumstances from the probability distribution of physical distances. This finding lends support to the idea that the characteristics of human visual space are determined probabilistically.

The empirical basis of color perception

Rationalizing the perceptual effects of spectral stimuli has been a major challenge in vision science for at least the last 200 years. Here we review evidence that this otherwise puzzling body of phenomenology is generated by an empirical strategy of perception in which the color an observer sees is entirely determined by the probability distribution of the possible sources of the stimulus. The rationale for this strategy in color vision, as in other visual perceptual domains, is the inherent ambiguity of the real-world origins of any spectral stimulus.

A probabilistic explanation of brightness scaling

The perceptions of lightness or brightness elicited by a visual target are linked to its luminance by a nonlinear function that varies according to the physical characteristics of the target and the background on which it is presented. Although no generally accepted explanation of this scaling relationship exists, it has long been considered a byproduct of low- or mid-level visual processing. Here we examine the possibility that brightness scaling is actually the signature of a biological strategy for dealing with inevitably ambiguous visual stimuli, in which percepts of lightness/brightness are determined by the probabilistic relationship between luminances in the image plane and their possible real-world sources.

Range image statistics can explain the anomalous perception of length

A long-standing puzzle in visual perception is that the apparent extent of a spatial interval (e.g., the distance between two points or the length of a line) does not simply accord with the length of the stimulus but varies as a function of orientation in the retinal image. Here, we show that this anomaly can be explained by the statistical relationship between the length of retinal projections and the length of their real-world sources. Using a laser range scanner, we acquired a database of natural images that included the three-dimensional location of every point in the scenes. An analysis of these range images showed that the average length of a physical interval in three-dimensional space changes systematically as a function of the orientation of the corresponding interval in the projected image, the variation being in good agreement with perceived length. This evidence implies that the perception of visual space is determined by the probability distribution of the possible real-world sources of retinal images.

Perception of objects that are translating and rotating

The motion of objects that are both translating and rotating can be decomposed into an infinite number of translational and rotational combinations. How, then, do such stimuli routinely elicit specific percepts and behavioral responses that are usually appropriate? A possible answer is that motion percepts are fully determined by the probability distributions of all the possible correspondences and differences in the stimulus sequence. To test the merits of this conceptual framework, we investigated the perceived motion elicited by a line that is both translating and rotating behind an aperture. When stimuli are presented such that a particular sequence of appearance and disappearance occurs at the aperture boundary, subjects report that the line is rotating only; furthermore, the perceived centers of rotation appear to describe a cycloidal trajectory, even when one aperture shape is replaced by another. These and other perceptual effects elicited by translating and rotating stimuli are all accurately predicted by the probability distribution of the possible sources of the physical movements, supporting the conclusion that motion perception is indeed generated by a wholly probabilistic strategy.

A rationale for the structure of color space

The colors perceived by humans in response to light stimuli are generally described in terms of four color categories (reds, greens, blues and yellows), the members of which are systematically arrayed around gray. This broadly accepted description of color sensation differs fundamentally from the light that induces it, which is neither 'circular' nor categorical. What, then, accounts for these discrepancies between the structure of color experience and the physical reality that underlies it? We suggest that these differences are based on two related requirements for successful color vision: (1) that spectra be ordered according to their physical similarities and differences; and (2) that this ordering be constrained by the four-color map problem.

An empirical explanation of the Chubb illusion

The perceived difference in brightness between elements of a patterned target is diminished when the target is embedded in a similar surround of higher luminance contrast (the Chubb illusion). Here we show that this puzzling effect can be explained by the degree to which imperfect transmittance is likely to have affected the light that reaches the eye. These observations indicate that this 'illusion' is yet another signature of the fundamentally empirical strategy of visual perception, in this case generated by the typical influence of transmittance on inherently ambiguous stimuli.

A wholly empirical explanation of perceived motion

Because the retinal activity generated by a moving object cannot specify which of an infinite number of possible physical displacements underlies the stimulus, its real-world cause is necessarily uncertain. How, then, do observers respond successfully to sequences of images whose provenance is ambiguous? Here we explore the hypothesis that the visual system solves this problem by a probabilistic strategy in which perceived motion is generated entirely according to the relative frequency of occurrence of the physical sources of the stimulus. The merits of this concept were tested by comparing the directions and speeds of moving lines reported by subjects to the values determined by the probability distribution of all the possible physical displacements underlying the stimulus. The velocities reported by observers in a variety of stimulus contexts can be accounted for in this way.

Why we see things the way we do: evidence for a wholly empirical strategy of vision

Many otherwise puzzling aspects of the way we see brightness, colour, orientation and motion can be understood in wholly empirical terms. The evidence reviewed here leads to the conclusion that visual percepts are based on patterns of reflex neural activity shaped entirely by the past success (or failure) of visually guided behaviour in response to the same or a similar retinal stimulus. As a result, the images we see accord with what the sources of the stimuli have typically turned out to be, rather than with the physical properties of the relevant objects. If vision does indeed depend upon this operational strategy to generate optimally useful perceptions of inevitably ambiguous stimuli, then the underlying neurobiological processes will eventually need to be understood within this conceptual framework.

The relevance of visual perception to cortical evolution and development

The quality of brightness--perhaps the simplest visual attribute we perceive--appears to be determined probabilistically. In this empirical conception of the perception of light, the stimulus-induced activity of visual cortical neurons does not encode the retinal image or the properties of the stimulus per se, but associations (percepts) determined by the relative probabilities of the possible sources of the stimulus. If this theory is correct, the rationale for the prolonged postnatal construction of visual circuitry--and the evolution of this visual scheme--is to strengthen and/or create by activity-dependent feedback the empirically determined association on which vision depends.

An empirical explanation of color contrast

For reasons not well understood, the color of a surface can appear quite different when placed in different chromatic surrounds. Here we explore the possibility that these color contrast effects are generated according to what the same or similar stimuli have turned out to signify in the past about the physical relationships between reflectance, illumination, and the spectral returns they produce. This hypothesis was evaluated by (i) comparing the physical relationships of reflectances, illuminants, and spectral returns with the perceptual phenomenology of color contrast and (ii) testing whether perceptions of color contrast are predictably changed by altering the probabilities of the possible sources of the stimulus. The results we describe are consistent with a wholly empirical explanation of color contrast effects.