The statistical structure of natural light patterns determines perceived light intensity

Natural Contrast Statistics Facilitate Human Face Categorization

The ability to detect faces in the environment is of utmost ecological importance for human social adaptation. While face categorization is efficient, fast and robust to sensory degradation, it is massively impaired when the facial stimulus does not match the natural contrast statistics of this visual category, i.e., the typically experienced ordered alternation of relatively darker and lighter regions of the face. To clarify this phenomenon, we characterized the contribution of natural contrast statistics to face categorization. Specifically, 31 human adults viewed various natural images of nonface categories at a rate of 12 Hz, with highly variable images of faces occurring every eight stimuli (1.5 Hz). As in previous studies, neural responses at 1.5 Hz as measured with high-density electroencephalography (EEG) provided an objective neural index of face categorization. Here, when face images were shown in their naturally experienced contrast statistics, the 1.5-Hz face categorization response emerged over occipito-temporal electrodes at very low contrast [5.1%, or 0.009 root-mean-square (RMS) contrast], quickly reaching optimal amplitude at 22.6% of contrast (i.e., RMS contrast of 0.041). Despite contrast negation preserving an image's spectral and geometrical properties, negative contrast images required twice as much contrast to trigger a face categorization response, and three times as much to reach optimum. These observations characterize how the internally stored natural contrast statistics of the face category facilitate visual processing for the sake of fast and efficient face categorization.

Cognition as a Mechanical Process

Cognition is often defined as a dual process of physical and non-physical mechanisms. This duality originated from past theory on the constituent parts of the natural world. Even though material causation is not an explanation for all natural processes, phenomena at the cellular level of life are modeled by physical causes. These phenomena include explanations for the function of organ systems, including the nervous system and information processing in the cerebrum. This review restricts the definition of cognition to a mechanistic process and enlists studies that support an abstract set of proximate mechanisms. Specifically, this process is approached from a large-scale perspective, the flow of information in a neural system. Study at this scale further constrains the possible explanations for cognition since the information flow is amenable to theory, unlike a lower-level approach where the problem becomes intractable. These possible hypotheses include stochastic processes for explaining the processes of cognition along with principles that support an abstract format for the encoded information.

Dynamic decorrelation as a unifying principle for explaining a broad range of brightness phenomena

The visual system is highly sensitive to spatial context for encoding luminance patterns. Context sensitivity inspired the proposal of many neural mechanisms for explaining the perception of luminance (brightness). Here we propose a novel computational model for estimating the brightness of many visual illusions. We hypothesize that many aspects of brightness can be explained by a dynamic filtering process that reduces the redundancy in edge representations on the one hand, while non-redundant activity is enhanced on the other. The dynamic filter is learned for each input image and implements context sensitivity. Dynamic filtering is applied to the responses of (model) complex cells in order to build a gain control map. The gain control map then acts on simple cell responses before they are used to create a brightness map via activity propagation. Our approach is successful in predicting many challenging visual illusions, including contrast effects, assimilation, and reverse contrast with the same set of model parameters.

Themes of advanced information processing in the primate brain

Here is a review of several empirical examples of information processing that occur in the primate cerebral cortex. These include visual processing, object identification and perception, information encoding, and memory. Also, there is a discussion of the higher scale neural organization, mainly theoretical, which suggests hypotheses on how the brain internally represents objects. Altogether they support the general attributes of the mechanisms of brain computation, such as efficiency, resiliency, data compression, and a modularization of neural function and their pathways. Moreover, the specific neural encoding schemes are expectedly stochastic, abstract and not easily decoded by theoretical or empirical approaches.

Influences of orientation on the Ponzo, contrast, and Craik-O'Brien-Cornsweet illusions

Explanations of the Ponzo size illusion, the simultaneous contrast illusion, and the Craik-O'Brien-Cornsweet brightness illusions involve either stimulus-driven processes (assimilation, enhanced contrast, and anchoring) or prior experiences. Real-world up-down asymmetries for typical direction of illumination and ground planes in our physical environment should influence these illusions if they are experience based, but not if they are stimulus driven. Results presented here demonstrate differences in illusion strengths between upright and inverted versions of all three illusions. A left-right asymmetry of the Cornsweet illusion was produced by manipulating the direction of illumination, providing further support for the involvement of an experience-based explanation. When the inducers were incompatible with the targets being located at the different distances, the Ponzo illusion persisted and so did the influence from orientation, providing evidence for involvement of processes other than size constancy. As defined here, upright for the brightness illusions is consistent with an interpretation of a shaded bulging surface and a 3D object resulting from a light-from-above assumption triggering compensation for varying illumination. Upright for the Ponzo illusion is consistent with the inducers in the form of converging lines being interpreted as railway tracks receding on the ground triggering size constancy effects. The implications of these results, and other results providing evidence against experience-based accounts of the illusions, are discussed.

Effect of Spatial Structure Defined by Binocular Disparity with Uniform Luminance on Lightness

We examined whether lightness is determined based on the experience of the relationship between a scene’s illumination and its spatial structure in actual environments. For this purpose, we measured some characteristics of scene structure and the illuminance in actual scenes and found some correlations between them. In the psychophysical experiments, a random-dot stereogram consisting of dots with uniform distribution was used to eliminate the effects of local luminance and texture contrasts. Participants matched the lightness of a presented target patch in the stimulus space to that of a comparison patch by adjusting the latter’s luminance. Results showed that the matched luminance tended to increase when the target patch was interpreted as receiving weak illumination in some conditions. These results suggest that the visual system can probably infer a scene’s illumination from a spatial structure without luminance distribution information under an illumination–spatial structure relation.

Representing Color Ensembles

Colors are rarely uniform, yet little is known about how people represent color distributions. We introduce a new method for studying color ensembles based on intertrial learning in visual search. Participants looked for an oddly colored diamond among diamonds with colors taken from either uniform or Gaussian color distributions. On test trials, the targets had various distances in feature space from the mean of the preceding distractor color distribution. Targets on test trials therefore served as probes into probabilistic representations of distractor colors. Test-trial response times revealed a striking similarity between the physical distribution of colors and their internal representations. The results demonstrate that the visual system represents color ensembles in a more detailed way than previously thought, coding not only mean and variance but, most surprisingly, the actual shape (uniform or Gaussian) of the distribution of colors in the environment.

Human vision is determined based on information theory

It is commonly accepted that the evolution of the human eye has been driven by the maximum intensity of the radiation emitted by the Sun. However, the interpretation of the surrounding environment is constrained not only by the amount of energy received but also by the information content of the radiation. Information is related to entropy rather than energy. The human brain follows Bayesian statistical inference for the interpretation of visual space. The maximization of information occurs in the process of maximizing the entropy. Here, we show that the photopic and scotopic vision absorption peaks in humans are determined not only by the intensity but also by the entropy of radiation. We suggest that through the course of evolution, the human eye has not adapted only to the maximum intensity or to the maximum information but to the optimal wavelength for obtaining information. On Earth, the optimal wavelengths for photopic and scotopic vision are 555 nm and 508 nm, respectively, as inferred experimentally. These optimal wavelengths are determined by the temperature of the star (in this case, the Sun) and by the atmospheric composition.

Perception and Reality: Why a Wholly Empirical Paradigm is Needed to Understand Vision

A central puzzle in vision science is how perceptions that are routinely at odds with physical measurements of real world properties can arise from neural responses that nonetheless lead to effective behaviors. Here we argue that the solution depends on: (1) rejecting the assumption that the goal of vision is to recover, however imperfectly, properties of the world; and (2) replacing it with a paradigm in which perceptions reflect biological utility based on past experience rather than objective features of the environment. Present evidence is consistent with the conclusion that conceiving vision in wholly empirical terms provides a plausible way to understand what we see and why.

Standard deviation of luminance distribution affects lightness and pupillary response

We examined whether the standard deviation (SD) of luminance distribution serves as information of illumination. We measured the lightness of a patch presented in the center of a scrambled-dot pattern while manipulating the SD of the luminance distribution. Results showed that lightness decreased as the SD of the surround stimulus increased. We also measured pupil diameter while viewing a similar stimulus. The pupil diameter decreased as the SD of luminance distribution of the stimuli increased. We confirmed that these results were not obtained because of the increase of the highest luminance in the stimulus. Furthermore, results of field measurements revealed a correlation between the SD of luminance distribution and illuminance in natural scenes. These results indicated that the visual system refers to the SD of the luminance distribution in the visual stimulus to estimate the scene illumination.

Properties of artificial neurons that report lightness based on accumulated experience with luminance

The responses of visual neurons in experimental animals have been extensively characterized. To ask whether these responses are consistent with a wholly empirical concept of visual perception, we optimized simple neural networks that responded according to the cumulative frequency of occurrence of local luminance patterns in retinal images. Based on this estimation of accumulated experience, the neuron responses showed classical center-surround receptive fields, luminance gain control and contrast gain control, the key properties of early level visual neurons determined in animal experiments. These results imply that a major purpose of pre-cortical neuronal circuitry is to contend with the inherently uncertain significance of luminance values in natural stimuli.

Properties of artificial networks evolved to contend with natural spectra

Understanding why spectra that are physically the same appear different in different contexts (color contrast), whereas spectra that are physically different appear similar (color constancy) presents a major challenge in vision research. Here, we show that the responses of biologically inspired neural networks evolved on the basis of accumulated experience with spectral stimuli automatically generate contrast and constancy. The results imply that these phenomena are signatures of a strategy that biological vision uses to circumvent the inverse optics problem as it pertains to light spectra, and that double-opponent neurons in early-level vision evolve to serve this purpose. This strategy provides a way of understanding the peculiar relationship between the objective world and subjective color experience, as well as rationalizing the relevant visual circuitry without invoking feature detection or image representation.

How biological vision succeeds in the physical world

Biological visual systems cannot measure the properties that define the physical world. Nonetheless, visually guided behaviors of humans and other animals are routinely successful. The purpose of this article is to consider how this feat is accomplished. Most concepts of vision propose, explicitly or implicitly, that visual behavior depends on recovering the sources of stimulus features either directly or by a process of statistical inference. Here we argue that, given the inability of the visual system to access the properties of the world, these conceptual frameworks cannot account for the behavioral success of biological vision. The alternative we present is that the visual system links the frequency of occurrence of biologically determined stimuli to useful perceptual and behavioral responses without recovering real-world properties. The evidence for this interpretation of vision is that the frequency of occurrence of stimulus patterns predicts many basic aspects of what we actually see. This strategy provides a different way of conceiving the relationship between objective reality and subjective experience, and offers a way to understand the operating principles of visual circuitry without invoking feature detection, representation, or probabilistic inference.

Multi-scale spatial concatenations of local features in natural scenes and scene classification

How does the visual system encode natural scenes? What are the basic structures of natural scenes? In current models of scene perception, there are two broad feature representations, global and local representations. Both representations are useful and have some successes; however, many observations on human scene perception seem to point to an intermediate-level representation. In this paper, we proposed natural scene structures, i.e., multi-scale spatial concatenations of local features, as an intermediate-level representation of natural scenes. To compile the natural scene structures, we first sampled a large number of multi-scale circular scene patches in a hexagonal configuration. We then performed independent component analysis on the patches and classified the independent components into a set of clusters using the K-means method. Finally, we obtained a set of natural scene structures, each of which is characterized by a set of dominant clusters of independent components. We examined a range of statistics of the natural scene structures, compiled from two widely used datasets of natural scenes, and modeled their spatial arrangements at larger spatial scales using adjacency matrices. We found that the natural scene structures include a full range of concatenations of visual features in natural scenes, and can be used to encode spatial information at various scales. We then selected a set of natural scene structures with high information, and used the occurring frequencies and the eigenvalues of the adjacency matrices to classify scenes in the datasets. We found that the performance of this model is comparable to or better than the state-of-the-art models on the two datasets. These results suggest that the natural scene structures are a useful intermediate-level representation of visual scenes for our understanding of natural scene perception.

Network connections that evolve to circumvent the inverse optics problem

A fundamental problem in vision science is how useful perceptions and behaviors arise in the absence of information about the physical sources of retinal stimuli (the inverse optics problem). Psychophysical studies show that human observers contend with this problem by using the frequency of occurrence of stimulus patterns in cumulative experience to generate percepts. To begin to understand the neural mechanisms underlying this strategy, we examined the connectivity of simple neural networks evolved to respond according to the cumulative rank of stimulus luminance values. Evolved similarities with the connectivity of early level visual neurons suggests that biological visual circuitry uses the same mechanisms as a means of creating useful perceptions and behaviors without information about the real world.

The origins and structure of quantitative concepts

"Number" is the single most influential quantitative dimension in modern human society. It is our preferred dimension for keeping track of almost everything, including distance, weight, time, temperature, and value. How did "number" become psychologically affiliated with all of these different quantitative dimensions? Humans and other animals process a broad range of quantitative information across many psychophysical dimensions and sensory modalities. The fact that adults can rapidly translate one dimension (e.g., loudness) into any other (e.g., handgrip pressure) has been long established by psychophysics research (Stevens, 1975 ). Recent literature has attempted to account for the development of the computational and neural mechanisms that underlie interactions between quantitative dimensions. We review evidence that there are fundamental cognitive and neural relations among different quantitative dimensions (number, size, time, pitch, loudness, and brightness). Then, drawing on theoretical frameworks that explain phenomena from cross-modal perception, we outline some possible conceptualizations for how different quantitative dimensions could come to be related over both ontogenetic and phylogenetic time scales.

Understanding vision in wholly empirical terms

This article considers visual perception, the nature of the information on which perceptions seem to be based, and the implications of a wholly empirical concept of perception and sensory processing for vision science. Evidence from studies of lightness, brightness, color, form, and motion all indicate that, because the visual system cannot access the physical world by means of retinal light patterns as such, what we see cannot and does not represent the actual properties of objects or images. The phenomenology of visual perceptions can be explained, however, in terms of empirical associations that link images whose meanings are inherently undetermined to their behavioral significance. Vision in these terms requires fundamentally different concepts of what we see, why, and how the visual system operates.

Distributions of observed death tolls govern sensitivity to human fatalities

How we react to humanitarian crises, epidemics, and other tragic events involving the loss of human lives depends largely on the extent to which we are moved by the size of their associated death tolls. Many studies have demonstrated that people generally exhibit a diminishing sensitivity to the number of human fatalities and, equivalently, a preference for risky (vs. sure) alternatives in decisions under risk involving human losses. However, the reason for this tendency remains unknown. Here we show that the distributions of event-related death tolls that people observe govern their evaluations of, and risk preferences concerning, human fatalities. In particular, we show that our diminishing sensitivity to human fatalities follows from the fact that these death tolls are approximately power-law distributed. We further show that, by manipulating the distribution of mortality-related events that people observe, we can alter their risk preferences in decisions involving fatalities. Finally, we show that the tendency to be risk-seeking in mortality-related decisions is lower in countries in which high-mortality events are more frequently observed. Our results support a model of magnitude evaluation based on memory sampling and relative judgment. This model departs from the utility-based approaches typically encountered in psychology and economics in that it does not rely on stable, underlying value representations to explain valuation and choice, or on choice behavior to derive value functions. Instead, preferences concerning human fatalities emerge spontaneously from the distributions of sampled events and the relative nature of the evaluation process.

An empirical explanation of the speed-distance effect

Understanding motion perception continues to be the subject of much debate, a central challenge being to account for why the speeds and directions seen accord with neither the physical movements of objects nor their projected movements on the retina. Here we investigate the varied perceptions of speed that occur when stimuli moving across the retina traverse different projected distances (the speed-distance effect). By analyzing a database of moving objects projected onto an image plane we show that this phenomenology can be quantitatively accounted for by the frequency of occurrence of image speeds generated by perspective transformation. These results indicate that speed-distance effects are determined empirically from accumulated past experience with the relationship between image speeds and moving objects.

Multi-scale lines and edges in V1 and beyond: brightness, object categorization and recognition, and consciousness

In this paper we present an improved model for line and edge detection in cortical area V1. This model is based on responses of simple and complex cells, and it is multi-scale with no free parameters. We illustrate the use of the multi-scale line/edge representation in different processes: visual reconstruction or brightness perception, automatic scale selection and object segregation. A two-level object categorization scenario is tested in which pre-categorization is based on coarse scales only and final categorization on coarse plus fine scales. We also present a multi-scale object and face recognition model. Processing schemes are discussed in the framework of a complete cortical architecture. The fact that brightness perception and object recognition may be based on the same symbolic image representation is an indication that the entire (visual) cortex is involved in consciousness.

An empirical explanation of the flash-lag effect

When a flash of light is presented in physical alignment with a moving object, the flash is perceived to lag behind the position of the object. This phenomenon, known as the flash-lag effect, has been of particular interest to vision scientists because of the challenge it presents to understanding how the visual system generates perceptions of objects in motion. Although various explanations have been offered, the significance of this effect remains a matter of debate. Here, we show that: (i) contrary to previous reports based on limited data, the flash-lag effect is an increasing nonlinear function of image speed; and (ii) this function is accurately predicted by the frequency of occurrence of image speeds generated by the perspective transformation of moving objects. These results support the conclusion that perceptions of the relative position of a moving object are determined by accumulated experience with image speeds, in this way allowing for visual behavior in response to real-world sources whose speeds and positions cannot be perceived directly.

Perceptual learning depends on perceptual constancy

Perceptual learning refers to experience-induced improvements in the pick-up of information. Perceptual constancy describes the fact that, despite variable sensory input, perceptual representations typically correspond to stable properties of objects. Here, we show evidence of a strong link between perceptual learning and perceptual constancy: Perceptual learning depends on constancy-based perceptual representations. Perceptual learning may involve changes in early sensory analyzers, but such changes may in general be constrained by categorical distinctions among the high-level perceptual representations to which they contribute. Using established relations of perceptual constancy and sensory inputs, we tested the ability to discover regularities in tasks that dissociated perceptual and sensory invariants. We found that human subjects could learn to classify based on a perceptual invariant that depended on an underlying sensory invariant but could not learn the identical sensory invariant when it did not correlate with a perceptual invariant. These results suggest that constancy-based representations, known to be important for thought and action, also guide learning and plasticity.

What are lightness illusions and why do we see them?

Lightness illusions are fundamental to human perception, and yet why we see them is still the focus of much research. Here we address the question by modelling not human physiology or perception directly as is typically the case but our natural visual world and the need for robust behaviour. Artificial neural networks were trained to predict the reflectance of surfaces in a synthetic ecology consisting of 3-D “dead-leaves” scenes under non-uniform illumination. The networks learned to solve this task accurately and robustly given only ambiguous sense data. In addition—and as a direct consequence of their experience—the networks also made systematic “errors” in their behaviour commensurate with human illusions, which includes brightness contrast and assimilation—although assimilation (specifically White's illusion) only emerged when the virtual ecology included 3-D, as opposed to 2-D scenes. Subtle variations in these illusions, also found in human perception, were observed, such as the asymmetry of brightness contrast. These data suggest that “illusions” arise in humans because (i) natural stimuli are ambiguous, and (ii) this ambiguity is resolved empirically by encoding the statistical relationship between images and scenes in past visual experience. Since resolving stimulus ambiguity is a challenge faced by all visual systems, a corollary of these findings is that human illusions must be experienced by all visual animals regardless of their particular neural machinery. The data also provide a more formal definition of illusion: the condition in which the true source of a stimulus differs from what is its most likely (and thus perceived) source. As such, illusions are not fundamentally different from non-illusory percepts, all being direct manifestations of the statistical relationship between images and scenes.

Sometimes the best way to understand how the visual brain works is to understand why it sometimes does not. Thus, visual illusions have been central to the science and philosophy of human consciousness for decades. Here we explain the root cause of brightness illusions, not by modelling human perception or its assumed physiological substrate (as is more typically done), but by modelling the basic challenge that all visual animals must resolve if they are to survive: the inherent ambiguity of sensory data. We do this by training synthetic neural networks to recognise surfaces under different lights in scenes with naturalistic structure. The result is that the networks not only solve this task robustly (i.e., they exhibit “lightness constancy”), they also—as a consequence—exhibit the same illusions of lightness that humans also see. In short, these synthetic systems not only get it right like we do, but also get it wrong like we do, too. This emergent coincidence strongly provides causal evidence that illusions (and by extension all percepts) represent the probable source of images in past visual experience, which has fundamental consequences for explaining how and why we see what we do. The study also suggests the first formal definition of what an illusion is: The condition in which the actual source of a stimulus differs from its most likely source.

Transient cortical excitation at the onset of visual fixation

Primates actively examine the visual world by rapidly shifting gaze (fixation) over the elements in a scene. Despite this fact, we typically study vision by presenting stimuli with gaze held constant. To better understand the dynamics of natural vision, we examined how the onset of visual fixation affects ongoing neuronal activity in the absence of visual stimulation. We used multiunit activity and current source density measurements to index neuronal firing patterns and underlying synaptic processes in macaque V1. Initial averaging of neural activity synchronized to the onset of fixation suggested that a brief period of cortical excitation follows each fixation. Subsequent single-trial analyses revealed that 1) neuronal oscillation phase transits from random to a highly organized state just after the fixation onset, 2) this phase concentration is accompanied by increased spectral power in several frequency bands, and 3) visual response amplitude is enhanced at the specific oscillatory phase associated with fixation. We hypothesize that nonvisual inputs are used by the brain to increase cortical excitability at fixation onset, thus "priming" the system for new visual inputs generated at fixation. Despite remaining mechanistic questions, it appears that analysis of fixation-related responses may be useful in studying natural vision.

Evolution of visually guided behavior in artificial agents

Recent work on brightness, color, and form has suggested that human visual percepts represent the probable sources of retinal images rather than stimulus features as such. Here we investigate the plausibility of this empirical concept of vision by allowing autonomous agents to evolve in virtual environments based solely on the relative success of their behavior. The responses of evolved agents to visual stimuli indicate that fitness improves as the neural network control systems gradually incorporate the statistical relationship between projected images and behavior appropriate to the sources of the inherently ambiguous images. These results: (1) demonstrate the merits of a wholly empirical strategy of animal vision as a means of contending with the inverse optics problem; (2) argue that the information incorporated into biological visual processing circuitry is the relationship between images and their probable sources; and (3) suggest why human percepts do not map neatly onto physical reality.

Simultaneous lightness contrast on plain and articulated surrounds

Simultaneous lightness contrast is stronger when the dark and light backgrounds of the classic display (where one of the targets is an increment and the other is a decrement) are replaced by articulated fields of equivalent average luminances. Although routinely attributed to articulation per se, this effect may simply result from the increase in highest luminance in the light articulated, vs plain, background; by locally darkening the decremental target, such an increase would amplify the difference between the targets. We disentangled the effects of highest luminance and articulation by measuring, separately, the magnitude of lightness contrast on dark and light plain and articulated backgrounds. We found that highest luminance and articulation contribute separately to the final illusion.

Comparison of Bayesian and empirical ranking approaches to visual perception

Much current vision research is predicated on the idea--and a rapidly growing body of evidence--that visual percepts are generated according to the empirical significance of light stimuli rather than their physical characteristics. As a result, an increasing number of investigators have asked how visual perception can be rationalized in these terms. Here, we compare two different theoretical frameworks for predicting what observers actually see in response to visual stimuli: Bayesian decision theory and empirical ranking theory. Deciding which of these approaches has greater merit is likely to determine how the statistical operations that apparently underlie visual perception are eventually understood.

Spectral statistics in natural scenes predict hue, saturation, and brightness

The perceptual color qualities of hue, saturation, and brightness do not correspond in any simple way to the physical characteristics of retinal stimuli, a fact that poses a major obstacle for any explanation of color vision. Here we test the hypothesis that these basic color attributes are determined by the statistical covariations in the spectral stimuli that humans have always experienced in typical visual environments. Using a database of 1,600 natural images, we analyzed the joint probability distributions of the physical variables most relevant to each of these perceptual qualities. The cumulative density functions derived from these distributions predict the major colorimetric functions that have been reported in psychophysical experiments over the last century.

Inhomogeneous surrounds, conflicting frameworks, and the double-anchoring theory of lightness

The empirical question of whether or not the lightness of a region is accounted for purely by the average luminance of its surround has a complex answer that depends on whether such a region is an increment, a decrement, or intermediate relative to the luminances of the contiguous surfaces. It is shown here that a new model of lightness, based on anchoring principles, predicts and clarifies such intricacies. In this model, the luminance of the target region determines its lightness in two ways: indirectly, by causing it to group with parts of its surround and thus defining the nested frameworks to which it belongs; and directly, by anchoring it to the highest luminance and to the average surround luminance in each of these frameworks. Inter- and intraindividual differences in lightness assessment are shown to emerge under grouping conditions that create unstable, conflicting frameworks.