Do cortical neurons process luminance or contrast to encode surface properties?

Isolation of brightness induction effects on target patches from adjacent surrounds and remote backgrounds

The brightness (perceived intensity) of a region of visual space depends on its luminance and on the luminance of nearby regions. This phenomenon is called brightness induction and includes both brightness contrast and assimilation. Historically, and on a purely descriptive level, brightness contrast refers to a directional shift in target brightness away from the brightness of an adjacent region while assimilation refers to a brightness shift toward that of an adjacent region. In order to understand mechanisms, it is important to differentiate the descriptive terms contrast and assimilation from the optical and/or neural processes, often similarly named, which cause the effects. Experiment 1 isolated the effect on target patch (64 cd/m2) matching luminance (brightness) of six surround-ring widths (0.1°–24.5°) varied over 11 surround-ring luminances (32–96 cd/m2). Using the same observers, Experiment 2 examined the effect of the identical surround-ring parameters on target patch matching luminance in the presence of a dark (0.0 cd/m2) and a bright (96 cd/m2) remote background. By differencing the results of Experiment 1 (the isolated effect of the surround-ring) from those of Experiment 2 (the combined effect of the surround-ring with the dark and bright remote background) we further isolated the effect of the remote background. The results reveal that surround-rings and remote backgrounds produce brightness contrast effects in the target patch that are of the same or opposite polarity depending on the luminance polarity of these regions relative to target patch luminance. The strength of brightness contrast from the surround-ring varied with surround-ring luminance and width. Brightness contrast (darkening) in the target from the bright remote background was relatively constant in magnitude across all surround-ring luminances and increased in magnitude with decreasing surround-ring width. Brightness contrast (brightening) from the isolated dark remote background also increased in magnitude with decreasing surround-ring width: however, despite some regional flattening of the functions due to the fixed luminance of the dark remote background, induction magnitude was much reduced in the presence of a surround-ring of greater luminance than the target patch indicating a non-linear interaction between the dark remote background and surround-ring luminance.

Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey

Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance.

How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.

Neuronal population mechanisms of lightness perception

The way that humans and animals perceive the lightness of an object depends on its physical luminance as well as its surrounding context. While neuronal responses throughout the visual pathway are modulated by context, the relationship between neuronal responses and lightness perception is poorly understood. We searched for a neuronal mechanism of lightness by recording responses of neuronal populations in monkey primary visual cortex (V1) and area V4 to stimuli that produce a lightness illusion in humans, in which the lightness of a disk depends on the context in which it is embedded. We found that the way individual units encode the luminance (or equivalently for our stimuli, contrast) of the disk and its context is extremely heterogeneous. This motivated us to ask whether the population representation in either V1 or V4 satisfies three criteria: 1) disk luminance is represented with high fidelity, 2) the context surrounding the disk is also represented, and 3) the representations of disk luminance and context interact to create a representation of lightness that depends on these factors in a manner consistent with human psychophysical judgments of disk lightness. We found that populations of units in both V1 and V4 fulfill the first two criteria but that we cannot conclude that the two types of information in either area interact in a manner that clearly predicts human psychophysical measurements: the interpretation of our population measurements depends on how subsequent areas read out lightness from the population responses. NEW & NOTEWORTHY A core question in visual neuroscience is how the brain extracts stable representations of object properties from the retinal image. We searched for a neuronal mechanism of lightness perception by determining whether the responses of neuronal populations in primary visual cortex and area V4 could account for a lightness illusion measured using human psychophysics. Our results suggest that comparing psychophysics with population recordings will yield insight into neuronal mechanisms underlying a variety of perceptual phenomena.

Contrast response functions in the visual wulst of the alert burrowing owl: a single-unit study

The neuronal representation of luminance contrast has not been thoroughly studied in birds. Here we present a detailed quantitative analysis of the contrast response of 120 individual neurons recorded from the visual wulst of awake burrowing owls (Athene cunicularia). Stimuli were sine-wave gratings presented within the cell classical receptive field and optimized in terms of eye preference, direction of drift, and spatiotemporal frequency. As contrast intensity was increased from zero to near 100%, most cells exhibited a monotonic response profile with a compressive, at times saturating, nonlinearity at higher contrasts. However, contrast response functions were found to have a highly variable shape across cells. With the view to capture a systematic trend in the data, we assessed the performance of four plausible models (linear, power, logarithmic, and hyperbolic ratio) using classical goodness-of-fit measures and more rigorous statistical tools for multimodel inferences based on the Akaike information criterion. From this analysis, we conclude that a high degree of model uncertainty is present in our data, meaning that no single descriptor is able on its own to capture the heterogeneous nature of single-unit contrast responses in the wulst. We further show that the generalizability of the hyperbolic ratio model established, for example, in the primary visual cortex of cats and monkeys is not tenable in the owl wulst mainly because most neurons in this area have a much wider dynamic range that starts at low contrast. The challenge for future research will be to understand the functional implications of these findings.

A unified account of perceptual layering and surface appearance in terms of gamut relativity

When we look at the world--or a graphical depiction of the world--we perceive surface materials (e.g. a ceramic black and white checkerboard) independently of variations in illumination (e.g. shading or shadow) and atmospheric media (e.g. clouds or smoke). Such percepts are partly based on the way physical surfaces and media reflect and transmit light and partly on the way the human visual system processes the complex patterns of light reaching the eye. One way to understand how these percepts arise is to assume that the visual system parses patterns of light into layered perceptual representations of surfaces, illumination and atmospheric media, one seen through another. Despite a great deal of previous experimental and modelling work on layered representation, however, a unified computational model of key perceptual demonstrations is still lacking. Here we present the first general computational model of perceptual layering and surface appearance--based on a boarder theoretical framework called gamut relativity--that is consistent with these demonstrations. The model (a) qualitatively explains striking effects of perceptual transparency, figure-ground separation and lightness, (b) quantitatively accounts for the role of stimulus- and task-driven constraints on perceptual matching performance, and (c) unifies two prominent theoretical frameworks for understanding surface appearance. The model thereby provides novel insights into the remarkable capacity of the human visual system to represent and identify surface materials, illumination and atmospheric media, which can be exploited in computer graphics applications.

A cortical edge-integration model of object-based lightness computation that explains effects of spatial context and individual differences

Previous work has demonstrated that perceived surface reflectance (lightness) can be modeled in simple contexts in a quantitatively exact way by assuming that the visual system first extracts information about local, directed steps in log luminance, then spatially integrates these steps along paths through the image to compute lightness (Rudd and Zemach, 2004, 2005, 2007). This method of computing lightness is called edge integration. Recent evidence (Rudd, 2013) suggests that human vision employs a default strategy to integrate luminance steps only along paths from a common background region to the targets whose lightness is computed. This implies a role for gestalt grouping in edge-based lightness computation. Rudd (2010) further showed the perceptual weights applied to edges in lightness computation can be influenced by the observer's interpretation of luminance steps as resulting from either spatial variation in surface reflectance or illumination. This implies a role for top-down factors in any edge-based model of lightness (Rudd and Zemach, 2005). Here, I show how the separate influences of grouping and attention on lightness can be modeled in tandem by a cortical mechanism that first employs top-down signals to spatially select regions of interest for lightness computation. An object-based network computation, involving neurons that code for border-ownership, then automatically sets the neural gains applied to edge signals surviving the earlier spatial selection stage. Only the borders that survive both processing stages are spatially integrated to compute lightness. The model assumptions are consistent with those of the cortical lightness model presented earlier by Rudd (2010, 2013), and with neurophysiological data indicating extraction of local edge information in V1, network computations to establish figure-ground relations and border ownership in V2, and edge integration to encode lightness and darkness signals in V4.

Human cortical areas involved in perception of surface glossiness

Glossiness is the visual appearance of an object's surface as defined by its surface reflectance properties. Despite its ecological importance, little is known about the neural substrates underlying its perception. In this study, we performed the first human neuroimaging experiments that directly investigated where the processing of glossiness resides in the visual cortex. First, we investigated the cortical regions that were more activated by observing high glossiness compared with low glossiness, where the effects of simple luminance and luminance contrast were dissociated by controlling the illumination conditions (Experiment 1). As cortical regions that may be related to the processing of glossiness, V2, V3, hV4, VO-1, VO-2, collateral sulcus (CoS), LO-1, and V3A/B were identified, which also showed significant correlation with the perceived level of glossiness. This result is consistent with the recent monkey studies that identified selective neural response to glossiness in the ventral visual pathway, except for V3A/B in the dorsal visual pathway, whose involvement in the processing of glossiness could be specific to the human visual system. Second, we investigated the cortical regions that were modulated by selective attention to glossiness (Experiment 2). The visual areas that showed higher activation to attention to glossiness than that to either form or orientation were identified as right hV4, right VO-2, and right V3A/B, which were commonly identified in Experiment 1. The results indicate that these commonly identified visual areas in the human visual cortex may play important roles in glossiness perception.

Paradoxical effect of spatially homogenous transparent fields on simultaneous contrast illusions

In simultaneous brightness contrast (SBC) demonstrations, identical mid-luminance disks appear different from each other when one is placed on a black background while the other is placed on a white background. The strength of SBC effects can be enhanced by placing a semi-transparent layer on top of the display (Meyer's effect). Here, we try to separate the causes of Meyer's effect by placing a spatially homogenous transparent layer over a standard SBC display, and systematically varying the transmission level (alpha=0, clear; alpha=1, opaque) and color (black, gray, white) of the semi-transparent layer. Spatially homogenous transparent layers, which lack spatial cues, cannot be unambiguously interpreted as transparent fields. We measure SBC strength with both matching and ranking procedures. Paradoxically, with black layers, increasing alpha level weakens SBC when measured with a ranking procedure (no Meyer's effect) and strengthens SBC when measured with a matching procedure (Meyer's effect). With white and gray layers, neither procedure produces Meyer's effect. We account for the differences between white and black layers by positing that the visual system separates luminance from contrast. The results suggest that observers attend to different information in the matching and ranking procedures.

Adaptive stimulus optimization for sensory systems neuroscience

In this paper, we review several lines of recent work aimed at developing practical methods for adaptive on-line stimulus generation for sensory neurophysiology. We consider various experimental paradigms where on-line stimulus optimization is utilized, including the classical optimal stimulus paradigm where the goal of experiments is to identify a stimulus which maximizes neural responses, the iso-response paradigm which finds sets of stimuli giving rise to constant responses, and the system identification paradigm where the experimental goal is to estimate and possibly compare sensory processing models. We discuss various theoretical and practical aspects of adaptive firing rate optimization, including optimization with stimulus space constraints, firing rate adaptation, and possible network constraints on the optimal stimulus. We consider the problem of system identification, and show how accurate estimation of non-linear models can be highly dependent on the stimulus set used to probe the network. We suggest that optimizing stimuli for accurate model estimation may make it possible to successfully identify non-linear models which are otherwise intractable, and summarize several recent studies of this type. Finally, we present a two-stage stimulus design procedure which combines the dual goals of model estimation and model comparison and may be especially useful for system identification experiments where the appropriate model is unknown beforehand. We propose that fast, on-line stimulus optimization enabled by increasing computer power can make it practical to move sensory neuroscience away from a descriptive paradigm and toward a new paradigm of real-time model estimation and comparison.

Population response to natural images in the primary visual cortex encodes local stimulus attributes and perceptual processing

The primary visual cortex (V1) is extensively studied with a large repertoire of stimuli, yet little is known about its encoding of natural images. Using voltage-sensitive dye imaging in behaving monkeys, we measured neural population response evoked in V1 by natural images presented during a face/scramble discrimination task. The population response showed two distinct phases of activity: an early phase that was spread over most of the imaged area, and a late phase that was spatially confined. To study the detailed relation between the stimulus and the population response, we used a simple encoding model to compute a continuous map of the expected neural response based on local attributes of the stimulus (luminance and contrast), followed by an analytical retinotopic transformation. Then, we computed the spatial correlation between the maps of the expected and observed response. We found that the early response was highly correlated with the local luminance of the stimulus and was sufficient to effectively discriminate between stimuli at the single trial level. The late response, on the other hand, showed a much lower correlation to the local luminance, was confined to central parts of the face images, and was highly correlated with the animal's perceptual report. Our study reveals a continuous spatial encoding of low- and high-level features of natural images in V1. The low level is directly linked to the stimulus basic local attributes and the high level is correlated with the perceptual outcome of the stimulus processing.

Effects of luminance on dynamic random-dot correlogram evoked visual potentials

Although dynamic random-dot correlogram evoked visual potentials (DRDC-VEPs) are a three-decade-old method to detect the cortical binocularity in humans and animals, our knowledge of the influence of fundamental stimulus parameters and the underlying cerebral processing mechanisms has remained limited. The purpose of this study was to evaluate the effect of luminance on DRDC-VEPs in adults. The variability and detectability of DRDC-VEPs were investigated under different stimulus luminance conditions with neutral density filters. Our results have demonstrated that DRDC-VEPs can be evoked in a wide luminance range, and the response amplitude was practically independent of luminance between 4.75 cd m(-2) and 0.015 cd m(-2), while DRDC-VEP latencies showed a strong linear correlation with log luminance. There is, however, a limit (0.06 cd m(-2)) below which DRDC-VEPs are not reliably recordable. Luminance reduction-induced delays in DRDC-VEP latencies cannot be explained simply by retinal mechanisms, since their regression slope does not follow the course of electroretinogram and cortical evoked potential latencies. Luminance independence of DRDC-VEP amplitude suggests that binocular correlation-processing cortical neurons receive input predominantly from the magnocellular visual pathway.

Active data collection for efficient estimation and comparison of nonlinear neural models

The stimulus-response relationship of many sensory neurons is nonlinear, but fully quantifying this relationship by a complex nonlinear model may require too much data to be experimentally tractable. Here we present a theoretical study of a general two-stage computational method that may help to significantly reduce the number of stimuli needed to obtain an accurate mathematical description of nonlinear neural responses. Our method of active data collection first adaptively generates stimuli that are optimal for estimating the parameters of competing nonlinear models and then uses these estimates to generate stimuli online that are optimal for discriminating these models. We applied our method to simple hierarchical circuit models, including nonlinear networks built on the spatiotemporal or spectral-temporal receptive fields, and confirmed that collecting data using our two-stage adaptive algorithm was far more effective for estimating and comparing competing nonlinear sensory processing models than standard nonadaptive methods using random stimuli.

V1 response timing and surface filling-in

There is ample evidence from demonstrations such as color induction and stabilized images that information from surface boundaries plays a special role in determining the perception of surface interiors. Surface interiors appear to "fill-in." Psychophysical experiments also show that surface perception involves a slow scale-dependent process distinct from mechanisms involved in contour perception. The present experiments aimed to test the hypothesis that surface perception is associated with relatively slow scale-dependent neural filling-in. We found that responses in macaque primary visual cortex (V1) are slower to surface interiors than responses to optimal bar stimuli. Moreover, we found that the response to a surface interior is delayed relative to the response to the surface's border and the extent of the delay is proportional to the distance between a receptive field and the border. These findings are consistent with some forms of neural filling-in and suggest that V1 may provide the neural substrate for perceptual filling-in.

Brightness and darkness as perceptual dimensions

A common-sense assumption concerning visual perception states that brightness and darkness cannot coexist at a given spatial location. One corollary of this assumption is that achromatic colors, or perceived grey shades, are contained in a one-dimensional (1-D) space varying from bright to dark. The results of many previous psychophysical studies suggest, by contrast, that achromatic colors are represented as points in a color space composed of two or more perceptual dimensions. The nature of these perceptual dimensions, however, presently remains unclear. Here we provide direct evidence that brightness and darkness form the dimensions of a two-dimensional (2-D) achromatic color space. This color space may play a role in the representation of object surfaces viewed against natural backgrounds, which simultaneously induce both brightness and darkness signals. Our 2-D model generalizes to the chromatic dimensions of color perception, indicating that redness and greenness (blueness and yellowness) also form perceptual dimensions. Collectively, these findings suggest that human color space is composed of six dimensions, rather than the conventional three.

Vision scientists have long adhered to the classic opponent-coding theory of vision, which states that bright–dark, red–green, and blue–yellow form mutually exclusive color pairs. According to this theory, it is not possible to see both brightness and darkness at a single spatial location, or an extended set of locations, such as a uniform surface. One corollary of this statement is that all perceivable grey shades vary along a continuum from bright to dark. At first glance, the notion that brightness and darkness cannot coexist on a single surface accords with our common-sense notion that a given grey shade cannot be simultaneously both brighter and darker than any other grey shade. The results presented here suggest that this common-sense notion is not supported by experimental data. Our results imply that a given grey shade can indeed be simultaneously brighter and darker than another grey shade. This seemingly paradoxical conclusion arises naturally if one assumes that brightness and darkness constitute the dimensions of a two-dimensional perceptual space in which points represent grey shades. Our results may encourage scientists working in related fields to question the assumption that perceptual variables, rather than sensory variables, are encoded in opponent pairs.

No functional magnetic resonance imaging evidence for brightness and color filling-in in early human visual cortex

The brightness and color of a surface depends on its contrast with nearby surfaces. For example, a gray surface can appear very light when surrounded by a black surface or dark when surrounded by a white surface. Some theories suggest that perceived surface brightness and color is represented explicitly by neural signals in cortical visual field maps; these neural signals are not initiated by the stimulus itself but rather by the contrast signals at the borders. Here, we use functional magnetic resonance imaging (fMRI) to search for such neural "filling-in" signals. Although we find the usual strong relationship between local contrast and fMRI response, when perceived brightness or color changes are induced by modulating a surrounding field, rather than the surface itself, we find there is no corresponding local modulation in primary visual cortex or other nearby retinotopic maps. Moreover, when we model the obtained fMRI responses, we find strong evidence for contributions of both local and long-range edge responses. We argue that such extended edge responses may be caused by neurons previously identified in neurophysiological studies as being brightness responsive, a characterization that may therefore need to be revised. We conclude that the visual field maps of human V1 and V2 do not contain filled-in, topographical representations of surface brightness and color.