V1 response timing and surface filling-in

Perceptual filling-in dispels the veridicality problem of conscious perception research

Conscious perceptual experiences are expected to correlate with content-specific brain activity. A veridicality problem arises when attempting to disentangle unconscious and conscious brain processes if conscious perceptual contents accurately match the physical nature of the stimulus. We argue that perceptual filling-in, a phenomenon whereby visual information inaccurately spreads across visual space, is a promising approach to circumvent the veridicality problem. Filling-in generates non-veridical although unambiguous percepts dissociated from stimulus input. In particular, the radial uniformity illusion induces a filling-in experience between a central disk and the surrounding periphery. We discuss how this illusion facilitates both the detection of neurophysiological responses and subjective phenomenological monitoring. We report behavioral effects from a large-sample (n = 200) psychophysics study and examine key stimulus parameters that drive the conscious filling-in experience. We propose that these data underpin future hypothesis-driven studies of filling-in to further delineate the neural mechanisms of conscious perception.

Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey

Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance.

How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.

Different roles of subcortical inputs in V1 responses to luminance and contrast

Cells in the primary visual cortex (V1) generally respond weakly to large uniform luminance stimuli. Only a subset of V1 cells is thought to encode uniform luminance information. In natural scenes, local luminance is an important feature for defining an object that varies and coexists with local spatial contrast. However, the strategies used by V1 cells to encode local mean luminance for spatial contrast stimuli remain largely unclear. Here, using extracellular recordings in anesthetized cats, we investigated the responses of V1 cells by comparing with those of retinal ganglion (RG) cells and lateral geniculate nucleus (LGN) cells to simultaneous and rapid changes in luminance and spatial contrast. Almost all V1 cells exhibited a strong monotonic increasing luminance tuning when they were exposed to high spatial contrast. Thus, V1 cells encode the luminance carried by spatial contrast stimuli with the monotonically increasing response function. Moreover, high contrast decreased luminance tuning of OFF cells but increased that of in ON cells in RG and LGN. The luminance and contrast tunings of LGN ON cells were highly separable as V1 cells, whereas those of LGN OFF cells were lowly separable. These asymmetrical effects of spatial contrast on ON/OFF channels might underlie the robust ability of V1 cells to perform luminance tuning when exposed to spatial contrast stimuli.

Dynamic decorrelation as a unifying principle for explaining a broad range of brightness phenomena

The visual system is highly sensitive to spatial context for encoding luminance patterns. Context sensitivity inspired the proposal of many neural mechanisms for explaining the perception of luminance (brightness). Here we propose a novel computational model for estimating the brightness of many visual illusions. We hypothesize that many aspects of brightness can be explained by a dynamic filtering process that reduces the redundancy in edge representations on the one hand, while non-redundant activity is enhanced on the other. The dynamic filter is learned for each input image and implements context sensitivity. Dynamic filtering is applied to the responses of (model) complex cells in order to build a gain control map. The gain control map then acts on simple cell responses before they are used to create a brightness map via activity propagation. Our approach is successful in predicting many challenging visual illusions, including contrast effects, assimilation, and reverse contrast with the same set of model parameters.

Feedback contribution to surface motion perception in the human early visual cortex

Human visual surface perception has neural correlates in early visual cortex, but the role of feedback during surface segmentation in human early visual cortex remains unknown. Feedback projections preferentially enter superficial and deep anatomical layers, which provides a hypothesis for the cortical depth distribution of fMRI activity related to feedback. Using ultra-high field fMRI, we report a depth distribution of activation in line with feedback during the (illusory) perception of surface motion. Our results fit with a signal re-entering in superficial depths of V1, followed by a feedforward sweep of the re-entered information through V2 and V3. The magnitude and sign of the BOLD response strongly depended on the presence of texture in the background, and was additionally modulated by the presence of illusory motion perception compatible with feedback. In summary, the present study demonstrates the potential of depth-resolved fMRI in tackling biomechanical questions on perception.

A Compound Computational Model for Filling-In Processes Triggered by Edges: Watercolor Illusions

The goal of our research was to develop a compound computational model with the ability to predict different variations of the “watercolor effects” and additional filling-in effects that are triggered by edges. The model is based on a filling-in mechanism solved by a Poisson equation, which considers the different gradients as “heat sources” after the gradients modification. The biased (modified) contours (edges) are ranked and determined according to their dominancy across the different chromatic and achromatic channels. The color and intensity of the perceived surface are calculated through a diffusive filling-in process of color triggered by the enhanced and biased edges of stimulus formed as a result of oriented double-opponent receptive fields. The model can successfully predict both the assimilative and non-assimilative watercolor effects, as well as a number of “conflicting” visual effects. Furthermore, the model can also predict the classic Craik–O'Brien–Cornsweet (COC) effect. In summary, our proposed computational model is able to predict most of the “conflicting” filling-in effects that derive from edges that have been recently described in the literature, and thus supports the theory that a shared visual mechanism is responsible for the vast variety of the “conflicting” filling-in effects that derive from edges.

Neural Coding for Shape and Texture in Macaque Area V4

The distinct visual sensations of shape and texture have been studied separately in cortex; therefore, it remains unknown whether separate neuronal populations encode each of these properties or one population carries a joint encoding. We directly compared shape and texture selectivity of individual V4 neurons in awake macaques (1 male, 1 female) and found that V4 neurons lie along a continuum from strong tuning for boundary curvature of shapes to strong tuning for perceptual dimensions of texture. Among neurons tuned to both attributes, tuning for shape and texture were largely separable, with the latter delayed by ∼30 ms. We also found that shape stimuli typically evoked stronger, more selective responses than did texture patches, regardless of whether the latter were contained within or extended beyond the receptive field. These results suggest that there are separate specializations in mid-level cortical processing for visual attributes of shape and texture.SIGNIFICANCE STATEMENT Object recognition depends on our ability to see both the shape of the boundaries of objects and properties of their surfaces. However, neuroscientists have never before examined how shape and texture are linked together in mid-level visual cortex. In this study, we used systematically designed sets of simple shapes and texture patches to probe the responses of individual neurons in the primate visual cortex. Our results provide the first evidence that some cortical neurons specialize in processing shape whereas others specialize in processing textures. Most neurons lie between the ends of this continuum, and in these neurons we find that shape and texture encoding are largely independent.

Neuronal population mechanisms of lightness perception

The way that humans and animals perceive the lightness of an object depends on its physical luminance as well as its surrounding context. While neuronal responses throughout the visual pathway are modulated by context, the relationship between neuronal responses and lightness perception is poorly understood. We searched for a neuronal mechanism of lightness by recording responses of neuronal populations in monkey primary visual cortex (V1) and area V4 to stimuli that produce a lightness illusion in humans, in which the lightness of a disk depends on the context in which it is embedded. We found that the way individual units encode the luminance (or equivalently for our stimuli, contrast) of the disk and its context is extremely heterogeneous. This motivated us to ask whether the population representation in either V1 or V4 satisfies three criteria: 1) disk luminance is represented with high fidelity, 2) the context surrounding the disk is also represented, and 3) the representations of disk luminance and context interact to create a representation of lightness that depends on these factors in a manner consistent with human psychophysical judgments of disk lightness. We found that populations of units in both V1 and V4 fulfill the first two criteria but that we cannot conclude that the two types of information in either area interact in a manner that clearly predicts human psychophysical measurements: the interpretation of our population measurements depends on how subsequent areas read out lightness from the population responses. NEW & NOTEWORTHY A core question in visual neuroscience is how the brain extracts stable representations of object properties from the retinal image. We searched for a neuronal mechanism of lightness perception by determining whether the responses of neuronal populations in primary visual cortex and area V4 could account for a lightness illusion measured using human psychophysics. Our results suggest that comparing psychophysics with population recordings will yield insight into neuronal mechanisms underlying a variety of perceptual phenomena.

Neural representation of form-contingent color filling-in in the early visual cortex

Perceptual filling-in exemplifies the constructive nature of visual processing. Color, a prominent surface property of visual objects, can appear to spread to neighboring areas that lack any color. We investigated cortical responses to a color filling-in illusion that effectively dissociates perceived color from the retinal input (van Lier, Vergeer, & Anstis, 2009). Observers adapted to a star-shaped stimulus with alternating red- and cyan-colored points to elicit a complementary afterimage. By presenting an achromatic outline that enclosed one of the two afterimage colors, perceptual filling-in of that color was induced in the unadapted central region. Visual cortical activity was monitored with fMRI, and analyzed using multivariate pattern analysis. Activity patterns in early visual areas (V1–V4) reliably distinguished between the two color-induced filled-in conditions, but only higher extrastriate visual areas showed the predicted correspondence with color perception. Activity patterns allowed for reliable generalization between filled-in colors and physical presentations of perceptually matched colors in areas V3 and V4, but not in earlier visual areas. These findings suggest that the perception of filled-in surface color likely requires more extensive processing by extrastriate visual areas, in order for the neural representation of surface color to become aligned with perceptually matched real colors.

A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs

Learning from a few examples and generalizing to markedly different situations are capabilities of human visual intelligence that are yet to be matched by leading machine learning models. By drawing inspiration from systems neuroscience, we introduce a probabilistic generative model for vision in which message-passing-based inference handles recognition, segmentation, and reasoning in a unified way. The model demonstrates excellent generalization and occlusion-reasoning capabilities and outperforms deep neural networks on a challenging scene text recognition benchmark while being 300-fold more data efficient. In addition, the model fundamentally breaks the defense of modern text-based CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart) by generatively segmenting characters without CAPTCHA-specific heuristics. Our model emphasizes aspects such as data efficiency and compositionality that may be important in the path toward general artificial intelligence.

Evidence of Stereoscopic Surface Disambiguation in the Responses of V1 Neurons

For the important task of binocular depth perception from complex natural-image stimuli, the neurophysiological basis for disambiguating multiple matches between the eyes across similar features has remained a long-standing problem. Recurrent interactions among binocular disparity-tuned neurons in the primary visual cortex (V1) could play a role in stereoscopic computations by altering responses to favor the most likely depth interpretation for a given image pair. Psychophysical research has shown that binocular disparity stimuli displayed in 1 region of the visual field can be extrapolated into neighboring regions that contain ambiguous depth information. We tested whether neurons in macaque V1 interact in a similar manner and found that unambiguous binocular disparity stimuli displayed in the surrounding visual fields of disparity-selective V1 neurons indeed modified their responses when either bistable stereoscopic or uniform featureless stimuli were presented within their receptive field centers. The delayed timing of the response behavior compared with the timing of classical surround suppression and multiple control experiments suggests that these modulations are carried out by slower disparity-specific recurrent connections among V1 neurons. These results provide explicit evidence that the spatial interactions that are predicted by cooperative algorithms play an important role in solving the stereo correspondence problem.

Neurons in primary visual cortex represent distribution of luminance

To efficiently detect a wide range of light-intensity changes, visual neurons must adapt to ambient luminance. However, how neurons in the primary visual cortex (V1) code the distribution of luminance remains unknown. We designed stimuli that represent rapid changes in luminance under different luminance distributions and investigated V1 neuron responses to these novel stimuli. We demonstrate that V1 neurons represent luminance changes by dynamically adjusting their responses when the luminance distribution changes. Many cells (35%) detected luminance changes by responding to dark stimuli when the distribution was dominated by bright stimuli, bright stimuli when dominated by dark stimuli, and both dark and bright stimuli when dominated by intermediate luminance stimuli; 13% of cells signaled the mean luminance that was varied with different distributions; the remaining 52% of cells gradually shifted the responses that were most sensitive to luminance changes when the luminance distribution varied. The remarkable response changes of the former two cell groups suggest their crucial roles in detecting luminance changes. These response characteristics demonstrate that V1 neurons are not only sensitive to luminance change, but also luminance distribution change. They encode luminance changes according to the luminance distribution. Mean cells represent the prevailing luminance and reversal cells represent the salient stimuli in the environment.

Physical connections between different SSVEP neural networks

This work investigates the mechanism of the Steady-State Visual Evoked Potential (SSVEP). One theory suggests that different SSVEP neural networks exist whose strongest response are located in different frequency bands. This theory is based on the fact that there are similar SSVEP frequency-amplitude response curves in these bands. Previous studies that employed simultaneous stimuli of different frequencies illustrated that the distribution of these networks were similar, but did not discuss the physical connection between them. By comparing the SSVEP power and distribution under a single-eye stimulus and a simultaneous, dual-eye stimulus, this work demonstrates that the distributions of different SSVEP neural networks are similar to each other and that there should be physical overlapping between them. According to the band-pass filter theory in a signal transferring channel, which we propose in this work for the first time, there are different amounts of neurons that are involved under repetitive stimuli of different frequencies and that the response intensity of each neuron is similar to each other so that the total response (i.e., the SSVEP) that is observed from the scalp is different.

V1 neurons respond to luminance changes faster than contrast changes

Luminance and contrast are two major attributes of objects in the visual scene. Luminance and contrast information received by visual neurons are often updated simultaneously. We examined the temporal response properties of neurons in the primary visual cortex (V1) to stimuli whose luminance and contrast were simultaneously changed by 50 Hz. We found that response tuning to luminance changes precedes tuning to contrast changes in V1. For most V1 neurons, the onset time of response tuning to luminance changes was shorter than that to contrast changes. Most neurons carried luminance information in the early response stage, while all neurons carried both contrast and luminance information in the late response stage. The early luminance response suggests that cortical processing for luminance is not as slow as previously thought.

Representation of Color Surfaces in V1: Edge Enhancement and Unfilled Holes

The neuronal mechanism underlying the representation of color surfaces in primary visual cortex (V1) is not well understood. We tested on color surfaces the previously proposed hypothesis that visual perception of uniform surfaces is mediated by an isomorphic, filled-in representation in V1. We used voltage-sensitive-dye imaging in fixating macaque monkeys to measure V1 population responses to spatially uniform chromatic (red, green, or blue) and achromatic (black or white) squares of different sizes (0.5°-8°) presented for 300 ms. Responses to both color and luminance squares early after stimulus onset were similarly edge-enhanced: for squares 1° and larger, regions corresponding to edges were activated much more than those corresponding to the center. At later times after stimulus onset, responses to achromatic squares' centers increased, partially "filling-in" the V1 representation of the center. The rising phase of the center response was slower for larger squares. Surprisingly, the responses to color squares behaved differently. For color squares of all sizes, responses remained edge-enhanced throughout the stimulus. There was no filling-in of the center. Our results imply that uniform filled-in representations of surfaces in V1 are not required for the perception of uniform surfaces and that chromatic and achromatic squares are represented differently in V1.

SIGNIFICANCE STATEMENT

We used voltage-sensitive dye imaging from V1 of behaving monkeys to test the hypothesis that visual perception of uniform surfaces is mediated by an isomorphic, filled-in representation. We found that the early population responses to chromatic and achromatic surfaces are edge enhanced, emphasizing the importance of edges in surface processing. Next, we show for color surfaces that responses remained edge-enhanced throughout the stimulus presentation whereas response to luminance surfaces showed a slow neuronal 'filling-in' of the center. Our results suggest that isomorphic representation is not a general code for uniform surfaces in V1.

A Neurocomputational account of the role of contour facilitation in brightness perception

A new filling-in model is proposed in order to account for challenging brightness illusions, where inducing background elements are spatially separated from the gray target such as dungeon, cube and grating illusions, bullseye display and ring patterns. This model implements the simple idea that neural response to low-contrast contour is enhanced (facilitated) by the presence of collinear or parallel high-contrast contours in its wider neighborhood. Contour facilitation is achieved via dendritic inhibition, which enables the computation of maximum function among inputs to the node. Recurrent application of maximum function leads to the propagation of the neural signal along collinear or parallel contour segments. When a strong global-contour signal is accompanied with a weak local-contour signal at the same location, conditions are met to produce brightness assimilation within the Filling-in Layer. Computer simulations showed that the model correctly predicts brightness appearance in all of the aforementioned illusions as well as in White's effect, Benary's cross, Todorović's illusion, checkerboard contrast, contrast-contrast illusion and various variations of the White's effect. The proposed model offers new insights on how geometric factors (contour colinearity or parallelism), together with contrast magnitude contribute to the brightness perception.

A contrast and surface code explains complex responses to black and white stimuli in V1

We investigated the cortical mechanisms underlying the visual perception of luminance-defined surfaces and the preference for black over white stimuli in the macaque primary visual cortex, V1. We measured V1 population responses with voltage-sensitive dye imaging in fixating monkeys that were presented with white or black squares of equal contrast around a mid-gray. Regions corresponding to the squares' edges exhibited higher activity than those corresponding to the center. Responses to black were higher than to white, surprisingly to a much greater extent in the representation of the square's center. Additionally, the square-evoked activation patterns exhibited spatial modulations along the edges and corners. A model comprised of neural mechanisms that compute local contrast, local luminance temporal modulations in the black and white directions, and cortical center-surround interactions, could explain the observed population activity patterns in detail. The model captured the weaker contribution of V1 neurons that respond to positive (white) and negative (black) luminance surfaces, and the stronger contribution of V1 neurons that respond to edge contrast. Also, the model demonstrated how the response preference for black could be explained in terms of stronger surface-related activation to negative luminance modulation. The spatial modulations along the edges were accounted for by surround suppression. Overall the results reveal the relative strength of edge contrast and surface signals in the V1 response to visual objects.

Asymmetrical color filling-in from the nasal to the temporal side of the blind spot

The physiological blind spot, corresponding to the optic disk in the retina, is a relatively large (6 × 8°) area in the visual field that receives no retinal input. However, we rarely notice the existence of it in daily life. This is because the blind spot fills in with the brightness, color, texture, and motion of the surround. The study of filling-in enables us to better understand the creative nature of the visual system, which generates perceptual information where there is none. Is there any retinotopic rule in the color filling-in of the blind spot? To find out, we used mono-colored and bi-colored annuli hugging the boundary of the blind spot. We found that mono-colored annuli filled in the blind spot uniformly. By contrast, bi-colored annuli, where one half had a given color, while the other half had a different one, filled in the blind spot asymmetrically. Specifically, the color surrounding the nasal half typically filled in about 75% of the blind spot area, whereas the color surrounding the temporal half filled in only about 25%. This asymmetry was dependent on the relative size of the half rings, but not the two colors used, and was absent when the bi-colored annulus was rotated by 90°. Here, the two colors on the upper and lower sides of the blind spot filled in the enclosed area equally. These results suggest that the strength of filling-in decreases with distance from the fovea consistent with the decrease of the cortical magnification factor.

Early representation of shape by onset synchronization of border-ownership-selective cells in the V1-V2 network

Construction of surface is a crucial step toward the representation of shape through the integration of local information. Physiological studies have reported that the primary visual cortex (V1) codes the medial axis (MA) that is a skeletal structure equidistant from nearby contours, suggesting the early representation of surface in V1. Although the neural basis of surface construction has not been clarified, the onset synchronization of border ownership (BO)-selective cells is a plausible candidate for the generation of surface. We investigated computationally the representation of surface in a biophysically detailed model of primate V1-V2 networks. The simulation results showed that the simultaneous arrival of signals from BO-selective cells evoked strong responses of V1 cells located around the MA. The simulation results lead to a prediction that the perception of the direction of figure (DOF) depends on the degree of synchronous presentation of contour. We conducted a psychophysical experiment and showed that the perception of the DOF is biased toward a highly synchronized contour. These results suggest a crucial role of the onset synchronization of BO-selective cells for the construction of early representation of shape.

The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.

Neural dynamics of image representation in the primary visual cortex

Horizontal connections in the primary visual cortex have been hypothesized to play a number of computational roles: association field for contour completion, surface interpolation, surround suppression, and saliency computation. Here, we argue that horizontal connections might also serve a critical role for computing the appropriate codes for image representation. That the early visual cortex or V1 explicitly represents the image we perceive has been a common assumption in computational theories of efficient coding (Olshausen and Field (1996)), yet such a framework for understanding the circuitry in V1 has not been seriously entertained in the neurophysiological community. In fact, a number of recent fMRI and neurophysiological studies cast doubt on the neural validity of such an isomorphic representation (Cornelissen et al., 2006; von der Heydt et al., 2003). In this study, we investigated, neurophysiologically, how V1 neurons respond to uniform color surfaces and show that spiking activities of neurons can be decomposed into three components: a bottom-up feedforward input, an articulation of color tuning and a contextual modulation signal that is inversely proportional to the distance away from the bounding contrast border. We demonstrate through computational simulations that the behaviors of a model for image representation are consistent with many aspects of our neural observations. We conclude that the hypothesis of isomorphic representation of images in V1 remains viable and this hypothesis suggests an additional new interpretation of the functional roles of horizontal connections in the primary visual cortex.

A possible role and basis of visual pathway selection in brightness induction

It is a well-known fact that the perceived brightness of any surface depends on the brightness of the surfaces that surround it. This phenomenon is termed as brightness induction. Isotropic arrays of multi-scale DoG (Difference of Gaussians) as well as cortical Oriented DoG (ODOG) and extensions thereof, like the Frequency-specific Locally Normalized ODOG (FLODOG) functions have been employed towards prediction of the direction of brightness induction in many brightness perception effects. But the neural basis of such spatial filters is seldom obvious. For instance, the visual information from retinal ganglion cells to such spatial filters, which have been generally speculated to appear at the early stage of cortical processing, are fed by at least three parallel channels viz. Parvocellular (P), Magnocellular (M) and Koniocellular (K) in the subcortical pathway, but the role of such pathways in brightness induction is generally not implicit. In this work, three different spatial filters based on an extended classical receptive field (ECRF) model of retinal ganglion cells, have been approximately related to the spatial contrast sensitivity functions of these three parallel channels. Based on our analysis involving different brightness perception effects, we propose that the M channel, with maximum conduction velocity, may have a special role for an initial sensorial perception. As a result, brightness assimilation may be the consequence of vision at a glance through the M pathway; contrast effect may be the consequence of a subsequent vision with scrutiny through the P channel; and the K pathway response may represent an intermediate situation resulting in ambiguity in brightness perception. The present work attempts to correlate this phenomenon of pathway selection with the complementary nature of these channels in terms of spatial frequency as well as contrast.

Spatial pattern of BOLD fMRI activation reveals cross-modal information in auditory cortex

Recent findings suggest that neural representations in early auditory cortex reflect not only the physical properties of a stimulus, but also high-level, top-down, and even cross-modal information. However, the nature of cross-modal information in auditory cortex remains poorly understood. Here, we used pattern analyses of fMRI data to ask whether early auditory cortex contains information about the visual environment. Our data show that 1) early auditory cortex contained information about a visual stimulus when there was no bottom-up auditory signal, and that 2) no influence of visual stimulation was observed in auditory cortex when visual stimuli did not provide a context relevant to audition. Our findings attest to the capacity of auditory cortex to reflect high-level, top-down, and cross-modal information and indicate that the spatial patterns of activation in auditory cortex reflect contextual/implied auditory information but not visual information per se.

Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.

Representation of surface luminance and contrast in primary visual cortex

In visual perception, object identification requires both the ability to define regions of uniform luminance and zones of luminance contrast. Neural processes underlying contrast detection have been well studied, while those defining luminance remain poorly understood and controversial. Partially because stimuli comprised of uniform luminance are relatively ineffective in driving responses of cortical neurons, little effort has been made to systematically compare responses of individual neurons to both uniform luminance and contrast. Using large static uniform luminance and contrast stimuli, modulated temporally in luminance or contrast, we found a continuum of responses ranging from a few cells modulated only by luminance (luminance-only), to many cells modulated by both luminance and contrast (luminance-contrast), and to many others modulated only by contrast (contrast-only) in primary visual cortex. Moreover, luminance-contrast cells had broader orientation tuning, larger receptive field (RF) and lower spatial frequency Preference, on average, than contrast-only cells. Contrast-only cells had contrast responses more linearly correlated to the spatial structure of their RFs than luminance-contrast cells. Taken together these results suggest that luminance and contrast are represented, to some degree, by independent mechanisms that may be shaped by different classes of subcortical and/or cortical inputs.

The timing of the cognitive cycle

We propose that human cognition consists of cascading cycles of recurring brain events. Each cognitive cycle senses the current situation, interprets it with reference to ongoing goals, and then selects an internal or external action in response. While most aspects of the cognitive cycle are unconscious, each cycle also yields a momentary "ignition" of conscious broadcasting. Neuroscientists have independently proposed ideas similar to the cognitive cycle, the fundamental hypothesis of the LIDA model of cognition. High-level cognition, such as deliberation, planning, etc., is typically enabled by multiple cognitive cycles. In this paper we describe a timing model LIDA's cognitive cycle. Based on empirical and simulation data we propose that an initial phase of perception (stimulus recognition) occurs 80-100 ms from stimulus onset under optimal conditions. It is followed by a conscious episode (broadcast) 200-280 ms after stimulus onset, and an action selection phase 60-110 ms from the start of the conscious phase. One cognitive cycle would therefore take 260-390 ms. The LIDA timing model is consistent with brain evidence indicating a fundamental role for a theta-gamma wave, spreading forward from sensory cortices to rostral corticothalamic regions. This posteriofrontal theta-gamma wave may be experienced as a conscious perceptual event starting at 200-280 ms post stimulus. The action selection component of the cycle is proposed to involve frontal, striatal and cerebellar regions. Thus the cycle is inherently recurrent, as the anatomy of the thalamocortical system suggests. The LIDA model fits a large body of cognitive and neuroscientific evidence. Finally, we describe two LIDA-based software agents: the LIDA Reaction Time agent that simulates human performance in a simple reaction time task, and the LIDA Allport agent which models phenomenal simultaneity within timeframes comparable to human subjects. While there are many models of reaction time performance, these results fall naturally out of a biologically and computationally plausible cognitive architecture.

Seeing grating-textured surface begins at the border

Two experiments were conducted to reveal that the human visual system represents grating texture surface using a border-to-interior strategy. This strategy dictates that the visual system first registers the surface boundary contour and then sequentially spreads texture from the border to the interior of the image. Our experiments measured the perceived grating texture surface at various stimulus durations after the onset of a grating texture image. We found that the grating texture is initially seen near the boundary contours, with eventual spreading inward to the center of the image. To quantify the observation, the extent of the texture spreading from the boundary contour is measured as a function of the stimulus duration (30-500 ms). This allows us to analyze the texture spreading in retinal and cortical distances, based on human fMRI studies of the cortical magnification factor in cortical areas V1-V4, and to derive the spreading speed. We found that the spreading speed is constant when scaled according to the cortical distance. Similar findings are obtained no matter whether the grating texture image is presented monocularly or dichoptically, suggesting the generality of the border-to-interior strategy for representing surfaces.

Neuronal activity in the visual cortex reveals the temporal order of cognitive operations

Most mental processes consist of a number of processing steps that are executed sequentially. The timing of the individual mental operations can usually only be estimated indirectly, from the pattern of reaction times. In vision, however, many processing steps are associated with the modulation of neuronal activity in early visual areas. Here we exploited this association to elucidate the time course of neuronal activity related to each of the self-paced mental processing steps in complex visual tasks. We trained monkeys to perform two tasks, search-trace and trace-search, which required performing a sequence of two operations: a visual search for a specific color and the mental tracing of a curve. We used multielectrode recording techniques to monitor the representations of multiple visual items in area V1 at the same time and found that the relevant curve as well as the target of visual search evoked enhanced neuronal activity with a timing that depended on the order of operations. This modulation of neuronal activity in early visual areas could allow these areas to (1) act as a cognitive blackboard that permits the exchange of information between successive processing steps of a sequential visual task and to (2) contribute to the orderly progression of task-dependent endogenous attention shifts that are driven by task structure and evolve over hundreds of milliseconds.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.

Feed-forward segmentation of figure-ground and assignment of border-ownership

Figure-ground is the segmentation of visual information into objects and their surrounding backgrounds. Two main processes herein are boundary assignment and surface segregation, which rely on the integration of global scene information. Recurrent processing either by intrinsic horizontal connections that connect surrounding neurons or by feedback projections from higher visual areas provide such information, and are considered to be the neural substrate for figure-ground segmentation. On the contrary, a role of feedforward projections in figure-ground segmentation is unknown. To have a better understanding of a role of feedforward connections in figure-ground organization, we constructed a feedforward spiking model using a biologically plausible neuron model. By means of surround inhibition our simple 3-layered model performs figure-ground segmentation and one-sided border-ownership coding. We propose that the visual system uses feed forward suppression for figure-ground segmentation and border-ownership assignment.

Rebound spiking as a neural mechanism for surface filling-in

Perceptual filling-in is the phenomenon where visual information is perceived although information is not physically present. For instance, the blind spot, which corresponds to the retinal location where there are no photoreceptor cells to capture the visual signals, is filled-in by the surrounding visual signals. The neural mechanism for such immediate filling-in of surfaces is unclear. By means of computational modeling, we show that surround inhibition produces rebound or after-discharge spiking in neurons that otherwise do not receive sensory information. The behavior of rebound spiking mimics the immediate surface filling-in illusion observed at the blind spot and also reproduces the filling-in of an empty object after a background flash, like in the color dove illusion. In conclusion, we propose rebound spiking as a possible neural mechanism for surface filling-in.

Recognition alters the spatial pattern of FMRI activation in early retinotopic cortex

Early retinotopic cortex has traditionally been viewed as containing a veridical representation of the low-level properties of the image, not imbued by high-level interpretation and meaning. Yet several recent results indicate that neural representations in early retinotopic cortex reflect not just the sensory properties of the image, but also the perceived size and brightness of image regions. Here we used functional magnetic resonance imaging pattern analyses to ask whether the representation of an object in early retinotopic cortex changes when the object is recognized compared with when the same stimulus is presented but not recognized. Our data confirmed this hypothesis: the pattern of response in early retinotopic visual cortex to a two-tone "Mooney" image of an object was more similar to the response to the full grayscale photo version of the same image when observers knew what the two-tone image represented than when they did not. Further, in a second experiment, high-level interpretations actually overrode bottom-up stimulus information, such that the pattern of response in early retinotopic cortex to an identified two-tone image was more similar to the response to the photographic version of that stimulus than it was to the response to the identical two-tone image when it was not identified. Our findings are consistent with prior results indicating that perceived size and brightness affect representations in early retinotopic visual cortex and, further, show that even higher-level information--knowledge of object identity--also affects the representation of an object in early retinotopic cortex.