Cortical processing of a brightness illusion

Brightness illusions drive a neuronal response in the primary visual cortex under top-down modulation

Brightness illusions are a powerful tool in studying vision, yet their neural correlates are poorly understood. Based on a human paradigm, we presented illusory drifting gratings to mice. Primary visual cortex (V1) neurons responded to illusory gratings, matching their direction selectivity for real gratings, and they tracked the spatial phase offset between illusory and real gratings. Illusion responses were delayed compared to real gratings, in line with the theory that processing illusions requires feedback from higher visual areas (HVAs). We provide support for this theory by showing a reduced V1 response to illusions, but not real gratings, following HVAs optogenetic inhibition. Finally, we used the pupil response (PR) as an indirect perceptual report and showed that the mouse PR matches the human PR to perceived luminance changes. Our findings resolve debates over whether V1 neurons are involved in processing illusions and highlight the involvement of feedback from HVAs.

Mesoscale organization of ventral and dorsal visual pathways in macaque monkey revealed by 7T fMRI

In human and nonhuman primate brains, columnar (mesoscale) organization has been demonstrated to underlie both lower and higher order aspects of visual information processing. Previous studies have focused on identifying functional preferences of mesoscale domains in specific areas; but there has been little understanding of how mesoscale domains may cooperatively respond to single visual stimuli across dorsal and ventral pathways. Here, we have developed ultrahigh-field 7 T fMRI methods to enable simultaneous mapping, in individual macaque monkeys, of response in both dorsal and ventral pathways to single simple color and motion stimuli. We provide the first evidence that anatomical V2 cytochrome oxidase-stained stripes are well aligned with fMRI maps of V2 stripes, settling a long-standing controversy. In the ventral pathway, a systematic array of paired color and luminance processing domains across V4 was revealed, suggesting a novel organization for surface information processing. In the dorsal pathway, in addition to high quality motion direction maps of MT, MST and V3A, alternating color and motion direction domains in V3 are revealed. As well, submillimeter motion domains were observed in peripheral LIPd and LIPv. In sum, our study provides a novel global snapshot of how mesoscale networks in the ventral and dorsal visual pathways form the organizational basis of visual objection recognition and vision for action.

Reconstructing visual illusory experiences from human brain activity

Visual illusions provide valuable insights into the brain's interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain's internal representations of the world.

Spatial frequency representation in V2 and V4 of macaque monkey

Spatial frequency (SF) is an important attribute in the visual scene and is a defining feature of visual processing channels. However, there remain many unsolved questions about how extrastriate areas in primate visual cortex code this fundamental information. Here, using intrinsic signal optical imaging in visual areas of V2 and V4 of macaque monkeys, we quantify the relationship between SF maps and (1) visual topography and (2) color and orientation maps. We find that in orientation regions, low to high SF is mapped orthogonally to orientation; in color regions, which are reported to contain orthogonal axes of color and lightness, low SFs tend to be represented more frequently than high SFs. This supports a population-based SF fluctuation related to the 'color/orientation' organizations. We propose a generalized hypercolumn model across cortical areas, comprised of two orthogonal parameters with additional parameters.

Computational modeling of color perception with biologically plausible spiking neural networks

Biologically plausible computational modeling of visual perception has the potential to link high-level visual experiences to their underlying neurons' spiking dynamic. In this work, we propose a neuromorphic (brain-inspired) Spiking Neural Network (SNN)-driven model for the reconstruction of colorful images from retinal inputs. We compared our results to experimentally obtained V1 neuronal activity maps in a macaque monkey using voltage-sensitive dye imaging and used the model to demonstrate and critically explore color constancy, color assimilation, and ambiguous color perception. Our parametric implementation allows critical evaluation of visual phenomena in a single biologically plausible computational framework. It uses a parametrized combination of high and low pass image filtering and SNN-based filling-in Poisson processes to provide adequate color image perception while accounting for differences in individual perception.

Neural correlates of lateral modulation and perceptual filling-in in center-surround radial sinusoidal gratings: an fMRI study

We investigated lateral modulation effects with functional magnetic resonance imaging. We presented radial sinusoidal gratings in random sequence: a scotoma grating with two arc-shaped blank regions (scotomata) in the periphery, one in the left and one in the right visual field, a center grating containing pattern only in the scotoma regions, and a full-field grating where the pattern occupied the whole screen. On each trial, one of the three gratings flickered in counterphase for 10 s, followed by a blank period. Observers were instructed to perform a fixation task and report whether filling-in was experienced during the scotoma condition. The results showed that the blood-oxygen-level-dependent signal was reduced in areas corresponding to the scotoma regions in the full-field compared to the center condition in V1 to V3 areas, indicating a lateral inhibition effect when the surround was added to the center pattern. The univariate analysis results showed no difference between the filling-in and no-filling-in trials. However, multivariate pattern analysis results showed that classifiers trained on activation pattern in V1 to V3 could differentiate between filling-in and no-filling-in trials, suggesting that the neural activation pattern in visual cortex correlated with the subjective percept.

Functionally specific and sparse domain-based micro-networks in monkey V1 and V2

The cerebral cortices of human and nonhuman primate brains are characterized by submillimeter functional domains. However, little is known about the connections of single functional domains. Here, in macaque monkey visual cortex, we have developed a targeted focal electrical stimulation method, coupled with functional optical imaging, to map cortical networks with submillimeter precision in vivo. We find that single functional domains are a part of highly specific and sparse intra-areal and inter-areal micro-networks. Across color-related and orientation-related functionalities, these micro-networks exhibit parallel connection patterns, suggesting a common domain-based architecture. Moreover, these micro-networks shift topographically at a submillimeter scale, suggesting that they serve as a fundamental unit for cortical information processing. Our findings establish a domain-based connectional architecture in the primate brain and present new constraints for cortical map representation.

Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey

Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance.

How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.

tACS phase-specifically biases brightness perception of flickering light

BACKGROUND

Visual phenomena like brightness illusions impressively demonstrate the highly constructive nature of perception. In addition to physical illumination, the subjective experience of brightness is related to temporal neural dynamics in visual cortex.

OBJECTIVE

Here, we asked whether biasing the temporal pattern of neural excitability in visual cortex by transcranial alternating current stimulation (tACS) modulates brightness perception of concurrent rhythmic visual stimuli.

METHODS

Participants performed a brightness discrimination task of two flickering lights, one of which was targeted by same-frequency electrical stimulation at varying phase shifts. tACS was applied with an occipital and a periorbital active control montage, based on simulations of electrical currents using finite element head models.

RESULTS

Experimental results reveal that flicker brightness perception is modulated dependent on the phase shift between sensory and electrical stimulation, solely under occipital tACS. Phase-specific modulatory effects by tACS were dependent on flicker-evoked neural phase stability at the tACS-targeted frequency, recorded prior to electrical stimulation. Further, the optimal timing of tACS application leading to enhanced brightness perception was correlated with the neural phase delay of the cortical flicker response.

CONCLUSIONS

Our results corroborate the role of temporally coordinated neural activity in visual cortex for brightness perception of rhythmic visual input in humans. Phase-specific behavioral modulations by tACS emphasize its efficacy to transfer perceptually relevant temporal information to the cortex. These findings provide an important step towards understanding the basis of visual perception and further confirm electrical stimulation as a tool for advancing controlled modulations of neural activity and related behavior.

Multiple gamma rhythms carry distinct spatial frequency information in primary visual cortex

Gamma rhythms in many brain regions, including the primary visual cortex (V1), are thought to play a role in information processing. Here, we report a surprising finding of 3 narrowband gamma rhythms in V1 that processed distinct spatial frequency (SF) signals and had different neural origins. The low gamma (LG; 25 to 40 Hz) rhythm was generated at the V1 superficial layer and preferred a higher SF compared with spike activity, whereas both the medium gamma (MG; 40 to 65 Hz), generated at the cortical level, and the high gamma HG; (65 to 85 Hz), originated precortically, preferred lower SF information. Furthermore, compared with the rates of spike activity, the powers of the 3 gammas had better performance in discriminating the edge and surface of simple objects. These findings suggest that gamma rhythms reflect the neural dynamics of neural circuitries that process different SF information in the visual system, which may be crucial for multiplexing SF information and synchronizing different features of an object.

Echolocating bats detect but misperceive a multidimensional incongruent acoustic stimulus

Coherent perception relies on integrating multiple dimensions of a sensory modality, for example, color and shape in vision. We reveal how different acoustic dimensions, specifically echo intensity and sonar aperture (or width), are important for correct perception by echolocating bats. We flew bats down a corridor blocked by objects with different intensity-aperture combinations. To our surprise, bats crashed straight into large (aperture) walls with weak echo intensity as if they did not exist. The echolocation behavior of the bats indicated that they did detect the wall, suggesting that crashing was not a result of limited sensory sensitivity, but of a perceptual deficit. We systematically manipulated intensity and aperture by changing the materials and width of different reflectors, and we conclude that a coherent echo-based percept is created only when these two acoustic dimensions have certain relations which are typical for objects in nature (e.g., large and intense or small and weak reflectors). Nevertheless, we show that these preferred relations are not innate. We show that young pups are not constrained to these relations and that new intensity-aperture associations can also be learned by adult bats.

Lateral modulation of orientation perception in center-surround sinusoidal stimuli: Divisive inhibition in perceptual filling-in

The perception of a target stimulus may be altered by its context. Perceptual filling-in is thought to be one example of lateral modulation, in which the percept of a central blank area is replaced by that of the surround. We investigated the mechanisms in eccentric vision underlying filling-in by selectively adapting the center (pedestal adapter), surround (annulus adapter), or both (disk adapter) in a sinusoidal grating and observed how the adaptation influences the orientation percept of a subsequently presented Gabor target, located at the same position as the adapter center. In a binary choice task, observers were to judge the orientation (clockwise or counterclockwise) of the target after adaptation. The tilt aftereffect (TAE), corresponding to an illusory tilt of a physically vertical Gabor target, depended both on the adapter orientation and the adapter type. The TAE, peaked between 10 degrees and 20 degrees adapter orientation, was strongest in the pedestal, followed by the disk, and weakest in the annulus adapter conditions. The difference between the disk and pedestal conditions implies lateral inhibition from the surround. Lacking physical overlap with the target, the annulus adapter nonetheless induced a small but significant TAE in the central area. The effect of filling-in on the TAE was estimated by comparing the results from trials with and without subjectively reported filling-in during adaptation to the annulus adapter. The TAE was greater when filling-in occurred during adaptation, suggesting a stronger lateral modulation effect on trials where filling-in was induced. The data were fit by a variant of a divisive inhibition model, in which the adaptation effect is captured by the increase of an additive constant in the denominator of the response function, whereas the surround modulation in the adapter is modeled by an excitatory sensitivity in the numerator.

Feedback contribution to surface motion perception in the human early visual cortex

Human visual surface perception has neural correlates in early visual cortex, but the role of feedback during surface segmentation in human early visual cortex remains unknown. Feedback projections preferentially enter superficial and deep anatomical layers, which provides a hypothesis for the cortical depth distribution of fMRI activity related to feedback. Using ultra-high field fMRI, we report a depth distribution of activation in line with feedback during the (illusory) perception of surface motion. Our results fit with a signal re-entering in superficial depths of V1, followed by a feedforward sweep of the re-entered information through V2 and V3. The magnitude and sign of the BOLD response strongly depended on the presence of texture in the background, and was additionally modulated by the presence of illusory motion perception compatible with feedback. In summary, the present study demonstrates the potential of depth-resolved fMRI in tackling biomechanical questions on perception.

Anatomy and Physiology of Macaque Visual Cortical Areas V1, V2, and V5/MT: Bases for Biologically Realistic Models

The cerebral cortex of primates encompasses multiple anatomically and physiologically distinct areas processing visual information. Areas V1, V2, and V5/MT are conserved across mammals and are central for visual behavior. To facilitate the generation of biologically accurate computational models of primate early visual processing, here we provide an overview of over 350 published studies of these three areas in the genus Macaca, whose visual system provides the closest model for human vision. The literature reports 14 anatomical connection types from the lateral geniculate nucleus of the thalamus to V1 having distinct layers of origin or termination, and 194 connection types between V1, V2, and V5, forming multiple parallel and interacting visual processing streams. Moreover, within V1, there are reports of 286 and 120 types of intrinsic excitatory and inhibitory connections, respectively. Physiologically, tuning of neuronal responses to 11 types of visual stimulus parameters has been consistently reported. Overall, the optimal spatial frequency (SF) of constituent neurons decreases with cortical hierarchy. Moreover, V5 neurons are distinct from neurons in other areas for their higher direction selectivity, higher contrast sensitivity, higher temporal frequency tuning, and wider SF bandwidth. We also discuss currently unavailable data that could be useful for biologically accurate models.

Spatiotemporal dynamics of brightness coding in human visual cortex revealed by the temporal context effect

Human visual perception is modulated by both temporal and spatial contexts. One type of modulation is apparent in the temporal context effect (TCE): In the presence of a constant luminance patch (a long flash), the perceived brightness of a short flash increases monotonically with onset asynchrony. The aim of the current study was to delineate the neural correlates of this illusory effect, particularly focusing on its dynamic neural representation among visual cortical areas. We reconstructed sources of magnetoencephalographic (MEG) data recorded from observers (6 male and 9 female human adults) experiencing the TCE. Together with retinotopic mapping, signals from different occipital lobe areas were extracted to investigate whether different visual areas have differential representation of the onset vs. offset synchronized short flashes. From the data, TCE related responses were observed in LO and V4 in the time window of 200-250 m s, while neuronal responses to physical luminances were observed in the early time window at around 100 m s across early visual cortex, such as V1 and V2, also in V4 and VO. Based on these findings, we suggest that two distinct processes might be involved in brightness coding: one bottom-up process which is stimulus energy driven and responds fast, and another process which may be broadly characterized as top-down or lateral, is context driven, and responds slower. For both processes, we found that V4 might play a critical role in dynamically integrating luminances into brightness perception, a finding that is consistent with the view of V4 as a bottom-up and top-down integration complex.

Everything is subjective under water surface, too: visual illusions in fish

The study of visual illusions has captured the attention of comparative psychologists since the last century, given the unquestionable advantage of investigating complex perceptual mechanisms with relatively simple visual patterns. To date, the observation of animal behavior in the presence of visual illusions has been largely confined to mammal and bird studies. Recently, there has been increasing interest in investigating fish, too. The attention has been particularly focused on guppies, redtail splitfin and bamboo sharks. Overall, the tested species were shown to experience a human-like perception of different illusory phenomena involving size, number, motion, brightness estimation and illusory contours. However, in some cases, no illusory effects, or evidence for a reverse illusion, were also reported. Here, we review the current state of the art in this field. We conclude that a wider investigation of visual illusions in fish is fundamental to form a broader comprehension of perceptual systems of vertebrates. Furthermore, we believe that this type of investigation could help us to address general important issues in perceptual studies, such as the role of ecology in shaping perceptual systems, the existence of interindividual variability in the visual perception of nonhuman species and the role of cortical activity in the emergence of visual illusions.

Columnar connectome: toward a mathematics of brain function

Understanding brain networks is important for many fields, including neuroscience, psychology, medicine, and artificial intelligence. To address this fundamental need, there are multiple ongoing connectome projects in the United States, Europe, and Asia producing brain connection maps with resolutions at macro- and microscales. However, still lacking is a mesoscale connectome. This viewpoint (1) explains the need for a mesoscale connectome in the primate brain (the columnar connectome), (2) presents a new method for acquiring such data rapidly on a large scale, and (3) proposes how one might use such data to achieve a mathematics of brain function.

A Compound Computational Model for Filling-In Processes Triggered by Edges: Watercolor Illusions

The goal of our research was to develop a compound computational model with the ability to predict different variations of the “watercolor effects” and additional filling-in effects that are triggered by edges. The model is based on a filling-in mechanism solved by a Poisson equation, which considers the different gradients as “heat sources” after the gradients modification. The biased (modified) contours (edges) are ranked and determined according to their dominancy across the different chromatic and achromatic channels. The color and intensity of the perceived surface are calculated through a diffusive filling-in process of color triggered by the enhanced and biased edges of stimulus formed as a result of oriented double-opponent receptive fields. The model can successfully predict both the assimilative and non-assimilative watercolor effects, as well as a number of “conflicting” visual effects. Furthermore, the model can also predict the classic Craik–O'Brien–Cornsweet (COC) effect. In summary, our proposed computational model is able to predict most of the “conflicting” filling-in effects that derive from edges that have been recently described in the literature, and thus supports the theory that a shared visual mechanism is responsible for the vast variety of the “conflicting” filling-in effects that derive from edges.

Neuronal population mechanisms of lightness perception

The way that humans and animals perceive the lightness of an object depends on its physical luminance as well as its surrounding context. While neuronal responses throughout the visual pathway are modulated by context, the relationship between neuronal responses and lightness perception is poorly understood. We searched for a neuronal mechanism of lightness by recording responses of neuronal populations in monkey primary visual cortex (V1) and area V4 to stimuli that produce a lightness illusion in humans, in which the lightness of a disk depends on the context in which it is embedded. We found that the way individual units encode the luminance (or equivalently for our stimuli, contrast) of the disk and its context is extremely heterogeneous. This motivated us to ask whether the population representation in either V1 or V4 satisfies three criteria: 1) disk luminance is represented with high fidelity, 2) the context surrounding the disk is also represented, and 3) the representations of disk luminance and context interact to create a representation of lightness that depends on these factors in a manner consistent with human psychophysical judgments of disk lightness. We found that populations of units in both V1 and V4 fulfill the first two criteria but that we cannot conclude that the two types of information in either area interact in a manner that clearly predicts human psychophysical measurements: the interpretation of our population measurements depends on how subsequent areas read out lightness from the population responses. NEW & NOTEWORTHY A core question in visual neuroscience is how the brain extracts stable representations of object properties from the retinal image. We searched for a neuronal mechanism of lightness perception by determining whether the responses of neuronal populations in primary visual cortex and area V4 could account for a lightness illusion measured using human psychophysics. Our results suggest that comparing psychophysics with population recordings will yield insight into neuronal mechanisms underlying a variety of perceptual phenomena.

Neural representation of form-contingent color filling-in in the early visual cortex

Perceptual filling-in exemplifies the constructive nature of visual processing. Color, a prominent surface property of visual objects, can appear to spread to neighboring areas that lack any color. We investigated cortical responses to a color filling-in illusion that effectively dissociates perceived color from the retinal input (van Lier, Vergeer, & Anstis, 2009). Observers adapted to a star-shaped stimulus with alternating red- and cyan-colored points to elicit a complementary afterimage. By presenting an achromatic outline that enclosed one of the two afterimage colors, perceptual filling-in of that color was induced in the unadapted central region. Visual cortical activity was monitored with fMRI, and analyzed using multivariate pattern analysis. Activity patterns in early visual areas (V1–V4) reliably distinguished between the two color-induced filled-in conditions, but only higher extrastriate visual areas showed the predicted correspondence with color perception. Activity patterns allowed for reliable generalization between filled-in colors and physical presentations of perceptually matched colors in areas V3 and V4, but not in earlier visual areas. These findings suggest that the perception of filled-in surface color likely requires more extensive processing by extrastriate visual areas, in order for the neural representation of surface color to become aligned with perceptually matched real colors.

Visual evoked potentials to an illusory change in brightness: the Craik-Cornsweet-O'Brien effect

Can brain electrical activity associated with the Craik-Cornsweet-O'Brien effect (CCOB) be identified in humans? Opposing luminance gradients met in the middle of a square image to create a luminance contrast-defined vertical border. The resulting rectangles on each side of the border were otherwise equiluminant, but appeared to differ in brightness, the CCOB effect. When the contrast gradients were swapped, the participants perceived darker and lighter rectangles trading places. This dynamic CCOB stimulus was reversed 1/s to elicit visual evoked potentials. The CCOB effect was absent in two control conditions. In one, the immediate contrast border, where the gradients met, was replaced by a dark vertical stripe; in the other, the outer segments of both rectangles, where the illusion would otherwise occur, were replaced by dark rectangles, leaving only the contrast-reversing gradients. Visual evoked potential components P1 and N2 were present for the CCOB stimuli, but not the control stimuli. Results are consistent with functional MRI and single unit evidence, suggesting that the brightness of the CCOB effect becomes dissociated from the luminance falling on the eye early in visual processing. These results favor explanations of brightness induction invoking rapid, early amplification of very low spatial-frequency information in the image to approximate natural scenes as opposed to a sluggish brightness adjustment spreading from the contrast border.

Representation of Color Surfaces in V1: Edge Enhancement and Unfilled Holes

The neuronal mechanism underlying the representation of color surfaces in primary visual cortex (V1) is not well understood. We tested on color surfaces the previously proposed hypothesis that visual perception of uniform surfaces is mediated by an isomorphic, filled-in representation in V1. We used voltage-sensitive-dye imaging in fixating macaque monkeys to measure V1 population responses to spatially uniform chromatic (red, green, or blue) and achromatic (black or white) squares of different sizes (0.5°-8°) presented for 300 ms. Responses to both color and luminance squares early after stimulus onset were similarly edge-enhanced: for squares 1° and larger, regions corresponding to edges were activated much more than those corresponding to the center. At later times after stimulus onset, responses to achromatic squares' centers increased, partially "filling-in" the V1 representation of the center. The rising phase of the center response was slower for larger squares. Surprisingly, the responses to color squares behaved differently. For color squares of all sizes, responses remained edge-enhanced throughout the stimulus. There was no filling-in of the center. Our results imply that uniform filled-in representations of surfaces in V1 are not required for the perception of uniform surfaces and that chromatic and achromatic squares are represented differently in V1.

SIGNIFICANCE STATEMENT

We used voltage-sensitive dye imaging from V1 of behaving monkeys to test the hypothesis that visual perception of uniform surfaces is mediated by an isomorphic, filled-in representation. We found that the early population responses to chromatic and achromatic surfaces are edge enhanced, emphasizing the importance of edges in surface processing. Next, we show for color surfaces that responses remained edge-enhanced throughout the stimulus presentation whereas response to luminance surfaces showed a slow neuronal 'filling-in' of the center. Our results suggest that isomorphic representation is not a general code for uniform surfaces in V1.

A temporal window for estimating surface brightness in the Craik-O'Brien-Cornsweet effect

The central edge of an opposing pair of luminance gradients (COC edge) makes adjoining regions with identical luminance appear to be different. This brightness illusion, called the Craik-O'Brien-Cornsweet effect (COCe), can be explained by low-level spatial filtering mechanisms (Dakin and Bex, 2003). Also, the COCe is greatly reduced when the stimulus lacks a frame element surrounding the COC edge (Purves et al., 1999). This indicates that the COCe can be modulated by extra contextual cues that are related to ideas about lighting priors. In this study, we examined whether processing for contextual modulation could be independent of the main COCe processing mediated by the filtering mechanism. We displayed the COC edge and frame element at physically different times. Then, while varying the onset asynchrony between them and changing the luminance contrast of the frame element, we measured the size of the COCe. We found that the COCe was observed in the temporal range of around 600–800 ms centered at the 0 ms (from around −400 to 400 ms in stimulus onset asynchrony), which was much larger than the range of typical visual persistency. More importantly, this temporal range did not change significantly regardless of differences in the luminance contrast of the frame element (5–100%), in the durations of COC edge and/or the frame element (50 or 200 ms), in the display condition (interocular or binocular), and in the type of lines constituting the frame element (solid or illusory lines). Results suggest that the visual system can bind the COC edge and frame element with a temporal window of ~1 s to estimate surface brightness. Information from the basic filtering mechanism and information of contextual cue are separately processed and are linked afterwards.

A contrast and surface code explains complex responses to black and white stimuli in V1

We investigated the cortical mechanisms underlying the visual perception of luminance-defined surfaces and the preference for black over white stimuli in the macaque primary visual cortex, V1. We measured V1 population responses with voltage-sensitive dye imaging in fixating monkeys that were presented with white or black squares of equal contrast around a mid-gray. Regions corresponding to the squares' edges exhibited higher activity than those corresponding to the center. Responses to black were higher than to white, surprisingly to a much greater extent in the representation of the square's center. Additionally, the square-evoked activation patterns exhibited spatial modulations along the edges and corners. A model comprised of neural mechanisms that compute local contrast, local luminance temporal modulations in the black and white directions, and cortical center-surround interactions, could explain the observed population activity patterns in detail. The model captured the weaker contribution of V1 neurons that respond to positive (white) and negative (black) luminance surfaces, and the stronger contribution of V1 neurons that respond to edge contrast. Also, the model demonstrated how the response preference for black could be explained in terms of stronger surface-related activation to negative luminance modulation. The spatial modulations along the edges were accounted for by surround suppression. Overall the results reveal the relative strength of edge contrast and surface signals in the V1 response to visual objects.

A model of proto-object based saliency

Organisms use the process of selective attention to optimally allocate their computational resources to the instantaneously most relevant subsets of a visual scene, ensuring that they can parse the scene in real time. Many models of bottom-up attentional selection assume that elementary image features, like intensity, color and orientation, attract attention. Gestalt psychologists, however, argue that humans perceive whole objects before they analyze individual features. This is supported by recent psychophysical studies that show that objects predict eye-fixations better than features. In this report we present a neurally inspired algorithm of object based, bottom-up attention. The model rivals the performance of state of the art non-biologically plausible feature based algorithms (and outperforms biologically plausible feature based algorithms) in its ability to predict perceptual saliency (eye fixations and subjective interest points) in natural scenes. The model achieves this by computing saliency as a function of proto-objects that establish the perceptual organization of the scene. All computational mechanisms of the algorithm have direct neural correlates, and our results provide evidence for the interface theory of attention.

Neural mechanism for sensing fast motion in dim light

Luminance is a fundamental property of visual scenes. A population of neurons in primary visual cortex (V1) is sensitive to uniform luminance. In natural vision, however, the retinal image often changes rapidly. Consequently the luminance signals visual cells receive are transiently varying. How V1 neurons respond to such luminance changes is unknown. By applying large static uniform stimuli or grating stimuli altering at 25 Hz that resemble the rapid luminance changes in the environment, we show that approximately 40% V1 cells responded to rapid luminance changes of uniform stimuli. Most of them strongly preferred luminance decrements. Importantly, when tested with drifting gratings, the preferred speeds of these cells were significantly higher than cells responsive to static grating stimuli but not to uniform stimuli. This responsiveness can be accounted for by the preferences for low spatial frequencies and high temporal frequencies. These luminance-sensitive cells subserve the detection of fast motion under the conditions of dim illumination.

Population response to natural images in the primary visual cortex encodes local stimulus attributes and perceptual processing

The primary visual cortex (V1) is extensively studied with a large repertoire of stimuli, yet little is known about its encoding of natural images. Using voltage-sensitive dye imaging in behaving monkeys, we measured neural population response evoked in V1 by natural images presented during a face/scramble discrimination task. The population response showed two distinct phases of activity: an early phase that was spread over most of the imaged area, and a late phase that was spatially confined. To study the detailed relation between the stimulus and the population response, we used a simple encoding model to compute a continuous map of the expected neural response based on local attributes of the stimulus (luminance and contrast), followed by an analytical retinotopic transformation. Then, we computed the spatial correlation between the maps of the expected and observed response. We found that the early response was highly correlated with the local luminance of the stimulus and was sufficient to effectively discriminate between stimuli at the single trial level. The late response, on the other hand, showed a much lower correlation to the local luminance, was confined to central parts of the face images, and was highly correlated with the animal's perceptual report. Our study reveals a continuous spatial encoding of low- and high-level features of natural images in V1. The low level is directly linked to the stimulus basic local attributes and the high level is correlated with the perceptual outcome of the stimulus processing.

Emergent filling in induced by motion integration reveals a high-level mechanism in filling in

The visual system is intelligent--it is capable of recovering a coherent surface from an incomplete one, a feat known as perceptual completion or filling in. Traditionally, it has been assumed that surface features are interpolated in a way that resembles the fragmented parts. Using displays featuring four circular apertures, we showed in the study reported here that a distinct completed feature (horizontal motion) arises from local ones (oblique motions)-we term this process emergent filling in. Adaptation to emergent filling-in motion generated a dynamic motion aftereffect that was not due to spreading of local motion from the isolated apertures. The filling-in motion aftereffect occurred in both modal and amodal completions, and it was modulated by selective attention. These findings highlight the importance of high-level interpolation processes in filling in and are consistent with the idea that during emergent filling in, the more cognitive-symbolic processes in later areas (e.g., the middle temporal visual area and the lateral occipital complex) provide important feedback signals to guide more isomorphic processes in earlier areas (V1 and V2).

Effects of luminance on dynamic random-dot correlogram evoked visual potentials

Although dynamic random-dot correlogram evoked visual potentials (DRDC-VEPs) are a three-decade-old method to detect the cortical binocularity in humans and animals, our knowledge of the influence of fundamental stimulus parameters and the underlying cerebral processing mechanisms has remained limited. The purpose of this study was to evaluate the effect of luminance on DRDC-VEPs in adults. The variability and detectability of DRDC-VEPs were investigated under different stimulus luminance conditions with neutral density filters. Our results have demonstrated that DRDC-VEPs can be evoked in a wide luminance range, and the response amplitude was practically independent of luminance between 4.75 cd m(-2) and 0.015 cd m(-2), while DRDC-VEP latencies showed a strong linear correlation with log luminance. There is, however, a limit (0.06 cd m(-2)) below which DRDC-VEPs are not reliably recordable. Luminance reduction-induced delays in DRDC-VEP latencies cannot be explained simply by retinal mechanisms, since their regression slope does not follow the course of electroretinogram and cortical evoked potential latencies. Luminance independence of DRDC-VEP amplitude suggests that binocular correlation-processing cortical neurons receive input predominantly from the magnocellular visual pathway.

Neural dynamics of image representation in the primary visual cortex

Horizontal connections in the primary visual cortex have been hypothesized to play a number of computational roles: association field for contour completion, surface interpolation, surround suppression, and saliency computation. Here, we argue that horizontal connections might also serve a critical role for computing the appropriate codes for image representation. That the early visual cortex or V1 explicitly represents the image we perceive has been a common assumption in computational theories of efficient coding (Olshausen and Field (1996)), yet such a framework for understanding the circuitry in V1 has not been seriously entertained in the neurophysiological community. In fact, a number of recent fMRI and neurophysiological studies cast doubt on the neural validity of such an isomorphic representation (Cornelissen et al., 2006; von der Heydt et al., 2003). In this study, we investigated, neurophysiologically, how V1 neurons respond to uniform color surfaces and show that spiking activities of neurons can be decomposed into three components: a bottom-up feedforward input, an articulation of color tuning and a contextual modulation signal that is inversely proportional to the distance away from the bounding contrast border. We demonstrate through computational simulations that the behaviors of a model for image representation are consistent with many aspects of our neural observations. We conclude that the hypothesis of isomorphic representation of images in V1 remains viable and this hypothesis suggests an additional new interpretation of the functional roles of horizontal connections in the primary visual cortex.

Color blobs in cortical areas V1 and V2 of the new world monkey Callithrix jacchus, revealed by non-differential optical imaging

Color vision is reserved to only few mammals, such as Old World monkeys and humans. Most Old World monkeys are trichromats. Among them, macaques were shown to exhibit functional domains of color-selectivity, in areas V1 and V2 of the visual cortex. Such color domains have not yet been shown in New World monkeys. In marmosets a sex-linked dichotomy results in dichromatic and trichromatic genotypes, rendering most male marmosets color-blind. Here we used trichromatic female marmosets to examine the intrinsic signal response in V1 and V2 to chromatic and achromatic stimuli, using optical imaging. To activate the subsystems individually, we used spatially homogeneous isoluminant color opponent (red/green, blue/yellow) and hue versus achromatic flicker (red/gray, green/gray, blue/gray, yellow/gray), as well as achromatic luminance flicker. In contrast to previous optical imaging studies in marmosets, we find clearly segregated color domains, similar to those seen in macaques. Red/green and red/gray flicker were found to be the appropriate stimulus for revealing color domains in single-condition maps. Blue/gray and blue/yellow flicker stimuli resulted in faint patch-patterns. A recently described multimodal vessel mapping approach allowed for an accurate alignment of the functional and anatomical datasets. Color domains were tightly colocalized with cytochrome oxidase blobs in V1 and with thin stripes in V2. Thus, our findings are in accord with 2-Deoxy-D-glucose studies performed in V1 of macaques and studies on color representation in V2. Our results suggest a similar organization of early cortical color processing in trichromats of both Old World and New World monkeys.

Population receptive field dynamics in human visual cortex

Seminal work in the early nineties revealed that the visual receptive field of neurons in cat primary visual cortex can change in location and size when artificial scotomas are applied. Recent work now suggests that these single neuron receptive field dynamics also pertain to the neuronal population receptive field (pRF) that can be measured in humans with functional magnetic resonance imaging (fMRI). To examine this further, we estimated the pRF in twelve healthy participants while masking the central portion of the visual field. We found that the pRF changes in location and size for two differently sized artificial scotomas, and that these pRF dynamics are most likely due to a combination of the neuronal receptive field position and size scatter as well as modulatory feedback signals from extrastriate visual areas.

Toward a unified theory of visual area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.

Illusions of brightness, length, and now size: comparable effects derived from the Craik-O'Brien-Cornsweet distributions

Analogous illusions of brightness, line length, and object size occur when stimulus distributions follow the Mach function. Similarly, analogous illusions of brightness and length are produced by stimulus distributions that follow the Craik-O'Brien-Cornsweet function. In Exp. 1, object size is studied using Craik-O'Brien-Cornsweet stimuli composed of 12 clip-art representations of candles, swords, forks, screwdrivers, or pens. Despite their equal dimensions, the three objects next to the taller gradient were judged larger than the three adjacent to the shorter gradient. In Exp. 2, variations of the sword and screwdriver figures having three separations of their constituent objects were used. The illusion produced in Exp. 1 was replicated and, for the figures composed of screwdrivers, increasing separation reduced the hypothesized mis-estimations. The relationship of these Craik-O'Brien-Cornsweet size illusions to their brightness and length counterparts mirrors the relationships previously reported for Mach stimuli; moreover, these findings converge to suggest that the visual system registers size of objects with a frequency code and that illusions of size appear when interactions among neurons disrupt this code.

Early Human Visual Cortex Encodes Surface Brightness Induced by Dynamic Context

Visual scene perception owes greatly to surface features such as color and brightness. Yet, early visual cortical areas predominantly encode surface boundaries rather than surface interiors. Whether human early visual cortex may nevertheless carry a small signal relevant for surface perception is a topic of debate. We induced brightness changes in a physically constant surface by temporally modulating the luminance of surrounding surfaces in seven human participants. We found that fMRI activity in the V2 representation of the constant surface was in antiphase to luminance changes of surrounding surfaces (i.e., activity was in-phase with perceived brightness changes). Moreover, the amplitude of the antiphase fMRI activity in V2 predicted the strength of illusory brightness perception. We interpret our findings as evidence for a surface-related signal in early visual cortex and discuss the neural mechanisms that may underlie that signal in concurrence with its possible interaction with the properties of the fMRI signal.

Bright illusions reduce the eye's pupil

We recorded by use of an infrared eye-tracker the pupil diameters of participants while they observed visual illusions of lightness or brightness. Four original illusions {based on Gaetano Kanisza's [Kanizsa G (1976) Subjective contours. Sci Am 234:48-52] and Akiyoshi Kitaoka's [Kitaoka A. (2005) Trick Eyes (Barnes & Noble, New Providence, NJ).] examples} were manipulated to obtain control conditions in which the perceived illusory luminance was either eliminated or reduced. All stimuli were equiluminant so that constrictions in pupillary size could not be ascribed to changes in light energy. We found that the pupillary diameter rapidly varied according to perceived brightness and lightness strength. Differences in local contrast information could be ruled out as an explanation because, in a second experiment, the observers maintained eye fixation in the center of the display; thus, differential stimulation of the fovea by local contrast changes could not be responsible for the pupillary differences. Hence, the most parsimonious explanation for the present findings is that pupillary responses to ambient light reflect the perceived brightness or lightness of the scene and not simply the amount of physical light energy entering the eye. Thus, the pupillary physiological response reflects the subjective perception of light and supports the idea that the brain's visual circuitry is shaped by visual experience with images and their possible sources.

Awareness of Central Luminance Edge is Crucial for the Craik-O'Brien-Cornsweet Effect

The Craik-O'Brien-Cornsweet (COC) effect demonstrates that perceived lightness depends not only on the retinal input at corresponding visual areas but also on distal retinal inputs. In the COC effect, the central edge of an opposing pair of luminance gradients (COC edge) makes adjoining regions with identical luminance appear to be different. To investigate the underlying mechanisms of the effect, we examined whether the subjective awareness of the COC edge is necessary for the generation of the effect. We manipulated the visibility of the COC edge using visual backward masking and continuous flash suppression while monitoring subjective reports regarding online percepts and aftereffects of adaptation. Psychophysical results showed that the online percept of the COC effect nearly vanishes in conditions where the COC edge is rendered invisible. On the other hand, the results of adaptation experiments showed that the COC edge is still processed at the early stage even under the perceptual suppression. These results suggest that processing of the COC edge at the early stage is not sufficient for generating the COC effect, and that subjective awareness of the COC edge is necessary.

Representation of surface luminance and contrast in primary visual cortex

In visual perception, object identification requires both the ability to define regions of uniform luminance and zones of luminance contrast. Neural processes underlying contrast detection have been well studied, while those defining luminance remain poorly understood and controversial. Partially because stimuli comprised of uniform luminance are relatively ineffective in driving responses of cortical neurons, little effort has been made to systematically compare responses of individual neurons to both uniform luminance and contrast. Using large static uniform luminance and contrast stimuli, modulated temporally in luminance or contrast, we found a continuum of responses ranging from a few cells modulated only by luminance (luminance-only), to many cells modulated by both luminance and contrast (luminance-contrast), and to many others modulated only by contrast (contrast-only) in primary visual cortex. Moreover, luminance-contrast cells had broader orientation tuning, larger receptive field (RF) and lower spatial frequency Preference, on average, than contrast-only cells. Contrast-only cells had contrast responses more linearly correlated to the spatial structure of their RFs than luminance-contrast cells. Taken together these results suggest that luminance and contrast are represented, to some degree, by independent mechanisms that may be shaped by different classes of subcortical and/or cortical inputs.

A motion direction map in macaque V2

In mammals, the perception of motion starts with direction-selective neurons in the visual cortex. Despite numerous studies in monkey primary and second visual cortex (V1 and V2), there has been no evidence of direction maps in these areas. In the present study, we used optical imaging methods to study the organization of motion response in macaque V1 and V2. In contrast to the findings in other mammals (e.g., cats and ferrets), we found no direction maps in macaque V1. Robust direction maps, however, were found in V2 thick/pale stripes and avoided thin stripes. In many cases direction maps were located within thick stripes and exhibited pinwheel or linear organizations. The presence of motion maps in V2 points to a newfound prominence of V2 in motion processing, for contributing to motion perception in the dorsal pathway and/or for motion cue-dependent form perception in the ventral pathway.

Visual pathways and psychophysical channels in the primate

The main cell systems of the retina that provide input to the striate cortex are now well described, although certain aspects of their anatomy and physiology remain contentious. Under simple stimulus conditions and in a threshold context psychophysical performance can often be assigned to one or other of these systems, and an identification of psychophysical channels with afferent pathways is justifiable. However, results from psychophysical studies using more complex stimulus conditions are more difficult to relate to 'front end' channels, and it is more difficult to separate the physiological contributions of afferent pathways from those of cortical mechanisms, in particular the separation of dorsal and ventral streams.

A new taxonomy for perceptual filling-in

Perceptual filling-in occurs when structures of the visual system interpolate information across regions of visual space where that information is physically absent. It is a ubiquitous and heterogeneous phenomenon, which takes place in different forms almost every time we view the world around us, such as when objects are occluded by other objects or when they fall behind the blind spot. Yet, to date, there is no clear framework for relating these various forms of perceptual filling-in. Similarly, whether these and other forms of filling-in share common mechanisms is not yet known. Here we present a new taxonomy to categorize the different forms of perceptual filling-in. We then examine experimental evidence for the processes involved in each type of perceptual filling-in. Finally, we use established theories of general surface perception to show how contextualizing filling-in using this framework broadens our understanding of the possible shared mechanisms underlying perceptual filling-in. In particular, we consider the importance of the presence of boundaries in determining the phenomenal experience of perceptual filling-in.

Lightness, brightness and transparency: a quarter century of new ideas, captivating demonstrations and unrelenting controversy

The past quarter century has witnessed considerable advances in our understanding of Lightness (perceived reflectance), Brightness (perceived luminance) and perceived Transparency (LBT). This review poses eight major conceptual questions that have engaged researchers during this period, and considers to what extent they have been answered. The questions concern 1. the relationship between lightness, brightness and perceived non-uniform illumination, 2. the brain site for lightness and brightness perception, 3 the effects of context on lightness and brightness, 4. the relationship between brightness and contrast for simple patch-background stimuli, 5. brightness "filling-in", 6. lightness anchoring, 7. the conditions for perceptual transparency, and 8. the perceptual representation of transparency. The discussion of progress on major conceptual questions inevitably requires an evaluation of which approaches to LBT are likely and which are unlikely to bear fruit in the long term, and which issues remain unresolved. It is concluded that the most promising developments in LBT are (a) models of brightness coding based on multi-scale filtering combined with contrast normalization, (b) the idea that the visual system decomposes the image into "layers" of reflectance, illumination and transparency, (c) that an understanding of image statistics is important to an understanding of lightness errors, (d) Whittle's logW metric for contrast-brightness, (e) the idea that "filling-in" is mediated by low spatial frequencies rather than neural spreading, and (f) that there exist multiple cues for identifying non-uniform illumination and transparency. Unresolved issues include how relative lightness values are anchored to produce absolute lightness values, and the perceptual representation of transparency. Bridging the gap between multi-scale filtering and layer decomposition approaches to LBT is a major task for future research.

The development of contour interpolation: evidence from subjective contours

Adults are skilled at perceiving subjective contours in regions without any local image information (e.g., Ginsburg, 1975; Kanizsa, 1976). Here we examined the development of this skill and the effect thereon of the support ratio (i.e., the ratio of the physically specified contours to the total contour length). Children (6-, 9-, and 12-year-olds) and adults discriminated between fat and skinny shapes formed by subjective or luminance-defined contours. By 9 years of age, children were as sensitive as adults to small differences in luminance-defined contours, but not until 12 years of age were children as sensitive as adults in performing the same task with subjective contours. Remarkably, 6-year-olds' sensitivity to subjective contours was independent of the support ratio, unlike that of older children and adults. The results suggest that, during middle childhood, the interpolation of subjective contours becomes tied to the support ratio, so that contours that are more likely to reflect the contours of real objects (i.e., highly supported contours) are more easily interpolated.

Simple cell response properties imply receptive field structure: balanced Gabor and/or bandlimited field functions

The classical receptive fields of simple cells in mammalian primary visual cortex demonstrate three cardinal response properties: (1) they do not respond to stimuli that are spatially homogeneous; (2) they respond best to stimuli in a preferred orientation (direction); and (3) they do not respond to stimuli in other, nonpreferred orientations (directions). We refer to these as the balanced field property, the maximum response direction property, and the zero response direction property, respectively. These empirically determined response properties are used to derive a complete characterization of elementary receptive field functions defined as products of a circularly symmetric weight function and a simple periodic carrier. Two disjoint classes of elementary receptive field functions result: the balanced Gabor class, a generalization of the traditional Gabor filter, and a bandlimited class whose Fourier transforms have compact support (i.e., are zero valued outside of a bounded range). The detailed specification of these two classes of receptive field functions from empirically based postulates may prove useful to neurophysiologists seeking to test alternative theories of simple cell receptive field structure and to computational neuroscientists seeking basis functions with which to model human vision.