The primary visual cortex fills in color

Neural correlates of dynamic lightness induction

The lightness of a surface depends not only on the amount of light reflected off, it but also on the context in which it is embedded. Despite a long history of research, neural correlates of context-dependent lightness perception remain a topic of ongoing debate. Here, we seek to expand on the existing literature by measuring functional magnetic resonance imaging (fMRI) responses to lightness variations induced by the context. During the fMRI experiment, we presented 10 participants with a dynamic stimulus in which either the luminance of a disk or its surround is modulated at four different frequencies ranging from 1 to 8 Hz. Behaviorally, when the surround luminance is modulated at low frequencies, participants perceive an illusory change in the lightness of the disk (lightness induction). In contrast, they perceive little or no induction at higher frequencies. Using this frequency dependence and controlling for long-range responses to border contrast and luminance changes, we found that activity in the primary visual cortex (V1) correlates with lightness induction, providing further evidence for the involvement of V1 in the processing of context-dependent lightness.

Brightness illusions drive a neuronal response in the primary visual cortex under top-down modulation

Brightness illusions are a powerful tool in studying vision, yet their neural correlates are poorly understood. Based on a human paradigm, we presented illusory drifting gratings to mice. Primary visual cortex (V1) neurons responded to illusory gratings, matching their direction selectivity for real gratings, and they tracked the spatial phase offset between illusory and real gratings. Illusion responses were delayed compared to real gratings, in line with the theory that processing illusions requires feedback from higher visual areas (HVAs). We provide support for this theory by showing a reduced V1 response to illusions, but not real gratings, following HVAs optogenetic inhibition. Finally, we used the pupil response (PR) as an indirect perceptual report and showed that the mouse PR matches the human PR to perceived luminance changes. Our findings resolve debates over whether V1 neurons are involved in processing illusions and highlight the involvement of feedback from HVAs.

Cortical depth profiles in primary visual cortex for illusory and imaginary experiences

Visual illusions and mental imagery are non-physical sensory experiences that involve cortical feedback processing in the primary visual cortex. Using laminar functional magnetic resonance imaging (fMRI) in two studies, we investigate if information about these internal experiences is visible in the activation patterns of different layers of primary visual cortex (V1). We find that imagery content is decodable mainly from deep layers of V1, whereas seemingly 'real' illusory content is decodable mainly from superficial layers. Furthermore, illusory content shares information with perceptual content, whilst imagery content does not generalise to illusory or perceptual information. Together, our results suggest that illusions and imagery, which differ immensely in their subjective experiences, also involve partially distinct early visual microcircuits. However, overlapping microcircuit recruitment might emerge based on the nuanced nature of subjective conscious experience.

Reconstructing visual illusory experiences from human brain activity

Visual illusions provide valuable insights into the brain's interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain's internal representations of the world.

Computational modeling of color perception with biologically plausible spiking neural networks

Biologically plausible computational modeling of visual perception has the potential to link high-level visual experiences to their underlying neurons' spiking dynamic. In this work, we propose a neuromorphic (brain-inspired) Spiking Neural Network (SNN)-driven model for the reconstruction of colorful images from retinal inputs. We compared our results to experimentally obtained V1 neuronal activity maps in a macaque monkey using voltage-sensitive dye imaging and used the model to demonstrate and critically explore color constancy, color assimilation, and ambiguous color perception. Our parametric implementation allows critical evaluation of visual phenomena in a single biologically plausible computational framework. It uses a parametrized combination of high and low pass image filtering and SNN-based filling-in Poisson processes to provide adequate color image perception while accounting for differences in individual perception.

Neural correlates of lateral modulation and perceptual filling-in in center-surround radial sinusoidal gratings: an fMRI study

We investigated lateral modulation effects with functional magnetic resonance imaging. We presented radial sinusoidal gratings in random sequence: a scotoma grating with two arc-shaped blank regions (scotomata) in the periphery, one in the left and one in the right visual field, a center grating containing pattern only in the scotoma regions, and a full-field grating where the pattern occupied the whole screen. On each trial, one of the three gratings flickered in counterphase for 10 s, followed by a blank period. Observers were instructed to perform a fixation task and report whether filling-in was experienced during the scotoma condition. The results showed that the blood-oxygen-level-dependent signal was reduced in areas corresponding to the scotoma regions in the full-field compared to the center condition in V1 to V3 areas, indicating a lateral inhibition effect when the surround was added to the center pattern. The univariate analysis results showed no difference between the filling-in and no-filling-in trials. However, multivariate pattern analysis results showed that classifiers trained on activation pattern in V1 to V3 could differentiate between filling-in and no-filling-in trials, suggesting that the neural activation pattern in visual cortex correlated with the subjective percept.

Attention impedes neural representation of interpolated orientation during perceptual completion

One of the most remarkable functional feats accomplished by visual system is the interpolation of missing retinal inputs based on surrounding information, a process known as perceptual completion. Perceptual completion enables the active construction of coherent, vivid percepts from spatially discontinuous visual information that is prevalent in real-life visual scenes. Despite mounting evidence linking sensory activity enhancement and perceptual completion, surprisingly little is known about whether and how attention, a fundamental modulator of sensory activities, affects perceptual completion. Using EEG-based time-resolved inverted encoding model (IEM), we reconstructed the moment-to-moment representation of the illusory grating that resulted from spatially interpolating the orientation of surrounding inducers. We found that, despite manipulation of observers' attentional focus, the illusory grating representation unfolded in time in a similar manner. Critically, attention to the surrounding inducers simultaneously attenuated the illusory grating representation and delayed its temporal development. Our findings disclosed, for the first time, the suppressive role of selective attention in perceptual completion and were suggestive of a fast, automatic neural machinery that implements the interpolation of missing visual information.

Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey

Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance.

How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.

Negative affect impedes perceptual filling-in in the uniformity illusion

The notion of cognitive penetrability, i.e., whether perceptual contents can in principle be influenced by non-perceptual factors, has sparked a significant debate over methodological concerns and the correct interpretation of existing findings. In this study, we combined predictive processing models of visual perception and affective states to investigate influences of affective valence on perceptual filling-in in extrafoveal vision. We tested how experimentally induced affect would influence the probability of perceptual filling-in occurring in the uniformity illusion (N = 50). Negative affect led to reduced occurrence rates and increased onset times of visual uniformity. This effect was selectively observed in illusionary trials, requiring perceptual filling-in, and not in control trials, where uniformity was the veridical percept, ruling out biased motor responses or deliberate judgments as confounding variables. This suggests an influential role of affective status on subsequent perceptual processing, specifically on how much weight is ascribed to priors as opposed to sensory evidence.

Visual Cortex Transcranial Direct Current Stimulation for Proliferative Diabetic Retinopathy Patients: A Double-Blinded Randomized Exploratory Trial

Proliferative diabetic retinopathy (PDR) is a severe complication of diabetes. PDR-related retinal hemorrhages often lead to severe vision loss. The main goals of management are to prevent visual impairment progression and improve residual vision. We explored the potential of transcranial direct current stimulation (tDCS) to enhance residual vision. tDCS applied to the primary visual cortex (V1) may improve visual input processing from PDR patients’ retinas. Eleven PDR patients received cathodal tDCS stimulation of V1 (1 mA for 10 min), and another eleven patients received sham stimulation (1 mA for 30 s). Visual acuity (logarithm of the minimum angle of resolution (LogMAR) scores) and number acuity (reaction times (RTs) and accuracy rates (ARs)) were measured before and immediately after stimulation. The LogMAR scores and the RTs of patients who received cathodal tDCS decreased significantly after stimulation. Cathodal tDCS has no significant effect on ARs. There were no significant changes in the LogMAR scores, RTs, and ARs of PDR patients who received sham stimulation. The results are compatible with our proposal that neuronal noise aggravates impaired visual function in PDR. The therapeutic effect indicates the potential of tDCS as a safe and effective vision rehabilitation tool for PDR patients.

Stimulus blanking reveals contrast-dependent transsaccadic feature transfer

Across saccadic eye movements, the visual system receives two successive static images corresponding to the pre- and the postsaccadic projections of the visual field on the retina. The existence of a mechanism integrating the content of these images is today still a matter of debate. Here, we studied the transfer of a visual feature across saccades using a blanking paradigm. Participants moved their eyes to a peripheral grating and discriminated a change in its orientation occurring during the eye movement. The grating was either constantly on the screen or briefly blanked during and after the saccade. Moreover, it either was of the same luminance as the background (i.e., isoluminant) or anisoluminant with respect to it. We found that for anisoluminant gratings, the orientation discrimination across saccades was improved when a blank followed the onset of the eye movement. Such effect was however abolished with isoluminant gratings. Additionally, performance was also improved when an anisoluminant grating presented before the saccade was followed by an isoluminant one. These results demonstrate that a detailed representation of the presaccadic image was transferred across saccades allowing participants to perform better on the transsaccadic orientation task. While such a transfer of visual orientation across saccade is masked in real-life anisoluminant conditions, the use of a blank and of an isoluminant postsaccadic grating allowed to reveal its existence.

Thalamocortical inhibitory dynamics support conscious perception

Whether thalamocortical interactions play a decisive role in conscious perception remains an open question. We presented rapid red/green color flickering stimuli, which induced the mental perception of either an illusory orange color or non-fused red and green colors. Using magnetoencephalography, we observed 6-Hz thalamic activity associated with thalamocortical inhibitory coupling only during the conscious perception of the illusory orange color. This sustained thalamic disinhibition was temporally coupled with higher visual cortical activation during the conscious perception of the orange color, providing neurophysiological evidence of the role of thalamocortical synchronization in conscious awareness of mental representation. Bayesian model comparison consistently supported the thalamocortical model in conscious perception. Taken together, experimental and theoretical evidence established the thalamocortical inhibitory network as a gateway to conscious mental representations.

Lateral modulation of orientation perception in center-surround sinusoidal stimuli: Divisive inhibition in perceptual filling-in

The perception of a target stimulus may be altered by its context. Perceptual filling-in is thought to be one example of lateral modulation, in which the percept of a central blank area is replaced by that of the surround. We investigated the mechanisms in eccentric vision underlying filling-in by selectively adapting the center (pedestal adapter), surround (annulus adapter), or both (disk adapter) in a sinusoidal grating and observed how the adaptation influences the orientation percept of a subsequently presented Gabor target, located at the same position as the adapter center. In a binary choice task, observers were to judge the orientation (clockwise or counterclockwise) of the target after adaptation. The tilt aftereffect (TAE), corresponding to an illusory tilt of a physically vertical Gabor target, depended both on the adapter orientation and the adapter type. The TAE, peaked between 10 degrees and 20 degrees adapter orientation, was strongest in the pedestal, followed by the disk, and weakest in the annulus adapter conditions. The difference between the disk and pedestal conditions implies lateral inhibition from the surround. Lacking physical overlap with the target, the annulus adapter nonetheless induced a small but significant TAE in the central area. The effect of filling-in on the TAE was estimated by comparing the results from trials with and without subjectively reported filling-in during adaptation to the annulus adapter. The TAE was greater when filling-in occurred during adaptation, suggesting a stronger lateral modulation effect on trials where filling-in was induced. The data were fit by a variant of a divisive inhibition model, in which the adaptation effect is captured by the increase of an additive constant in the denominator of the response function, whereas the surround modulation in the adapter is modeled by an excitatory sensitivity in the numerator.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Spatiotopic and saccade-specific transsaccadic memory for object detail

The content and nature of transsaccadic memory are still a matter of debate. Brief postsaccadic target blanking was demonstrated to recover transsaccadic memory and defeat saccadic suppression of displacement. We examined whether blanking would also support transsaccadic transfer of detailed form information. Observers saccaded to a peripheral, checkerboard-like stimulus and reported whether an intrasaccadic change had occurred in its upper or lower half. On half of the trials, the stimulus was blanked for 200 ms with saccade onset. In a fixation condition, observers kept fixation but the stimulus was displaced from periphery to fixation, mimicking the retinal events of the saccade condition. Results show that stimulus blanking improves transsaccadic change detection, with performance being far superior to the retinally equivalent fixation condition. Our findings argue in favor of a remapped memory trace that can be accessed only in the blanking condition, when not being overwritten by the salient postsaccadic stimulus.

Feedback contribution to surface motion perception in the human early visual cortex

Human visual surface perception has neural correlates in early visual cortex, but the role of feedback during surface segmentation in human early visual cortex remains unknown. Feedback projections preferentially enter superficial and deep anatomical layers, which provides a hypothesis for the cortical depth distribution of fMRI activity related to feedback. Using ultra-high field fMRI, we report a depth distribution of activation in line with feedback during the (illusory) perception of surface motion. Our results fit with a signal re-entering in superficial depths of V1, followed by a feedforward sweep of the re-entered information through V2 and V3. The magnitude and sign of the BOLD response strongly depended on the presence of texture in the background, and was additionally modulated by the presence of illusory motion perception compatible with feedback. In summary, the present study demonstrates the potential of depth-resolved fMRI in tackling biomechanical questions on perception.

Resolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling

The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N = 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENT A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.

Reduced Fading of Visual Afterimages after Transcranial Magnetic Stimulation over Early Visual Cortex

In the complete absence of small transients in visual inputs (e.g., by experimentally stabilizing an image on the retina or in everyday life during intent staring), information perceived by the eyes will fade from the perceptual experience. Although the mechanisms of visual fading remain poorly understood, one possibility is that higher level brain regions actively suppress the stable visual signals via targeted feedback onto early visual cortex (EVC). Here, we used positive afterimages and multisensory conflict to induce gestalt-like fading of participants' own hands. In two separate experiments, participants rated the perceived quality of their hands both before and after transcranial magnetic stimulation (TMS) was applied over EVC. In a first experiment, triple-pulse TMS was able to make a faded hand appear less faded after the pulses were applied, compared with placebo pulses. A second experiment demonstrated that this was because triple-pulse TMS slowed down fading of the removed hand that otherwise occurs naturally over time. Interestingly, TMS similarly affected the left and right hands, despite being applied only over the right EVC. Together, our results suggest that TMS over EVC attenuates the effects of visual fading in positive afterimages, and it might do so by crossing transcollosal connections or via multimodal integration sites in which both hands are represented.

Neural circuits for long-range color filling-in

Surface color appearance depends on both local surface chromaticity and global context. How are these inter-dependencies supported by cortical networks? Combining functional imaging and psychophysics, we examined if color from long-range filling-in engages distinct pathways from responses caused by a field of uniform chromaticity. We find that color from filling-in is best classified and best correlated with appearance by two dorsal areas, V3A and V3B/KO. In contrast, a field of uniform chromaticity is best classified by ventral areas hV4 and LO. Dynamic causal modeling revealed feedback modulation from area V3A to areas V1 and LO for filling-in, contrasting with feedback from LO modulating areas V1 and V3A for a matched uniform chromaticity. These results indicate a dorsal stream role in color filling-in via feedback modulation of area V1 coupled with a cross-stream modulation of ventral areas suggesting that local and contextual influences on color appearance engage distinct neural networks.

Neural representation of form-contingent color filling-in in the early visual cortex

Perceptual filling-in exemplifies the constructive nature of visual processing. Color, a prominent surface property of visual objects, can appear to spread to neighboring areas that lack any color. We investigated cortical responses to a color filling-in illusion that effectively dissociates perceived color from the retinal input (van Lier, Vergeer, & Anstis, 2009). Observers adapted to a star-shaped stimulus with alternating red- and cyan-colored points to elicit a complementary afterimage. By presenting an achromatic outline that enclosed one of the two afterimage colors, perceptual filling-in of that color was induced in the unadapted central region. Visual cortical activity was monitored with fMRI, and analyzed using multivariate pattern analysis. Activity patterns in early visual areas (V1–V4) reliably distinguished between the two color-induced filled-in conditions, but only higher extrastriate visual areas showed the predicted correspondence with color perception. Activity patterns allowed for reliable generalization between filled-in colors and physical presentations of perceptually matched colors in areas V3 and V4, but not in earlier visual areas. These findings suggest that the perception of filled-in surface color likely requires more extensive processing by extrastriate visual areas, in order for the neural representation of surface color to become aligned with perceptually matched real colors.

Adaptive and maladaptive neural compensatory consequences of sensory deprivation-From a phantom percept perspective

It is suggested that the brain undergoes plastic changes in order to adapt to changing environmental needs. Sensory deprivation results in decreased input to the brain leading to adaptive or maladaptive changes. Although several theories hypothesize the mechanism of these adaptive and maladaptive changes, the course of action taken by the brain heavily depends on the age of incidence of damage. The growing body of literature on the topic proposes that maladaptive changes in the brain are instrumental in creating phantom percepts, defined as the perception of a sensory experience in the absence of a physical stimulus. The current article reviews the mechanisms of adaptive and maladaptive plasticity in the brain in congenital, early, and late-onset sensory deprivation in conjunction with the phantom percepts in the different sensory domains. We propose that the mechanisms of adaptive and maladaptive plasticity fall under a universal construct of updating hierarchical Bayesian prediction errors. This theory of the Bayesian brain hypothesizes that the brain constantly compares its internal milieu with changing environmental cues and either adjusts its predictions or discards the change, depending on the novelty or salience of the external stimulus. We propose that adaptive plasticity reflects both successful bottom-up compensation and top-down updating of the model while maladaptive plasticity reflects failure in one or both mechanisms, resulting in a constant prediction-error. Finally, we hypothesize that phantom percepts are generated by the brain as a solution to this prediction error and are thus a manifestation of unsuccessful adaptation to sensory deprivation.

Early suppression effect in human primary visual cortex during Kanizsa illusion processing: A magnetoencephalographic evidence

Detection of illusory contours (ICs) such as Kanizsa figures is known to depend primarily upon the lateral occipital complex. Yet there is no universal agreement on the role of the primary visual cortex in this process; some existing evidence hints that an early stage of the visual response in V1 may involve relative suppression to Kanizsa figures compared with controls. Iso-oriented luminance borders, which are responsible for Kanizsa illusion, may evoke surround suppression in V1 and adjacent areas leading to the reduction in the initial response to Kanizsa figures. We attempted to test the existence, as well as to find localization and timing of the early suppression effect produced by Kanizsa figures in adult nonclinical human participants. We used two sizes of visual stimuli (4.5 and 9.0°) in order to probe the effect at two different levels of eccentricity; the stimuli were presented centrally in passive viewing conditions. We recorded magnetoencephalogram, which is more sensitive than electroencephalogram to activity originating from V1 and V2 areas. We restricted our analysis to the medial occipital area and the occipital pole, and to a 40-120 ms time window after the stimulus onset. By applying threshold-free cluster enhancement technique in combination with permutation statistics, we were able to detect the inverted IC effect-a relative suppression of the response to the Kanizsa figures compared with the control stimuli. The current finding is highly compatible with the explanation involving surround suppression evoked by iso-oriented collinear borders. The effect may be related to the principle of sparse coding, according to which V1 suppresses representations of inner parts of collinear assemblies as being informationally redundant. Such a mechanism is likely to be an important preliminary step preceding object contour detection.

Reconstructing representations of dynamic visual objects in early visual cortex

As raw sensory data are partial, our visual system extensively fills in missing details, creating enriched percepts based on incomplete bottom-up information. Despite evidence for internally generated representations at early stages of cortical processing, it is not known whether these representations include missing information of dynamically transforming objects. Long-range apparent motion (AM) provides a unique test case because objects in AM can undergo changes both in position and in features. Using fMRI and encoding methods, we found that the "intermediate" orientation of an apparently rotating grating, never presented in the retinal input but interpolated during AM, is reconstructed in population-level, feature-selective tuning responses in the region of early visual cortex (V1) that corresponds to the retinotopic location of the AM path. This neural representation is absent when AM inducers are presented simultaneously and when AM is visually imagined. Our results demonstrate dynamic filling-in in V1 for object features that are interpolated during kinetic transformations.

Representation of Color Surfaces in V1: Edge Enhancement and Unfilled Holes

The neuronal mechanism underlying the representation of color surfaces in primary visual cortex (V1) is not well understood. We tested on color surfaces the previously proposed hypothesis that visual perception of uniform surfaces is mediated by an isomorphic, filled-in representation in V1. We used voltage-sensitive-dye imaging in fixating macaque monkeys to measure V1 population responses to spatially uniform chromatic (red, green, or blue) and achromatic (black or white) squares of different sizes (0.5°-8°) presented for 300 ms. Responses to both color and luminance squares early after stimulus onset were similarly edge-enhanced: for squares 1° and larger, regions corresponding to edges were activated much more than those corresponding to the center. At later times after stimulus onset, responses to achromatic squares' centers increased, partially "filling-in" the V1 representation of the center. The rising phase of the center response was slower for larger squares. Surprisingly, the responses to color squares behaved differently. For color squares of all sizes, responses remained edge-enhanced throughout the stimulus. There was no filling-in of the center. Our results imply that uniform filled-in representations of surfaces in V1 are not required for the perception of uniform surfaces and that chromatic and achromatic squares are represented differently in V1.

SIGNIFICANCE STATEMENT

We used voltage-sensitive dye imaging from V1 of behaving monkeys to test the hypothesis that visual perception of uniform surfaces is mediated by an isomorphic, filled-in representation. We found that the early population responses to chromatic and achromatic surfaces are edge enhanced, emphasizing the importance of edges in surface processing. Next, we show for color surfaces that responses remained edge-enhanced throughout the stimulus presentation whereas response to luminance surfaces showed a slow neuronal 'filling-in' of the center. Our results suggest that isomorphic representation is not a general code for uniform surfaces in V1.

The physiology and psychophysics of the color-form relationship: a review

The relationship between color and form has been a long standing issue in visual science. A picture of functional segregation and topographic clustering emerges from anatomical and electrophysiological studies in animals, as well as by brain imaging studies in human. However, one of the many roles of chromatic information is to support form perception, and in some cases it can do so in a way superior to achromatic (luminance) information. This occurs both at an early, contour-detection stage, as well as in late, higher stages involving spatial integration and the perception of global shapes. Pure chromatic contrast can also support several visual illusions related to form-perception. On the other hand, form seems a necessary prerequisite for the computation and assignment of color across space, and there are several respects in which the color of an object can be influenced by its form. Evidently, color and form are mutually dependent. Electrophysiological studies have revealed neurons in the visual brain able to signal contours determined by pure chromatic contrast, the spatial tuning of which is similar to that of neurons carrying luminance information. It seems that, especially at an early stage, form is processed by several, independent systems that interact with each other, each one having different tuning characteristics in color space. At later processing stages, mechanisms able to combine information coming from different sources emerge. A clear interaction between color and form is manifested by the fact that color-form contingencies can be observed in various perceptual phenomena such as adaptation aftereffects and illusions. Such an interaction suggests a possible early binding between these two attributes, something that has been verified by both electrophysiological and fMRI studies.

Involvement of the Extrageniculate System in the Perception of Optical Illusions: A Functional Magnetic Resonance Imaging Study

Research on the neural processing of optical illusions can provide clues for understanding the neural mechanisms underlying visual perception. Previous studies have shown that some visual areas contribute to the perception of optical illusions such as the Kanizsa triangle and Müller-Lyer figure; however, the neural mechanisms underlying the processing of these and other optical illusions have not been clearly identified. Using functional magnetic resonance imaging (fMRI), we determined which brain regions are active during the perception of optical illusions. For our study, we enrolled 18 participants. The illusory optical stimuli consisted of many kana letters, which are Japanese phonograms. During the shape task, participants stated aloud whether they perceived the shapes of two optical illusions as being the same or not. During the word task, participants read aloud the kana letters in the stimuli. A direct comparison between the shape and word tasks showed activation of the right inferior frontal gyrus, left medial frontal gyrus, and right pulvinar. It is well known that there are two visual pathways, the geniculate and extrageniculate systems, which belong to the higher-level and primary visual systems, respectively. The pulvinar belongs to the latter system, and the findings of the present study suggest that the extrageniculate system is involved in the cognitive processing of optical illusions.

Shape perception simultaneously up- and downregulates neural activity in the primary visual cortex

An essential part of visual perception is the grouping of local elements (such as edges and lines) into coherent shapes. Previous studies have shown that this grouping process modulates neural activity in the primary visual cortex (V1) that is signaling the local elements [1-4]. However, the nature of this modulation is controversial. Some studies find that shape perception reduces neural activity in V1 [2, 5, 6], while others report increased V1 activity during shape perception [1, 3, 4, 7-10]. Neurocomputational theories that cast perception as a generative process [11-13] propose that feedback connections carry predictions (i.e., the generative model), while feedforward connections signal the mismatch between top-down predictions and bottom-up inputs. Within this framework, the effect of feedback on early visual cortex may be either enhancing or suppressive, depending on whether the feedback signal is met by congruent bottom-up input. Here, we tested this hypothesis by quantifying the spatial profile of neural activity in V1 during the perception of illusory shapes using population receptive field mapping. We find that shape perception concurrently increases neural activity in regions of V1 that have a receptive field on the shape but do not receive bottom-up input and suppresses activity in regions of V1 that receive bottom-up input that is predicted by the shape. These effects were not modulated by task requirements. Together, these findings suggest that shape perception changes lower-order sensory representations in a highly specific and automatic manner, in line with theories that cast perception in terms of hierarchical generative models.

Attentional modulations of the early and later stages of the neural processing of visual completion

The brain effortlessly recognizes objects even when the visual information belonging to an object is widely separated, as well demonstrated by the Kanizsa-type illusory contours (ICs), in which a contour is perceived despite the fragments of the contour being separated by gaps. Such large-range visual completion has long been thought to be preattentive, whereas its dependence on top-down influences remains unclear. Here, we report separate modulations by spatial attention and task relevance on the neural activities in response to the ICs. IC-sensitive event-related potentials that were localized to the lateral occipital cortex were modulated by spatial attention at an early processing stage (130–166 ms after stimulus onset) and modulated by task relevance at a later processing stage (234–290 ms). These results not only demonstrate top-down attentional influences on the neural processing of ICs but also elucidate the characteristics of the attentional modulations that occur in different phases of IC processing.

Asymmetrical color filling-in from the nasal to the temporal side of the blind spot

The physiological blind spot, corresponding to the optic disk in the retina, is a relatively large (6 × 8°) area in the visual field that receives no retinal input. However, we rarely notice the existence of it in daily life. This is because the blind spot fills in with the brightness, color, texture, and motion of the surround. The study of filling-in enables us to better understand the creative nature of the visual system, which generates perceptual information where there is none. Is there any retinotopic rule in the color filling-in of the blind spot? To find out, we used mono-colored and bi-colored annuli hugging the boundary of the blind spot. We found that mono-colored annuli filled in the blind spot uniformly. By contrast, bi-colored annuli, where one half had a given color, while the other half had a different one, filled in the blind spot asymmetrically. Specifically, the color surrounding the nasal half typically filled in about 75% of the blind spot area, whereas the color surrounding the temporal half filled in only about 25%. This asymmetry was dependent on the relative size of the half rings, but not the two colors used, and was absent when the bi-colored annulus was rotated by 90°. Here, the two colors on the upper and lower sides of the blind spot filled in the enclosed area equally. These results suggest that the strength of filling-in decreases with distance from the fovea consistent with the decrease of the cortical magnification factor.

Topographic representation of an occluded object and the effects of spatiotemporal context in human early visual areas

Occlusion is a primary challenge facing the visual system in perceiving object shapes in intricate natural scenes. Although behavior, neurophysiological, and modeling studies have shown that occluded portions of objects may be completed at the early stage of visual processing, we have little knowledge on how and where in the human brain the completion is realized. Here, we provide functional magnetic resonance imaging (fMRI) evidence that the occluded portion of an object is indeed represented topographically in human V1 and V2. Specifically, we find the topographic cortical responses corresponding to the invisible object rotation in V1 and V2. Furthermore, by investigating neural responses for the occluded target rotation within precisely defined cortical subregions, we could dissociate the topographic neural representation of the occluded portion from other types of neural processing such as object edge processing. We further demonstrate that the early topographic representation in V1 can be modulated by prior knowledge of a whole appearance of an object obtained before partial occlusion. These findings suggest that primary "visual" area V1 has the ability to process not only visible or virtually (illusorily) perceived objects but also "invisible" portions of objects without concurrent visual sensation such as luminance enhancement to these portions. The results also suggest that low-level image features and higher preceding cognitive context are integrated into a unified topographic representation of occluded portion in early areas.

Perception of elasticity in the kinetic illusory object with phase differences in inducer motion

BACKGROUND

It is known that subjective contours are perceived even when a figure involves motion. However, whether this includes the perception of rigidity or deformation of an illusory surface remains unknown. In particular, since most visual stimuli used in previous studies were generated in order to induce illusory rigid objects, the potential perception of material properties such as rigidity or elasticity in these illusory surfaces has not been examined. Here, we elucidate whether the magnitude of phase difference in oscillation influences the visual impressions of an object's elasticity (Experiment 1) and identify whether such elasticity perceptions are accompanied by the shape of the subjective contours, which can be assumed to be strongly correlated with the perception of rigidity (Experiment 2).

METHODOLOGY/PRINCIPAL FINDINGS

In Experiment 1, the phase differences in the oscillating motion of inducers were controlled to investigate whether they influenced the visual impression of an illusory object's elasticity. The results demonstrated that the impression of the elasticity of an illusory surface with subjective contours was systematically flipped with the degree of phase difference. In Experiment 2, we examined whether the subjective contours of a perceived object appeared linear or curved using multi-dimensional scaling analysis. The results indicated that the contours of a moving illusory object were perceived as more curved than linear in all phase-difference conditions.

CONCLUSIONS/SIGNIFICANCE

These findings suggest that the phase difference in an object's motion is a significant factor in the material perception of motion-related elasticity.

Contrast magnitude and polarity effects on color filling-in along cardinal color axes

Color filling-in is the phenomenon in which the color of a visual area is perceived as the color that is only presented in an adjacent area. In a stimulus with multiple edges, color filling-in can occur along any edge and in both centripetal and centrifugal directions when maintaining steady fixation. The current study aimed to investigate the role of chromatic contrast magnitude and polarity along the two chromaticity cardinal axes and the interaction of the axes in the color filling-in process. In Experiment 1, the color filling-in process was examined using stimuli with three different regions and two edges. The three regions had chromaticities that varied only in one of the chromaticity axes. In Experiment 2, the regions along both edges differed in chromaticity along both axes. The results showed that the contrast magnitudes and polarity relationship of the two edges worked together to determine the filled-in direction and time course of the filled-in percepts. Further, the results pointed to a common mechanism mediating the color filling-in process along the two cardinal axes, and the two axes did not act independently in this process.

Emergent filling in induced by motion integration reveals a high-level mechanism in filling in

The visual system is intelligent--it is capable of recovering a coherent surface from an incomplete one, a feat known as perceptual completion or filling in. Traditionally, it has been assumed that surface features are interpolated in a way that resembles the fragmented parts. Using displays featuring four circular apertures, we showed in the study reported here that a distinct completed feature (horizontal motion) arises from local ones (oblique motions)-we term this process emergent filling in. Adaptation to emergent filling-in motion generated a dynamic motion aftereffect that was not due to spreading of local motion from the isolated apertures. The filling-in motion aftereffect occurred in both modal and amodal completions, and it was modulated by selective attention. These findings highlight the importance of high-level interpolation processes in filling in and are consistent with the idea that during emergent filling in, the more cognitive-symbolic processes in later areas (e.g., the middle temporal visual area and the lateral occipital complex) provide important feedback signals to guide more isomorphic processes in earlier areas (V1 and V2).

Neural dynamics of image representation in the primary visual cortex

Horizontal connections in the primary visual cortex have been hypothesized to play a number of computational roles: association field for contour completion, surface interpolation, surround suppression, and saliency computation. Here, we argue that horizontal connections might also serve a critical role for computing the appropriate codes for image representation. That the early visual cortex or V1 explicitly represents the image we perceive has been a common assumption in computational theories of efficient coding (Olshausen and Field (1996)), yet such a framework for understanding the circuitry in V1 has not been seriously entertained in the neurophysiological community. In fact, a number of recent fMRI and neurophysiological studies cast doubt on the neural validity of such an isomorphic representation (Cornelissen et al., 2006; von der Heydt et al., 2003). In this study, we investigated, neurophysiologically, how V1 neurons respond to uniform color surfaces and show that spiking activities of neurons can be decomposed into three components: a bottom-up feedforward input, an articulation of color tuning and a contextual modulation signal that is inversely proportional to the distance away from the bounding contrast border. We demonstrate through computational simulations that the behaviors of a model for image representation are consistent with many aspects of our neural observations. We conclude that the hypothesis of isomorphic representation of images in V1 remains viable and this hypothesis suggests an additional new interpretation of the functional roles of horizontal connections in the primary visual cortex.

Feature binding across different visual dimensions

The human brain represents different kinds of visual feature dimensions in different ways. For example, surface features exhibit some properties that are very different from contour features, and some feature dimensions may be represented more extensively in either the dorsal or the ventral visual stream. Given such differences, we investigated feature binding across different feature dimensions and whether some feature dimensions might be more easily bound together than others. In Experiment 1, we looked at cross-dimension bindings for all combinations of color, orientation, and shape dimensions, while at the same time controlling for feature discriminability. Rates of correct binding, illusory conjunctions, and feature errors were equivalent in all cases. There was no bias so that some feature dimensions were easier to combine than others. In Experiment 2, we manipulated the difficulty of feature discrimination for the key, target-defining feature and the report feature. Rates of binding errors increased with difficulty of the key feature, but not with that of the report feature. The accuracy of feature discrimination could be dissociated from the accuracy of binding the feature to an object. Across both experiments, the accuracy of feature binding was independent of specific feature dimensions or perceptibility. These findings are discussed in relation to both feature integration and multiple-stage accounts of visual feature integration.

Perceptual filling-in of negative coloured afterimages

Filling-in of brightness, colour, and texture refers to the perceptual spreading of surround features onto the target area. This phenomenon has been extensively investigated in real images. Here, we study colour spreading in afterimages of disk-ring patterns, using a variety of colour combinations. We show that colour filling-in and filling-out in afterimages follow rules different from those found for real images, suggesting that the brain treats afterimages as genuine 'stimuli'.

Early Human Visual Cortex Encodes Surface Brightness Induced by Dynamic Context

Visual scene perception owes greatly to surface features such as color and brightness. Yet, early visual cortical areas predominantly encode surface boundaries rather than surface interiors. Whether human early visual cortex may nevertheless carry a small signal relevant for surface perception is a topic of debate. We induced brightness changes in a physically constant surface by temporally modulating the luminance of surrounding surfaces in seven human participants. We found that fMRI activity in the V2 representation of the constant surface was in antiphase to luminance changes of surrounding surfaces (i.e., activity was in-phase with perceived brightness changes). Moreover, the amplitude of the antiphase fMRI activity in V2 predicted the strength of illusory brightness perception. We interpret our findings as evidence for a surface-related signal in early visual cortex and discuss the neural mechanisms that may underlie that signal in concurrence with its possible interaction with the properties of the fMRI signal.

Awareness of Central Luminance Edge is Crucial for the Craik-O'Brien-Cornsweet Effect

The Craik-O'Brien-Cornsweet (COC) effect demonstrates that perceived lightness depends not only on the retinal input at corresponding visual areas but also on distal retinal inputs. In the COC effect, the central edge of an opposing pair of luminance gradients (COC edge) makes adjoining regions with identical luminance appear to be different. To investigate the underlying mechanisms of the effect, we examined whether the subjective awareness of the COC edge is necessary for the generation of the effect. We manipulated the visibility of the COC edge using visual backward masking and continuous flash suppression while monitoring subjective reports regarding online percepts and aftereffects of adaptation. Psychophysical results showed that the online percept of the COC effect nearly vanishes in conditions where the COC edge is rendered invisible. On the other hand, the results of adaptation experiments showed that the COC edge is still processed at the early stage even under the perceptual suppression. These results suggest that processing of the COC edge at the early stage is not sufficient for generating the COC effect, and that subjective awareness of the COC edge is necessary.

Illusory contours over pathological retinal scotomas

Our visual percepts are not fully determined by physical stimulus inputs. Thus, in visual illusions such as the Kanizsa figure, inducers presented at the corners allow one to perceive the bounding contours of the figure in the absence of luminance-defined borders. We examined the discrimination of the curvature of these illusory contours that pass across retinal scotomas caused by macular degeneration. In contrast with previous studies with normal-sighted subjects that showed no perception of these illusory contours in the region of physiological scotomas at the optic nerve head, we demonstrated perfect discrimination of the curvature of the illusory contours over the pathological retinal scotoma. The illusion occurred despite the large scar around the macular lesion, strongly reducing discrimination of whether the inducer openings were acute or obtuse and suggesting that the coarse information in the inducers (low spatial frequency) sufficed. The result that subjective contours can pass through the pathological retinal scotoma suggests that the visual cortex, despite the loss of bottom-up input, can use low-spatial frequency information from the inducers to form a neural representation of new complex geometrical shapes inside the scotoma.

Dissociable neural correlates of contour completion and contour representation in illusory contour perception

Object recognition occurs even when environmental information is incomplete. Illusory contours (ICs), in which a contour is perceived though the contour edges are incomplete, have been extensively studied as an example of such a visual completion phenomenon. Despite the neural activity in response to ICs in visual cortical areas from low (V1 and V2) to high (LOC: the lateral occipital cortex) levels, the details of the neural processing underlying IC perception are largely not clarified. For example, how do the visual areas function in IC perception and how do they interact to archive the coherent contour perception? IC perception involves the process of completing the local discrete contour edges (contour completion) and the process of representing the global completed contour information (contour representation). Here, functional magnetic resonance imaging was used to dissociate contour completion and contour representation by varying each in opposite directions. The results show that the neural activity was stronger to stimuli with more contour completion than to stimuli with more contour representation in V1 and V2, which was the reverse of that in the LOC. When inspecting the neural activity change across the visual pathway, the activation remained high for the stimuli with more contour completion and increased for the stimuli with more contour representation. These results suggest distinct neural correlates of contour completion and contour representation, and the possible collaboration between the two processes during IC perception, indicating a neural connection between the discrete retinal input and the coherent visual percept.

What kinds of contours bound the reach of filled-in color?

Is a retinal representation of an edge necessary to constrain the reach of color filling-in? If so, then color filling-in should not be constrained by illusory contours, because they do not exist at a retinal level. Alternatively, if color filling-in is constrained by contours at a perceptual level of neural representation, regardless of whether there is a retinal representation, then color filling-in should be constrained by illusory contours. To address this question, a variety of real luminance edges and illusory contours were presented under conditions designed to cause color filling-in. The results showed that illusory contours bounded the reach of color filling-in. A neural representation of a contour may first exist at a retinal level or a cortical level; in either case, the contour exists at a perceptual level and bounds color filling-in.

Seeing grating-textured surface begins at the border

Two experiments were conducted to reveal that the human visual system represents grating texture surface using a border-to-interior strategy. This strategy dictates that the visual system first registers the surface boundary contour and then sequentially spreads texture from the border to the interior of the image. Our experiments measured the perceived grating texture surface at various stimulus durations after the onset of a grating texture image. We found that the grating texture is initially seen near the boundary contours, with eventual spreading inward to the center of the image. To quantify the observation, the extent of the texture spreading from the boundary contour is measured as a function of the stimulus duration (30-500 ms). This allows us to analyze the texture spreading in retinal and cortical distances, based on human fMRI studies of the cortical magnification factor in cortical areas V1-V4, and to derive the spreading speed. We found that the spreading speed is constant when scaled according to the cortical distance. Similar findings are obtained no matter whether the grating texture image is presented monocularly or dichoptically, suggesting the generality of the border-to-interior strategy for representing surfaces.

Neuronal activity in the visual cortex reveals the temporal order of cognitive operations

Most mental processes consist of a number of processing steps that are executed sequentially. The timing of the individual mental operations can usually only be estimated indirectly, from the pattern of reaction times. In vision, however, many processing steps are associated with the modulation of neuronal activity in early visual areas. Here we exploited this association to elucidate the time course of neuronal activity related to each of the self-paced mental processing steps in complex visual tasks. We trained monkeys to perform two tasks, search-trace and trace-search, which required performing a sequence of two operations: a visual search for a specific color and the mental tracing of a curve. We used multielectrode recording techniques to monitor the representations of multiple visual items in area V1 at the same time and found that the relevant curve as well as the target of visual search evoked enhanced neuronal activity with a timing that depended on the order of operations. This modulation of neuronal activity in early visual areas could allow these areas to (1) act as a cognitive blackboard that permits the exchange of information between successive processing steps of a sequential visual task and to (2) contribute to the orderly progression of task-dependent endogenous attention shifts that are driven by task structure and evolve over hundreds of milliseconds.

A new taxonomy for perceptual filling-in

Perceptual filling-in occurs when structures of the visual system interpolate information across regions of visual space where that information is physically absent. It is a ubiquitous and heterogeneous phenomenon, which takes place in different forms almost every time we view the world around us, such as when objects are occluded by other objects or when they fall behind the blind spot. Yet, to date, there is no clear framework for relating these various forms of perceptual filling-in. Similarly, whether these and other forms of filling-in share common mechanisms is not yet known. Here we present a new taxonomy to categorize the different forms of perceptual filling-in. We then examine experimental evidence for the processes involved in each type of perceptual filling-in. Finally, we use established theories of general surface perception to show how contextualizing filling-in using this framework broadens our understanding of the possible shared mechanisms underlying perceptual filling-in. In particular, we consider the importance of the presence of boundaries in determining the phenomenal experience of perceptual filling-in.

"Brain-reading" of perceived colors reveals a feature mixing mechanism underlying perceptual filling-in in cortical area V1

Visual filling-in occurs when a retinally stabilized object undergoes perceptual fading. As the term "filling-in" implies, it is commonly believed that information about the apparently vanished object is lost and replaced solely by information arising from the surrounding background. Here we report multivoxel pattern analysis fMRI data that challenge this long-held belief. When subjects view blue disks on a red background while fixating, the stimulus and background appear to turn a uniform purple upon perceptual fading, suggesting that a feature mixing mechanism may underlie color filling-in. We find that ensemble fMRI signals in retinotopic visual areas reliably predict (i) which of three colors a subject reports seeing; (ii) whether a subject is in a perceptually filled-in state or not; and (iii) furthermore, while subjects are in the perceptual state of filling-in, the BOLD signal activation pattern in the sub-areas of V1 corresponding to the location of the blue disks behaves as if subjects are in fact viewing a perceptually mixed color (purple), rather than the color of the disks (blue) or the color of the background (red). These results imply that the mechanism of filling-in in stimuli in which figure and background surfaces are equated is a process of "feature mixing", not "feature replacement". These data indicate that feature mixing may involve cortical areas as early as V1.