Texture Segregation Causes Early Figure Enhancement and Later Ground Suppression in Areas V1 and V4 of Visual Cortex

Adaptation and exogenous attention interact in the early visual cortex: A TMS study

Transcranial magnetic stimulation (TMS) to early visual cortex modulates the effect of adaptation and eliminates the effect of exogenous (involuntary) attention on contrast sensitivity. Here, we investigated whether adaptation modulates exogenous attention under TMS to V1/V2. Observers performed an orientation discrimination task while attending to one of two stimuli, with or without adaptation. Following an attentional cue, two stimuli were presented in the stimulated region and its contralateral symmetric region. A response cue indicated the stimulus whose orientation observers had to discriminate. Without adaptation, in the distractor-stimulated condition, contrast sensitivity increased at the attended location and decreased at the unattended location via response gain-but these effects were eliminated in the target-stimulated condition. Critically, after adaptation, exogenous attention altered performance similarly in both distractor-stimulated and target-stimulated conditions. These results reveal that (1) adaptation and attention interact in the early visual cortex, and (2) adaptation shields exogenous attention from TMS effects.

V1 neurons are tuned to perceptual borders in natural scenes

The visual system needs to identify perceptually relevant borders to segment complex natural scenes. The primary visual cortex (V1) is thought to extract local borders, and higher visual areas are thought to identify the perceptually relevant borders between objects and the background. To test this conjecture, we used natural images that had been annotated by human observers who marked the perceptually relevant borders. We assessed the effect of perceptual relevance on V1 responses using human neuroimaging, macaque electrophysiology, and computational modeling. We report that perceptually relevant borders elicit stronger responses in the early visual cortex than irrelevant ones, even if simple features, such as contrast and the energy of oriented filters, are matched. Moreover, V1 neurons discriminate perceptually relevant borders surprisingly fast, during the early feedforward-driven activity at a latency of ~50 ms, indicating that they are tuned to the features that characterize them. We also revealed a delayed, contextual effect that enhances the V1 responses that are elicited by perceptually relevant borders at a longer latency. Our results reveal multiple mechanisms that allow V1 neurons to infer the layout of objects in natural images.

Border-ownership tuning determines the connectivity between V4 and V1 in the macaque visual system

Cortical feedback connections are extremely numerous but the logic of connectivity between higher and lower areas remains poorly understood. Feedback from higher visual areas to primary visual cortex (V1) has been shown to enhance responses on perceptual figures compared to backgrounds, an effect known as figure-background modulation (FBM). A likely source of this feedback are border-ownership (BO) selective cells in mid-tier visual areas (e.g. V4) which represent the location of figures. We examined the connectivity between V4 cells and V1 cells using noise-correlations and micro-stimulation to estimate connectivity strength. We show that connectivity is consistent with a model in which BO-tuned V4 cells send positive feedback in the direction of their preferred figure and negative feedback in the opposite direction. This connectivity scheme can recreate patterns of FBM observed in previous studies. These results provide insights into the cortical connectivity underlying figure-background perception and establish a link between FBM and BO-tuning.

Local contour features contribute to figure-ground segregation in monkey V4 neural populations and human perception

Figure-ground (FG) segregation is a crucial step towards the recognition of objects in natural scenes. Gestalt psychologists have emphasized the importance of contour features in perception of FG. Recent electrophysiological studies have identified a neural population in V4 that shows FG-dependent modulation (FG neurons). However, whether the contour features contribute to the modulation of the response patterns of the neural population remains unclear. In the present study, we quantified the contour features associated with Gestalt factors in local natural stimuli and examined whether salient contour-features evoked reliable perceptual and neural responses by analyzing response consistency (stability) across trials. The results showed the tendency that the more salient contour-features evoked the greater consistencies in the perceptual FG judgments and population-based neural responses in FG determination; a greater partial correlation for curvature and weaker correlations for closure and parallelism. Multiple linear regression analyses demonstrated that the perceptual consistency depended similarly on curvature and closure, and the neural consistency depended significantly on curvature but weakly on closure. We further observed a strong correlation between the consistencies in the perceptual and neural responses, i.e., stimuli that evoked more stable percepts tended to evoke more stable neural responses. These results indicate that local contour-features modulate the responses of the neural population in V4 and contribute to the perception of FG organization.

Population coding for figure-ground texture segregation in macaque V1 and V4

Object recognition often involves the brain segregating objects from their surroundings. Neurophysiological studies of figure-ground texture segregation have yielded inconsistent results, particularly on whether V1 neurons can perform figure-ground texture segregation or just detect texture borders. To address this issue from a population perspective, we utilized two-photon calcium imaging to simultaneously record the responses of large samples of V1 and V4 neurons to figure-ground texture stimuli in awake, fixating macaques. The average response changes indicate that V1 neurons mainly detect texture borders, while V4 neurons are involved in figure-ground segregation. However, population analysis (SVM decoding of PCA-transformed neuronal responses) reveal that V1 neurons not only detect figure-ground borders, but also contribute to figure-ground texture segregation, although requiring substantially more principal components than V4 neurons to reach a 75 % decoding accuracy. Individually, V1/V4 neurons showing larger (negative/positive) figure-ground response differences contribute more to figure-ground segregation. But for V1 neurons, the contribution becomes significant only when many principal components are considered. We conclude that V1 neurons participate in figure-ground segregation primarily by defining the figure borders, and the poorly structured figure-ground information V1 neurons carry could be further utilized by V4 neurons to accomplish figure-ground segregation.

Adaptation and exogenous attention interact in the early visual cortex: A TMS study

Transcranial magnetic stimulation (TMS) to early visual cortex modulates the effect of adaptation and eliminates the effect of exogenous (involuntary) attention on contrast sensitivity. Here we investigated whether adaptation modulates exogenous attention under TMS to V1/V2. Observers performed an orientation discrimination task while attending to one of two stimuli, with or without adaptation. Following an attentional cue, two stimuli were presented in the stimulated region and its contralateral symmetric region. A response cue indicated the stimulus whose orientation observers had to discriminate. Without adaptation, in the distractor-stimulated condition, contrast sensitivity increased at the attended location and decreased at the unattended location via response gain-but these effects were eliminated in the target-stimulated condition. Critically, after adaptation, exogenous attention altered performance similarly in both distractor-stimulated and target-stimulated conditions. These results reveal that (1) adaptation and attention interact in the early visual cortex, and (2) adaptation shields exogenous attention from TMS effects.

Top-down influence of areas 21a and 7 differently affects the surround suppression of V1 neurons in cats

Surround suppression (SS) is a phenomenon whereby a neuron's response to stimuli in its central receptive field (cRF) is suppressed by stimuli extending to its surround receptive field (sRF). Recent evidence show that top-down influence contributed to SS in the primary visual cortex (V1). However, how the top-down influence from different high-level cortical areas affects SS in V1 has not been comparatively observed. The present study applied transcranial direct current stimulation (tDCS) to modulate the neural activity in area 21a (A21a) and area 7 (A7) of cats and examined the changes in the cRF and sRF of V1 neurons. We found that anode-tDCS at A21a reduced V1 neurons' cRF size and increased their response to visual stimuli in cRF, causing an improved SS strength. By contrast, anode-tDCS at A7 increased V1 neurons' sRF size and response to stimuli in cRF, also enhancing the SS. Modeling analysis based on DoG function indicated that the increased SS of V1 neurons after anode-tDCS at A21a could be explained by a center-only mechanism, whereas the improved SS after anode-tDCS at A7 might be mediated through a combined center and surround mechanism. In conclusion, A21a and A7 may affect the SS of V1 neurons through different mechanisms.

Different patterns of foreground and background processing contribute to texture segregation in humans: an electrophysiological study

Background

Figure-ground segregation is a necessary process for accurate visual recognition. Previous neurophysiological and human brain imaging studies have suggested that foreground-background segregation relies on both enhanced foreground representation and suppressed background representation. However, in humans, it is not known when and how foreground and background processing play a role in texture segregation.

Methods

To answer this question, it is crucial to extract and dissociate the neural signals elicited by the foreground and background of a figure texture with high temporal resolution. Here, we combined an electroencephalogram (EEG) recording and a temporal response function (TRF) approach to specifically track the neural responses to the foreground and background of a figure texture from the overall EEG recordings in the luminance-tracking TRF. A uniform texture was included as a neutral condition. The texture segregation visual evoked potential (tsVEP) was calculated by subtracting the uniform TRF from the foreground and background TRFs, respectively, to index the specific segregation activity.

Results

We found that the foreground and background of a figure texture were processed differently during texture segregation. In the posterior region of the brain, we found a negative component for the foreground tsVEP in the early stage of foreground-background segregation, and two negative components for the background tsVEP in the early and late stages. In the anterior region, we found a positive component for the foreground tsVEP in the late stage, and two positive components for the background tsVEP in the early and late stages of texture processing.

Discussion

In this study we investigated the temporal profile of foreground and background processing during texture segregation in human participants at a high time resolution. The results demonstrated that the foreground and background jointly contribute to figure-ground segregation in both the early and late phases of texture processing. Our findings provide novel evidence for the neural correlates of foreground-background modulation during figure-ground segregation in humans.

The representation of occluded image regions in area V1 of monkeys and humans

Neuronal activity in the primary visual cortex (V1) is driven by feedforward input from within the neurons' receptive fields (RFs) and modulated by contextual information in regions surrounding the RF. The effect of contextual information on spiking activity occurs rapidly and is therefore challenging to dissociate from feedforward input. To address this challenge, we recorded the spiking activity of V1 neurons in monkeys viewing either natural scenes or scenes where the information in the RF was occluded, effectively removing the feedforward input. We found that V1 neurons responded rapidly and selectively to occluded scenes. V1 responses elicited by occluded stimuli could be used to decode individual scenes and could be predicted from those elicited by non-occluded images, indicating that there is an overlap between visually driven and contextual responses. We used representational similarity analysis to show that the structure of V1 representations of occluded scenes measured with electrophysiology in monkeys correlates strongly with the representations of the same scenes in humans measured with functional magnetic resonance imaging (fMRI). Our results reveal that contextual influences rapidly alter V1 spiking activity in monkeys over distances of several degrees in the visual field, carry information about individual scenes, and resemble those in human V1. VIDEO ABSTRACT.

Correlation between neural responses and human perception in figure-ground segregation

Segmentation of a natural scene into objects (figures) and background (ground) is one of crucial functions for object recognition and scene understanding. Recent studies have investigated neural mechanisms underlying figure-ground (FG) segregation and reported neural modulation to FG in the intermediate-level visual area, V4, of macaque monkeys (FG neurons). However, whether FG neurons contribute to the perception of FG segregation has not been clarified. To examine the contribution of FG neurons, we examined the correlations between perceptual consistency (PC), which quantified perceptual ambiguity in FG determination, and the reliability of neural signals in response to FG. First, we evaluated PCs for the images that were used in the previous neural recording in V4; specifically, we measured how consistently FG can be determined across trials and participants for each stimulus. The PCs were widely distributed, so that we identified the ambiguity in FG segregation for each stimulus. Next, we analyzed the correlation between the PCs and the reliability of neural modulation to FG. We found that the stimuli with higher PCs evoked more consistent and greater modulation in the responses of single neurons than those with lower PCs. Since perception is expected to show a greater correlation with responses of neural population compared to those of single neurons, we examined the correlation between the PCs and the consistency of the population responses in FG determination. Stimuli with higher PCs evoked higher population consistency than those with lower PCs. Finally, we analyzed the correlation between the PCs and neural latencies in FG modulation. We found that the stimuli with higher PCs showed shorter reaction times in FG perception and evoked shorter modulation latencies in FG neurons. These results indicate that the responses of FG neurons recorded from macaque monkeys show significant correlations with human FG perception, suggesting that V4 neurons with FG-dependent responses contribute to the perception of FG segregation.

Independent repetition suppression in macaque area V2 and inferotemporal cortex

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.NEW & NOTEWORTHY When a complexly structured natural image is presented twice in rapid succession, neurons in inferotemporal area TE exhibit repetition suppression, responding less strongly to the second than to the first presentation. We have explored whether this phenomenon is confined to high-order areas where neurons respond selectively to such images and thus carry information relevant to recognizing a repeat. We have found surprisingly that repetition suppression occurs even in low-order visual area V2.

Cognitive penetrability of scene representations based on horizontal image disparities

The structure of natural scenes is signaled by many visual cues. Principal amongst them are the binocular disparities created by the laterally separated viewpoints of the two eyes. Disparity cues are believed to be processed hierarchically, first in terms of local measurements of absolute disparity and second in terms of more global measurements of relative disparity that allow extraction of the depth structure of a scene. Psychophysical and oculomotor studies have suggested that relative disparities are particularly relevant to perception, whilst absolute disparities are not. Here, we compare neural responses to stimuli that isolate the absolute disparity cue with stimuli that contain additional relative disparity cues, using the high temporal resolution of EEG to determine the temporal order of absolute and relative disparity processing. By varying the observers' task, we assess the extent to which each cue is cognitively penetrable. We find that absolute disparity is extracted before relative disparity, and that task effects arise only at or after the extraction of relative disparity. Our results indicate a hierarchy of disparity processing stages leading to the formation of a proto-object representation upon which higher cognitive processes can act.

A microcircuit model involving parvalbumin, somatostatin, and vasoactive intestinal polypeptide inhibitory interneurons for the modulation of neuronal oscillation during visual processing

Various subtypes of inhibitory interneurons contact one another to organize cortical networks. Most cortical inhibitory interneurons express 1 of 3 genes: parvalbumin (PV), somatostatin (SOM), or vasoactive intestinal polypeptide (VIP). This diversity of inhibition allows the flexible regulation of neuronal responses within and between cortical areas. However, the exact roles of these interneuron subtypes and of excitatory pyramidal (Pyr) neurons in regulating neuronal network activity and establishing perception (via interactions between feedforward sensory and feedback attentional signals) remain largely unknown. To explore the regulatory roles of distinct neuronal types in cortical computation, we developed a computational microcircuit model with biologically plausible visual cortex layers 2/3 that combined Pyr neurons and the 3 inhibitory interneuron subtypes to generate network activity. In simulations with our model, inhibitory signals from PV and SOM neurons preferentially induced neuronal firing at gamma (30-80 Hz) and beta (20-30 Hz) frequencies, respectively, in agreement with observed physiological results. Furthermore, our model indicated that rapid inhibition from VIP to SOM subtypes underlies marked attentional modulation for low-gamma frequency (30-50 Hz) in Pyr neuron responses. Our results suggest the distinct but cooperative roles of inhibitory interneuron subtypes in the establishment of visual perception.

Figure-ground responsive fields of monkey V4 neurons estimated from natural image patches

Neurons in visual area V4 modulate their responses depending on the figure-ground (FG) organization in natural images containing a variety of shapes and textures. To clarify whether the responses depend on the extents of the figure and ground regions in and around the classical receptive fields (CRFs) of the neurons, we estimated the spatial extent of local figure and ground regions that evoked FG-dependent responses (RF-FGs) in natural images and their variants. Specifically, we applied the framework of spike triggered averaging (STA) to the combinations of neural responses and human-marked segmentation images (FG labels) that represent the extents of the figure and ground regions in the corresponding natural image stimuli. FG labels were weighted by the spike counts in response to the corresponding stimuli and averaged over. The bias due to the nonuniformity of FG labels was compensated by subtracting the ensemble average of FG labels from the weighted average. Approximately 50% of the neurons showed effective RF-FGs, and a large number exhibited structures that were similar to those observed in virtual neurons with ideal FG-dependent responses. The structures of the RF-FGs exhibited a subregion responsive to a preferred side (figure or ground) around the CRF center and a subregion responsive to a non-preferred side in the surroundings. The extents of the subregions responsive to figure were smaller than those responsive to ground in agreement with the Gestalt rule. We also estimated RF-FG by an adaptive filtering (AF) method, which does not require spherical symmetry (whiteness) in stimuli. RF-FGs estimated by AF and STA exhibited similar structures, supporting the veridicality of the proposed STA. To estimate the contribution of nonlinear processing in addition to linear processing, we estimated nonlinear RF-FGs based on the framework of spike triggered covariance (STC). The analyses of the models based on STA and STC did not show inconsiderable contribution of nonlinearity, suggesting spatial variance of FG regions. The results lead to an understanding of the neural responses that underlie the segregation of figures and the construction of surfaces in intermediate-level visual areas.

Functional recursion of orientation cues in figure-ground separation

Visual texture is an important cue to figure-ground organization. While processing of texture differences is a prerequisite for the use of this cue to extract figure-ground organization, these stages are distinct processes. One potential indicator of this distinction is the possibility that texture statistics play a different role in the figure vs. in the ground. To determine whether this is the case, we probed figure-ground processing with a family of local image statistics that specified textures that varied in the strength and spatial scale of structure, and the extent to which features are oriented. For image statistics that generated approximately isotropic textures, the threshold for identification of figure-ground structure was determined by the difference in correlation strength in figure vs. ground, independent of whether the correlations were present in figure, ground, or both. However, for image statistics with strong orientation content, thresholds were up to two times higher for correlations in the ground, vs. the figure. This held equally for texture-defined objects with convex or concave boundaries, indicating that these threshold differences are driven by border ownership, not boundary shape. Similar threshold differences were found for presentation times ranging from 125 to 500 ms. These findings identify a qualitative difference in how texture is used for figure-ground analysis, vs. texture discrimination. Additionally, it reveals a functional recursion: texture differences are needed to identify tentative boundaries and consequent scene organization into figure and ground, but then scene organization modifies sensitivity to texture differences according to the figure-ground assignment.

Mechanisms of Surround Suppression Effect on the Contrast Sensitivity of V1 Neurons in Cats

Surround suppression (SS) is a phenomenon that a neuron’s response to visual stimuli within the classical receptive field (cRF) is suppressed by a concurrent stimulation in the surrounding receptive field (sRF) beyond the cRF. Studies show that SS affects neuronal response contrast sensitivity in the primary visual cortex (V1). However, the underlying mechanisms remain unclear. Here, we examined SS effect on the contrast sensitivity of cats’ V1 neurons with different preferred SFs using external noise-masked visual stimuli and perceptual template model (PTM) analysis at the system level. The contrast sensitivity was evaluated by the inverted threshold contrast of neurons in response to circular gratings of different contrasts in the cRF with or without an annular grating in the sRF. Our results showed that SS significantly reduced the contrast sensitivity of cats’ V1 neurons. The SS-induced reduction of contrast sensitivity was not correlated with SS strength but was dependent on neuron’s preferred SF, with a larger reduction for neurons with low preferred SFs than those with high preferred SFs. PTM analysis of threshold versus external noise contrast (TvC) functions indicated that SS decreased contrast sensitivity by increasing both the internal additive noise and impact of external noise for neurons with low preferred SFs, but improving only internal additive noise for neurons with high preferred SFs. Furthermore, the SS effect on the contrast-response function of low- and high-SF neurons also exhibited different mechanisms in contrast gain and response gain. Collectively, these results suggest that the mechanisms of SS effect on neuronal contrast sensitivity may depend on neuronal populations with different SFs.

Columnar processing of border ownership in primate visual cortex

To understand a visual scene, the brain segregates figures from background by assigning borders to foreground objects. Neurons in primate visual cortex encode which object owns a border (border ownership), but the underlying circuitry is not understood. Here, we used multielectrode probes to record from border ownership-selective units in different layers in macaque visual area V4 to study the laminar organization and timing of border ownership selectivity. We find that border ownership selectivity occurs first in deep layer units, in contrast to spike latency for small stimuli in the classical receptive field. Units on the same penetration typically share the preferred side of border ownership, also across layers, similar to orientation preference. Units are often border ownership-selective for a range of border orientations, where the preferred sides of border ownership are systematically organized in visual space. Together our data reveal a columnar organization of border ownership in V4 where the earliest border ownership signals are not simply inherited from upstream areas, but computed by neurons in deep layers, and may thus be part of signals fed back to upstream cortical areas or the oculomotor system early after stimulus onset. The finding that preferred border ownership is clustered and can cover a wide range of spatially contiguous locations suggests that the asymmetric context integrated by these neurons is provided in a systematically clustered manner, possibly through corticocortical feedback and horizontal connections.

To understand a visual scene, the brain needs to identify objects and distinguish them from background. A border marks the transition from object to background, but to differentiate which side of the border belongs to the object and which to background, the brain must integrate information across space. An early signature of this computation is that brain cells signal which side of a border is ‘owned’ by an object, also known as border ownership. But how the brain computes border ownership remains unknown.

The optic nerve is a cable-like group of nerve cells that transmits information from the eye to the brain’s visual processing areas and into the visual cortex. This flow of information is often described as traveling in a feedforward direction, away from the eyes to progressively more specialized areas in the visual cortex. However, there are also numerous feedback connections in the brain, running backward from more specialized to less specialized cortical areas.

To better understand the role of these feedforward and feedback circuits in the visual processing of object borders, Franken and Reynolds made use of their stereotyped projection patterns across the cortex layers. Feedforward connections terminate in the middle layers of a cortical area, whereas feedback connections terminate in upper and lower layers. Since time is required for information to traverse the cortical layers, dissecting the timing of border ownership signals may reveal if border ownership is computed in a feedforward or feedback manner. To find out more, electrodes were used to record neural activity in the upper, middle and lower layers of the visual cortex of two rhesus monkeys as they were presented with a set of abstract scenes composed of simple shapes on a background.

This revealed that cells signaling border ownership in deep layers of the cortex did so before the signals appeared in the middle layer. This suggests that feedback rather than feedforward is required to compute border ownership. Moreover, Franken and Reynolds found evidence that cells that prefer the same side of border ownership are clustered in columns, showing how these neural circuits are organized within the visual cortex.

In summary, Franken and Reynolds found that the circuits of the primate brain that compute border ownership occur as columns, in which cells in deep layers signal border ownership first, suggesting that border ownership relies on feedback from more specialized areas. A better understanding of how feedback in the brain works to process visual information helps us appreciate what happens when these systems are impaired.

Gestalts at threshold could reveal Gestalts as predictions

We review and revisit the predictive processing inspired “Gestalts as predictions” hypothesis. The study of Gestalt phenomena at and below threshold can help clarify the role of higher-order object selective areas and feedback connections in mid-level vision. In two psychophysical experiments assessing manipulations of contrast and configurality we showed that: (1) Gestalt phenomena are robust against saliency manipulations across the psychometric function even below threshold (with the accuracy gains and higher saliency associated with Gestalts being present even around chance performance); and (2) peak differences between Gestalt and control conditions happened around the time where responses to Gestalts are starting to saturate (mimicking the differential contrast response profile of striate vs. extra-striate visual neurons). In addition, Gestalts are associated with steeper psychometric functions in all experiments. We propose that these results reflect the differential engagement of object-selective areas in Gestalt phenomena and of information- or percept-based processing, as opposed to energy- or stimulus-based processing, more generally. In addition, the presence of nonlinearities in the psychometric functions suggest differential top-down modulation of the early visual cortex. We treat this as a proof of principle study, illustrating that classic psychophysics can help assess possible involvement of hierarchical predictive processing in Gestalt phenomena.

The essential role of recurrent processing for figure-ground perception in mice

The segregation of figures from the background is an important step in visual perception. In primary visual cortex, figures evoke stronger activity than backgrounds during a delayed phase of the neuronal responses, but it is unknown how this figure-ground modulation (FGM) arises and whether it is necessary for perception. Here, we show, using optogenetic silencing in mice, that the delayed V1 response phase is necessary for figure-ground segregation. Neurons in higher visual areas also exhibit FGM and optogenetic silencing of higher areas reduced FGM in V1. In V1, figures elicited higher activity of vasoactive intestinal peptide-expressing (VIP) interneurons than the background, whereas figures suppressed somatostatin-positive interneurons, resulting in an increased activation of pyramidal cells. Optogenetic silencing of VIP neurons reduced FGM in V1, indicating that disinhibitory circuits contribute to FGM. Our results provide insight into how lower and higher areas of the visual cortex interact to shape visual perception.

Correspondence between Monkey Visual Cortices and Layers of a Saliency Map Model Based on a Deep Convolutional Neural Network for Representations of Natural Images

Attentional selection is a function that allocates the brain’s computational resources to the most important part of a visual scene at a specific moment. Saliency map models have been proposed as computational models to predict attentional selection within a spatial location. Recent saliency map models based on deep convolutional neural networks (DCNNs) exhibit the highest performance for predicting the location of attentional selection and human gaze, which reflect overt attention. Trained DCNNs potentially provide insight into the perceptual mechanisms of biological visual systems. However, the relationship between artificial and neural representations used for determining attentional selection and gaze location remains unknown. To understand the mechanism underlying saliency map models based on DCNNs and the neural system of attentional selection, we investigated the correspondence between layers of a DCNN saliency map model and monkey visual areas for natural image representations. We compared the characteristics of the responses in each layer of the model with those of the neural representation in the primary visual (V1), intermediate visual (V4), and inferior temporal (IT) cortices. Regardless of the DCNN layer level, the characteristics of the responses were consistent with that of the neural representation in V1. We found marked peaks of correspondence between V1 and the early level and higher-intermediate-level layers of the model. These results provide insight into the mechanism of the trained DCNN saliency map model and suggest that the neural representations in V1 play an important role in computing the saliency that mediates attentional selection, which supports the V1 saliency hypothesis.

Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis

Object recognition relies on different transformations of the retinal input, carried out by the visual system, that range from local contrast to object shape and category. While some of those transformations are thought to occur at specific stages of the visual hierarchy, the features they represent are correlated (e.g., object shape and identity) and selectivity for the same feature overlaps in many brain regions. This may be explained either by collinearity across representations or may instead reflect the coding of multiple dimensions by the same cortical population. Moreover, orthogonal and shared components may differently impact distinctive stages of the visual hierarchy. We recorded functional MRI activity while participants passively attended to object images and employed a statistical approach that partitioned orthogonal and shared object representations to reveal their relative impact on brain processing. Orthogonal shape representations (silhouette, curvature, and medial axis) independently explained distinct and overlapping clusters of selectivity in the occitotemporal and parietal cortex. Moreover, we show that the relevance of shared representations linearly increases moving from posterior to anterior regions. These results indicate that the visual cortex encodes shared relations between different features in a topographic fashion and that object shape is encoded along different dimensions, each representing orthogonal features.NEW & NOTEWORTHY There are several possible ways of characterizing the shape of an object. Which shape description better describes our brain responses while we passively perceive objects? Here, we employed three competing shape models to explain brain representations when viewing real objects. We found that object shape is encoded in a multidimensional fashion and thus defined by the interaction of multiple features.

Object Selection by Automatic Spreading of Top-Down Attentional Signals in V1

What is selected when attention is directed to a specific location of the visual field? Theories of object-based attention have suggested that when spatial attention is directed to part of an object, attention does not simply enhance the attended location but automatically spreads to enhance all locations that comprise the object. Here, we tested this hypothesis by reconstructing the distribution of attention from primary visual cortex (V1) population neuronal activity patterns in 24 human adults (17 female) using functional magnetic resonance imaging (fMRI) and population-based receptive field (prf) mapping. We find that attention spreads from a spatially cued location to the underlying object, and enhances all spatial locations that comprise the object. Importantly, this spreading was also evident when the object was not task relevant. These data suggest that attentional selection automatically operates at an object level, facilitating the reconstruction of coherent objects from fragmented representations in early visual cortex.SIGNIFICANCE STATEMENT Object perception is an astonishing feat of the visual system. When visual information about orientation, shape, and color enters through our eyes, it has yet to be integrated into a coherent representation of an object. But which visual features constitute a single object and which features belong to the background? The brain mechanisms underpinning object perception are yet to be understood. We now demonstrate that one candidate mechanism, the successive activation of all parts of an object, occurs in early visual cortex and results in a detailed representation of the object following Gestalt principles. Furthermore, our results suggest that object selection occurs automatically, without involving voluntary control.

A source for awareness-dependent figure-ground segregation in human prefrontal cortex

Figure-ground modulation, i.e., the enhancement of neuronal responses evoked by the figure relative to the background, has three complementary components: edge modulation (boundary detection), center modulation (region filling), and background modulation (background suppression). However, the neuronal mechanisms mediating these three modulations and how they depend on awareness remain unclear. For each modulation, we compared both the cueing effect produced in a Posner paradigm and fMRI blood oxygen-level dependent (BOLD) signal in primary visual cortex (V1) evoked by visible relative to invisible orientation-defined figures. We found that edge modulation was independent of awareness, whereas both center and background modulations were strongly modulated by awareness, with greater modulations in the visible than the invisible condition. Effective-connectivity analysis further showed that the awareness-dependent region-filling and background-suppression processes in V1 were not derived through intracortical interactions within V1, but rather by feedback from the frontal eye field (FEF) and dorsolateral prefrontal cortex (DLPFC), respectively. These results indicate a source for an awareness-dependent figure-ground segregation in human prefrontal cortex.

Population coding of figure and ground in natural image patches by V4 neurons

Segmentation of a natural scene into objects and background is a fundamental but challenging task for recognizing objects. Investigating intermediate-level visual cortical areas with a focus on local information is a crucial step towards understanding the formation of the cortical representations of figure and ground. We examined the activity of a population of macaque V4 neurons during the presentation of natural image patches and their respective variations. The natural image patches were optimized to exclude the influence of global context but included various characteristics of local stimulus. Around one fourth of the patch-responsive V4 neurons exhibited significant modulation of firing activity that was dependent on the positional relation between the figural region of the stimulus and the classical receptive field of the neuron. However, the individual neurons showed low consistency in figure-ground modulation across a variety of image patches (55–62%), indicating that individual neurons were capable of correctly signaling figure and ground only for a limited number of stimuli. We examined whether integration of the activity of multiple neurons enabled higher consistency across a variety of natural patches by training a support vector machine to classify figure and ground of the stimuli from the population firing activity. The integration of the activity of a few tens of neurons yielded discrimination accuracy much greater than that of single neurons (up to 85%), suggesting a crucial role of population coding for figure-ground discrimination in natural images.

Topographic signatures of global object perception in human visual cortex

Our visual system readily groups dynamic fragmented input into global objects. How the brain represents global object perception remains however unclear. To address this question, we recorded brain responses using functional magnetic resonance imaging whilst observers viewed a dynamic bistable stimulus that could either be perceived globally (i.e., as a grouped and coherently moving shape) or locally (i.e., as ungrouped and incoherently moving elements). We further estimated population receptive fields and used these to back-project the brain activity measured during stimulus perception into visual space via a searchlight procedure. Global perception resulted in universal suppression of responses in lower visual cortex accompanied by wide-spread enhancement in higher object-sensitive cortex. However, follow-up experiments indicated that higher object-sensitive cortex is suppressed if global perception lacks shape grouping, and that grouping-related suppression can be diffusely confined to stimulated sites and accompanied by background enhancement once stimulus size is reduced. These results speak to a non-generic involvement of higher object-sensitive cortex in perceptual grouping and point to an enhancement-suppression mechanism mediating the perception of figure and ground.

•

Lower visual cortex activity to grouped vs ungrouped dynamic stimuli is suppressed.

•

When grouping a shape, activity in higher object-sensitive cortex is enhanced.

•

Without shape grouping, activity in higher object-sensitive cortex is suppressed.

•

Grouping-related suppression can be diffusely confined to stimulated cortical sites.

Cortical State Fluctuations across Layers of V1 during Visual Spatial Perception

Many factors modulate the state of cortical activity, but the importance of cortical state variability for sensory perception remains debated. We trained mice to detect spatially localized visual stimuli and simultaneously measured local field potentials and excitatory and inhibitory neuron populations across layers of primary visual cortex (V1). Cortical states with low spontaneous firing and correlations in excitatory neurons, and suppression of 3- to 7-Hz oscillations in layer 4, accurately predicted single-trial visual detection. Our results show that cortical states exert strong effects at the initial stage of cortical processing in V1 and can play a prominent role for visual spatial behavior in mice.

Resolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling

The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N = 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENT A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.

Conscious Processing and the Global Neuronal Workspace Hypothesis

We review the central tenets and neuroanatomical basis of the global neuronal workspace (GNW) hypothesis, which attempts to account for the main scientific observations regarding the elementary mechanisms of conscious processing in the human brain. The GNW hypothesis proposes that, in the conscious state, a non-linear network ignition associated with recurrent processing amplifies and sustains a neural representation, allowing the corresponding information to be globally accessed by local processors. We examine this hypothesis in light of recent data that contrast brain activity evoked by either conscious or non-conscious contents, as well as during conscious or non-conscious states, particularly general anesthesia. We also discuss the relationship between the intertwined concepts of conscious processing, attention, and working memory.

Localization of movable electrodes in a multi-electrode microdrive in nonhuman primates

BACKGROUND

Recently, large-scale semi-chronic recording systems have been developed, unique in their capability to record simultaneously from multiple individually moveable electrodes. As these recording systems can cover a large area, knowledge of the exact location of each individual electrode is crucial. Currently, the only method of keeping track of electrode depth and thus location is through detailed notebook keeping on neural activity.

NEW METHOD

We have improved the electrode localization by combining pre- and postoperative anatomical magnetic resonance imaging (MRI) scans with high resolution computed tomography (CT) scans throughout the experiment, and validated our method by comparing the resulting location estimates with traditional notebook-keeping. Finally, the actual location of a selection of electrodes was marked at the end of the experiment by creating small metallic depositions using electrical stimulation, and thereby made visible on MRI.

RESULTS

Combining CT scans with a high resolution, artefact reducing sequence during the experiment with a preoperative MRI scan provides crucial information about the exact electrode location of multielectrode arrays with individually moveable electrodes.

COMPARISON WITH EXISTING METHODS

The information obtained from the hybrid CT-MR image and the notes on spiking activity showed a similar pattern, with the clear advantage of the visualization of the exact position of the electrodes using our method.

CONCLUSIONS

The described technique allows for a precise anatomical identification of the recorded brain areas and thus to draw strong conclusions about the role of each targeted cortical area in the behavior under study.

Towards a Unified View on Pathways and Functions of Neural Recurrent Processing

There are three neural feedback pathways to the primary visual cortex (V1): corticocortical, pulvinocortical, and cholinergic. What are the respective functions of these three projections? Possible functions range from contextual modulation of stimulus processing and feedback of high-level information to predictive processing (PP). How are these functions subserved by different pathways and can they be integrated into an overarching theoretical framework? We propose that corticocortical and pulvinocortical connections are involved in all three functions, whereas the role of cholinergic projections is limited by their slow response to stimuli. PP provides a broad explanatory framework under which stimulus-context modulation and high-level processing are subsumed, involving multiple feedback pathways that provide mechanisms for inferring and interpreting what sensory inputs are about.

Image Segmentation Based on Relative Motion and Relative Disparity Cues in Topographically Organized Areas of Human Visual Cortex

The borders between objects and their backgrounds create discontinuities in image feature maps that can be used to recover object shape. Here we used functional magnetic resonance imaging to identify cortical areas that encode two of the most important image segmentation cues: relative motion and relative disparity. Relative motion and disparity cues were isolated by defining a central 2-degree disk using random-dot kinematograms and stereograms, respectively. For motion, the disk elicited retinotopically organized activations starting in V1 and extending through V2 and V3. In the surrounding region, we observed phase-inverted activations indicative of suppression, extending out to at least 6 degrees of retinal eccentricity. For disparity, disk activations were only found in V3, while suppression was observed in all early visual areas. Outside of early visual cortex, several areas were sensitive to both types of cues, most notably LO1, LO2 and V3B, making them additional candidate areas for motion- and disparity-cue combination. Adding an orthogonal task at fixation did not diminish these effects, and in fact led to small but measurable disk activations in V1 and V2 for disparity. The overall pattern of extra-striate activations is consistent with recent three-stream models of cortical organization.

Figure-Ground Modulation in the Human Lateral Geniculate Nucleus Is Distinguishable from Top-Down Attention

Nearly all of the information that reaches the primary visual cortex (V1) of the brain passes from the retina through the lateral geniculate nucleus (LGN) of the thalamus. Although the LGN's role in relaying feedforward signals from the retina to the cortex is well understood [1, 2], the functional role of the extensive feedback it receives from the cortex has remained elusive [3-6]. Here, we investigated whether corticothalamic feedback may contribute to perceptual processing in the LGN in a manner that is distinct from top-down effects of attention [7-10]. We used high-resolution fMRI at 7 Tesla to simultaneously measure responses to orientation-defined figures in the human LGN and V1. We found robust enhancement of perceptual figures throughout the early visual system, which could be distinguished from the effects of covert spatial attention [11-13]. In a second experiment, we demonstrated that figure enhancement occurred in the LGN even when the figure and surrounding background were presented dichoptically (i.e., to different eyes). As binocular integration primarily occurs in V1 [14, 15], these results implicate a mechanism of automatic, contextually sensitive feedback from binocular visual cortex underlying figure-ground modulation in the LGN. Our findings elucidate the functional mechanisms of this core function of the visual system [16-18], which allows people to segment and detect meaningful figures in complex visual environments. The involvement of the LGN in this rich, contextually informed visual processing-despite showing minimal feedforward selectivity for visual features [19, 20]-underscores the role of recurrent processing at the earliest stages of visual processing.

Common spatiotemporal processing of visual features shapes object representation

Biological vision relies on representations of the physical world at different levels of complexity. Relevant features span from simple low-level properties, as contrast and spatial frequencies, to object-based attributes, as shape and category. However, how these features are integrated into coherent percepts is still debated. Moreover, these dimensions often share common biases: for instance, stimuli from the same category (e.g., tools) may have similar shapes. Here, using magnetoencephalography, we revealed the temporal dynamics of feature processing in human subjects attending to objects from six semantic categories. By employing Relative Weights Analysis, we mitigated collinearity between model-based descriptions of stimuli and showed that low-level properties (contrast and spatial frequencies), shape (medial-axis) and category are represented within the same spatial locations early in time: 100–150 ms after stimulus onset. This fast and overlapping processing may result from independent parallel computations, with categorical representation emerging later than the onset of low-level feature processing, yet before shape coding. Categorical information is represented both before and after shape, suggesting a role for this feature in the refinement of categorical matching.

The time course of different surround suppression mechanisms

What we see depends on the spatial context in which it appears. Previous work has linked the suppression of perceived contrast by surrounding stimuli to reduced neural responses in early visual cortex. This surround suppression depends on at least two separable neural mechanisms, "low-level" and "higher level," which can be differentiated by their response characteristics. We used electroencephalography to demonstrate for the first time that human occipital neural responses show evidence of these two suppression mechanisms. Eighteen adults (10 women, 8 men) each participated in three experimental sessions, in which they viewed visual stimuli through a mirror stereoscope. The first session was used to identify the C1 component, while the second and third comprised the main experiment. Event-related potentials were measured in response to center gratings either with no surround or with surrounding gratings oriented parallel or orthogonal, and presented in either the same eye (monoptic) or the opposite eye (dichoptic). We found that the earliest component of an event-related potential (C1; ∼60 ms) was suppressed by surrounding stimuli, but that suppression did not depend on surround configuration. This suggests a suppression mechanism that is not tuned for relative orientation acting on the earliest cortical response to the target. A later response component (N1; ∼160 ms) showed stronger suppression for parallel and monoptic surrounds, consistent with our earlier psychophysical results and a second form of suppression that is binocular and orientation tuned. We conclude that these two forms of surround suppression have distinct response time courses in the human visual system, which can be differentiated using electrophysiology.

The Segmentation of Proto-Objects in the Monkey Primary Visual Cortex

During visual perception, the brain enhances the representations of image regions that belong to figures and suppresses those that belong to the background. Natural images contain many regions that initially appear to be part of a figure when analyzed locally (proto-objects) but are actually part of the background if the whole image is considered. These proto-grounds must be correctly assigned to the background to allow correct shape identification and guide behavior. To understand how the brain resolves this conflict between local and global processing, we recorded neuronal activity from the primary visual cortex (V1) of macaque monkeys while they discriminated between n/u shapes that have a central proto-ground region. We studied the fine-grained spatiotemporal profile of neural activity evoked by the n/u shape and found that neural representation of the object proceeded from a coarse-to-fine resolution. Approximately 100 ms after the stimulus onset, the representation of the proto-ground region was enhanced together with the rest of the n/u surface, but after ∼115 ms, the proto-ground was suppressed back to the level of the background. Suppression of the proto-ground was only present in animals that had been trained to perform the shape-discrimination task, and it predicted the choice of the animal on a trial-by-trial basis. Attention enhanced figure-ground modulation, but it had no effect on the strength of proto-ground suppression. The results indicate that the accuracy of scene segmentation is sharpened by a suppressive process that resolves local ambiguities by assigning proto-grounds to the background.

Layer 3 Dynamically Coordinates Columnar Activity According to Spatial Context

To reduce statistical redundancy of natural inputs and increase the sparseness of coding, neurons in primary visual cortex (V1) show tuning for stimulus size and surround suppression. This integration of spatial information is a fundamental, context-dependent neural operation involving extensive neural circuits that span across all cortical layers of a V1 column, and reflects both feedforward and feedback processing. However, how spatial integration is dynamically coordinated across cortical layers remains poorly understood. We recorded single- and multiunit activity and local field potentials across V1 layers of awake mice (both sexes) while they viewed stimuli of varying size and used dynamic Bayesian model comparisons to identify when laminar activity and interlaminar functional interactions showed surround suppression, the hallmark of spatial integration. We found that surround suppression is strongest in layer 3 (L3) and L4 activity, where suppression is established within ∼10 ms after response onset, and receptive fields dynamically sharpen while suppression strength increases. Importantly, we also found that specific directed functional connections were strongest for intermediate stimulus sizes and suppressed for larger ones, particularly for connections from L3 targeting L5 and L1. Together, the results shed light on the different functional roles of cortical layers in spatial integration and on how L3 dynamically coordinates activity across a cortical column depending on spatial context.SIGNIFICANCE STATEMENT Neurons in primary visual cortex (V1) show tuning for stimulus size, where responses to stimuli exceeding the receptive field can be suppressed (surround suppression). We demonstrate that functional connectivity between V1 layers can also have a surround-suppressed profile. A particularly prominent role seems to have layer 3, the functional connections to layers 5 and 1 of which are strongest for stimuli of optimal size and decreased for large stimuli. Our results therefore point toward a key role of layer 3 in coordinating activity across the cortical column according to spatial context.

Figure-ground perception in the awake mouse and neuronal activity elicited by figure-ground stimuli in primary visual cortex

Figure-ground segregation is the process by which the visual system identifies image elements of figures and segregates them from the background. Previous studies examined figure-ground segregation in the visual cortex of monkeys where figures elicit stronger neuronal responses than backgrounds. It was demonstrated in anesthetized mice that neurons in the primary visual cortex (V1) of mice are sensitive to orientation contrast, but it is unknown whether mice can perceptually segregate figures from a background. Here, we examined figure-ground perception of mice and found that mice can detect figures defined by an orientation that differs from the background while the figure size, position or phase varied. Electrophysiological recordings in V1 of awake mice revealed that the responses elicited by figures were stronger than those elicited by the background and even stronger at the edge between figure and background. A figural response could even be evoked in the absence of a stimulus in the V1 receptive field. Current-source-density analysis suggested that the extra activity was caused by synaptic inputs into layer 2/3. We conclude that the neuronal mechanisms of figure-ground segregation in mice are similar to those in primates, enabling investigation with the powerful techniques for circuit analysis now available in mice.

Scene complexity modulates degree of feedback activity during object detection in natural scenes

Selective brain responses to objects arise within a few hundreds of milliseconds of neural processing, suggesting that visual object recognition is mediated by rapid feed-forward activations. Yet disruption of neural responses in early visual cortex beyond feed-forward processing stages affects object recognition performance. Here, we unite these discrepant findings by reporting that object recognition involves enhanced feedback activity (recurrent processing within early visual cortex) when target objects are embedded in natural scenes that are characterized by high complexity. Human participants performed an animal target detection task on natural scenes with low, medium or high complexity as determined by a computational model of low-level contrast statistics. Three converging lines of evidence indicate that feedback was selectively enhanced for high complexity scenes. First, functional magnetic resonance imaging (fMRI) activity in early visual cortex (V1) was enhanced for target objects in scenes with high, but not low or medium complexity. Second, event-related potentials (ERPs) evoked by target objects were selectively enhanced at feedback stages of visual processing (from ~220 ms onwards) for high complexity scenes only. Third, behavioral performance for high complexity scenes deteriorated when participants were pressed for time and thus less able to incorporate the feedback activity. Modeling of the reaction time distributions using drift diffusion revealed that object information accumulated more slowly for high complexity scenes, with evidence accumulation being coupled to trial-to-trial variation in the EEG feedback response. Together, these results suggest that while feed-forward activity may suffice to recognize isolated objects, the brain employs recurrent processing more adaptively in naturalistic settings, using minimal feedback for simple scenes and increasing feedback for complex scenes.

Saliency model based on a neural population for integrating figure direction and organizing Border Ownership

Attentional selection is a function of the brain that allocates computational resources momentarily to the most important part of a visual scene. Saliency map models have been used to predict the location of attentional selection and gaze. Border Ownership (BO) indicates the direction of the figure with respect to the border. I here propose a biologically plausible saliency model based on neural population for integrating the activities of intermediate-level visual areas with neurons selective to BO. A variety of BO organizations produces a population of model neurons that represent the grouping structure. In the model I propose, the interactions and the population responses of these model neurons underlie the determination of saliency and the accurate prediction of gaze location. I tested 100 patterns for BO organizations and found that the proposed saliency model not only reproduced the characteristics of perceptual organization but also captured object locations in natural images. Furthermore, the saliency model based on the population responses of the BO organization significantly improved the gaze prediction accuracy compared with previous saliency-based models. These results suggest a crucial role for a wide variety of BO organizations and neural population coding to determine saliency mediating attentional selection and to predict gaze location.

GABA, not BOLD, reveals dissociable learning-dependent plasticity mechanisms in the human brain

Experience and training have been shown to facilitate our ability to extract and discriminate meaningful patterns from cluttered environments. Yet, the human brain mechanisms that mediate our ability to learn by suppressing noisy and irrelevant signals remain largely unknown. To test the role of suppression in perceptual learning, we combine fMRI with MR Spectroscopy measurements of GABA, as fMRI alone does not allow us to discern inhibitory vs. excitatory mechanisms. Our results demonstrate that task-dependent GABAergic inhibition relates to functional brain plasticity and behavioral improvement. Specifically, GABAergic inhibition in the occipito-temporal cortex relates to dissociable learning mechanisms: decreased GABA for noise filtering, while increased GABA for feature template retuning. Perturbing cortical excitability during training with tDCs alters performance in a task-specific manner, providing evidence for a direct link between suppression and behavioral improvement. Our findings propose dissociable GABAergic mechanisms that optimize our ability to make perceptual decisions through training.

When searching for a friend in the crowd or telling identical twins apart, your visual system must solve a complex puzzle. It must ignore all irrelevant information (e.g., unfamiliar faces in the crowd) and focus on key features (e.g., your friend’s familiar face) that will allow you to make a decision. We become better at solving complex visual discriminations with practice. But exactly how the brain achieves this improved performance is unclear.

To answer this question, Frangou et al. trained healthy volunteers on two such visual tasks. The first (target detection task) involved locating a target (e.g. circular shape made of dots among randomly distributed dots in the background), a task similar to identifying a friend in the crowd. The second (feature discrimination task) involved assigning highly alike shapes in two different categories, similar to telling apart identical twins. To solve this problem, volunteers had to identify distinct features that allowed them to distinguishthese shapes. During training on this task, they updated and refined the representation of these distinct features in their brain. This enabled them to make finer discriminations and assign each image correctly to one of the two categories.

While the volunteers trained on the tasks, Frangou et al. measured levels of a chemical called GABA in brain areas that process visual information. GABA is the brain's main inhibitory molecule and controls the activity of neurons. As the volunteers learned the two tasks, their brains showed opposite changes in GABA levels. In the first, target detection task, individuals did better if their GABA decreased during training. In the second, feature discrimination task, they achieved more if their GABA increased during training. To confirm these findings, Frangou et al. used a second technique to activate or suppress processing in visual areas of the brain. Activating visual areas enhanced performance on the target detection task. Suppressing them enhanced performance on the fine discrimination task. These changes are thus consistent with those seen in GABA levels.

As well as revealing how we learn to make decisions based on the information from our eyes, these findings suggest that adjusting brain activity could help patients regain skills lost as a result of eye-related or neurological conditions. Understanding the role of GABA in brain plasticity is also relevant to conditions like autism and psychosis, which have been shown to relate to changes in brain inhibition.

Interactions Elicited by the Contradiction Between Figure Direction Discrimination and Figure-Ground Segregation

Figure-ground (FG) segregation that separates an object from the rest of the image is a fundamental problem in vision science. A majority of neurons in monkey V2 showed the selectivity to border ownership (BO) that indicates which side of a contour owns the border. Although BO could be a precursor of FG segregation, the contribution of BO to FG segregation has not been clarified. Because FG segregation is the perception of the global region that belongs to an object, whereas BO determination provides the local direction of figure (DOF) along a contour, a spatial integration of BO might be expected for the generation of FG. To understand the mechanisms underlying the perception of figural regions, we investigated the interaction between the local BO determination and the global FG segregation through the quantitative analysis of the visual perception and the spatiotemporal characteristics of eye movements. We generated a set of novel stimuli in which translucency induces local DOF along the contour and global FG independently so that DOF and FG could be either consistent or contradictory. The perceptual responses showed better performance in DOF discrimination than FG segregation, supporting distinct mechanisms for the DOF discrimination and the FG segregation. We examined whether the contradiction between DOF and FG modulates the eye movement while participants judged DOF and FG. The duration of the first eye fixation was modulated by the contradiction during FG segregation but not DOF discrimination, suggesting a sequential processing from the BO determination to the FG segregation. These results of human perception and eye fixation provide important clues for understanding the visual processing for FG segregation.

Cognition as a Window into Neuronal Population Space

Understanding how cognitive processes affect the responses of sensory neurons may clarify the relationship between neuronal population activity and behavior. However, tools for analyzing neuronal activity have not kept up with technological advances in recording from large neuronal populations. Here, we describe prevalent hypotheses of how cognitive processes affect sensory neurons, driven largely by a model based on the activity of single neurons or pools of neurons as the units of computation. We then use simple simulations to expand this model to a new conceptual framework that focuses on subspaces of population activity as the relevant units of computation, uses comparisons between brain areas or to behavior to guide analyses of these subspaces, and suggests that population activity is optimized to decode the large variety of stimuli and tasks that animals encounter in natural behavior. This framework provides new ways of understanding the ever-growing quantity of recorded population activity data.

Foreground-Background Segmentation Revealed during Natural Image Viewing

One of the major challenges in visual neuroscience is represented by foreground-background segmentation. Data from nonhuman primates show that segmentation leads to two distinct, but associated processes: the enhancement of neural activity during figure processing (i.e., foreground enhancement) and the suppression of background-related activity (i.e., background suppression). To study foreground-background segmentation in ecological conditions, we introduce a novel method based on parametric modulation of low-level image properties followed by application of simple computational image-processing models. By correlating the outcome of this procedure with human fMRI activity, measured during passive viewing of 334 natural images, we produced easily interpretable "correlation images" from visual populations. Results show evidence of foreground enhancement in all tested regions, from V1 to lateral occipital complex (LOC), while background suppression occurs in V4 and LOC only. Correlation images derived from V4 and LOC revealed a preserved spatial resolution of foreground textures, indicating a richer representation of the salient part of natural images, rather than a simplistic model of object shape. Our results indicate that scene segmentation occurs during natural viewing, even when individuals are not required to perform any particular task.

High-Level Prediction Signals in a Low-Level Area of the Macaque Face-Processing Hierarchy

Theories like predictive coding propose that lower-order brain areas compare their inputs to predictions derived from higher-order representations and signal their deviation as a prediction error. Here, we investigate whether the macaque face-processing system, a three-level hierarchy in the ventral stream, employs such a coding strategy. We show that after statistical learning of specific face sequences, the lower-level face area ML computes the deviation of actual from predicted stimuli. But these signals do not reflect the tuning characteristic of ML. Rather, they exhibit identity specificity and view invariance, the tuning properties of higher-level face areas AL and AM. Thus, learning appears to endow lower-level areas with the capability to test predictions at a higher level of abstraction than what is afforded by the feedforward sweep. These results provide evidence for computational architectures like predictive coding and suggest a new quality of functional organization of information-processing hierarchies beyond pure feedforward schemes.

Distinct Feedforward and Feedback Effects of Microstimulation in Visual Cortex Reveal Neural Mechanisms of Texture Segregation

The visual cortex is hierarchically organized, with low-level areas coding for simple features and higher areas for complex ones. Feedforward and feedback connections propagate information between areas in opposite directions, but their functional roles are only partially understood. We used electrical microstimulation to perturb the propagation of neuronal activity between areas V1 and V4 in monkeys performing a texture-segregation task. In both areas, microstimulation locally caused a brief phase of excitation, followed by inhibition. Both these effects propagated faithfully in the feedforward direction from V1 to V4. Stimulation of V4, however, caused little V1 excitation, but it did yield a delayed suppression during the late phase of visually driven activity. This suppression was pronounced for the V1 figure representation and weaker for background representations. Our results reveal functional differences between feedforward and feedback processing in texture segregation and suggest a specific modulating role for feedback connections in perceptual organization.