Comparison of spatial summation properties of neurons in macaque V1 and V2

Orientation selectivity mapping in the visual cortex

The orientation map is one of the most well-studied functional maps of the visual cortex. However, results from the literature are of different qualities. Clear boundaries among different orientation domains and blurred uncertain distinctions were shown in different studies. These unclear imaging results will lead to an inaccuracy in depicting cortical structures, and the lack of consideration in experimental design will also lead to biased depictions of the cortical features. How we accurately define orientation domains will impact the entire field of research. In this study, we test how spatial frequency (SF), stimulus size, location, chromatic, and data processing methods affect the orientation functional maps (including a large area of dorsal V4, and parts of dorsal V1) acquired by intrinsic signal optical imaging. Our results indicate that, for large imaging fields, large grating stimuli with mixed SF components should be considered to acquire the orientation map. A diffusion model image enhancement based on the difference map could further improve the map quality. In addition, the similar outcomes of achromatic and chromatic gratings indicate two alternative types of afferents from LGN, pooling in V1 to generate cue-invariant orientation selectivity.

Visual surround suppression at the neural and perceptual levels

Surround suppression was initially identified as a phenomenon at the neural level in which stimuli outside the neuron's receptive field alone cannot activate responses but can modulate neural responses to stimuli covered inside the receptive field. Subsequent studies showed that surround suppression is not only a critical property of neurons across species and brain areas but also has been found in visual perceptions. More importantly, surround suppression varies across individuals and shows significant differences between normal controls and patients with certain mental disorders. Here, we combined results from related literature and summarized the findings derived from physiological and psychophysical evidence. We first outline the basic properties of surround suppression in the visual system and perceptions. Then, we mainly summarize the differences in perceptual surround suppression among different human subjects. Our review suggests that there is no consensus regarding whether the strength of perceptual surround suppression could be used as an effective index to distinguish particular populations. Then, we summarized the similar mechanisms for surround suppression and cognitive impairments to further explore the potential clinical applications of surround suppression. A clearer understanding of the mechanisms of surround suppression in neural responses and perceptions is necessary for facilitating its clinical applications.

Non-uniform contextual interactions in the visual cortex place fundamental limits on spatial vision

A prevailing assumption in our understanding of how neurons in the primary visual cortex (V1) integrate contextual information is that such processes are spatially uniform. Conversely, perceptual phenomena such as visual crowding, the impaired ability to accurately recognize a target stimulus among distractors, suggest that interactions among stimuli are distinctly non-uniform. Prior studies have shown flankers at specific spatial geometries exert differential effects on target perception. To resolve this discrepancy, we investigated how flanker geometry impacted the representation of a target stimulus in the laminar microcircuits of V1. Our study reveals flanker location differentially impairs stimulus representation in excitatory neurons in the superficial and input layers of V1 by tuned suppression and untuned facilitation of orientation responses. Mechanistically, this effect can be explained by asymmetrical spatial kernels in a normalization model of cortical activity. Strikingly, these non-uniform modulations of neural representation mirror perceptual anisotropies. These results establish the non-uniform spatial integration of information in the earliest stages of cortical processing as a fundamental limitation of spatial vision.

The Architecture of Object-Based Attention

The allocation of attention to objects raises several intriguing questions: What are objects, how does attention access them, what anatomical regions are involved? Here, we review recent progress in the field to determine the mechanisms underlying object-based attention. First, findings from unconscious priming and cueing suggest that the preattentive targets of object-based attention can be fully developed object representations that have reached the level of identity. Next, the control of object-based attention appears to come from ventral visual areas specialized in object analysis that project downward to early visual areas. How feedback from object areas can accurately target the object's specific locations and features is unknown but recent work in autoencoding has made this plausible. Finally, we suggest that the three classic modes of attention may not be as independent as is commonly considered, and instead could all rely on object-based attention. Specifically, studies show that attention can be allocated to the separated members of a group-without affecting the space between them-matching the defining property of feature-based attention. At the same time, object-based attention directed to a single small item has the properties of space-based attention. We outline the architecture of object-based attention, the novel predictions it brings, and discuss how it works in parallel with other attention pathways.

Generalizing biological surround suppression based on center surround similarity via deep neural network models

Sensory perception is dramatically influenced by the context. Models of contextual neural surround effects in vision have mostly accounted for Primary Visual Cortex (V1) data, via nonlinear computations such as divisive normalization. However, surround effects are not well understood within a hierarchy, for neurons with more complex stimulus selectivity beyond V1. We utilized feedforward deep convolutional neural networks and developed a gradient-based technique to visualize the most suppressive and excitatory surround. We found that deep neural networks exhibited a key signature of surround effects in V1, highlighting center stimuli that visually stand out from the surround and suppressing responses when the surround stimulus is similar to the center. We found that in some neurons, especially in late layers, when the center stimulus was altered, the most suppressive surround surprisingly can follow the change. Through the visualization approach, we generalized previous understanding of surround effects to more complex stimuli, in ways that have not been revealed in visual cortices. In contrast, the suppression based on center surround similarity was not observed in an untrained network. We identified further successes and mismatches of the feedforward CNNs to the biology. Our results provide a testable hypothesis of surround effects in higher visual cortices, and the visualization approach could be adopted in future biological experimental designs.

Electrical stimulation thresholds differ between V1 and V2

Cortical visual prostheses are designed to treat blindness by restoring visual perceptions through artificial electrical stimulation of the primary visual cortex (V1). Intracortical microelectrodes produce the smallest visual percepts and thus higher resolution vision - like a higher density of pixels on a monitor. However, intracortical microelectrodes must maintain a minimum spacing to preserve tissue integrity. One solution to increase the density of percepts is to implant and stimulate multiple visual areas, such as V1 and V2, although the properties of microstimulation in V2 remain largely unexplored. We provide a direct comparison of V1 and V2 microstimulation in two common marmoset monkeys. We find similarities in response trends between V1 and V2 but differences in threshold, neural activity duration, and spread of activity at the threshold current. This has implications for using multi-area stimulation to increase the resolution of cortical visual prostheses.

Size tuning of neural response variability in laminar circuits of macaque primary visual cortex

A defining feature of the cortex is its laminar organization, which is likely critical for cortical information processing. For example, visual stimuli of different size evoke distinct patterns of laminar activity. Visual information processing is also influenced by the response variability of individual neurons and the degree to which this variability is correlated among neurons. To elucidate laminar processing, we studied how neural response variability across the layers of macaque primary visual cortex is modulated by visual stimulus size. Our laminar recordings revealed that single neuron response variability and the shared variability among neurons are tuned for stimulus size, and this size-tuning is layer-dependent. In all layers, stimulation of the receptive field (RF) reduced single neuron variability, and the shared variability among neurons, relative to their pre-stimulus values. As the stimulus was enlarged beyond the RF, both single neuron and shared variability increased in supragranular layers, but either did not change or decreased in other layers. Surprisingly, we also found that small visual stimuli could increase variability relative to baseline values. Our results suggest multiple circuits and mechanisms as the source of variability in different layers and call for the development of new models of neural response variability.

Contributions of low- and high-level contextual mechanisms to human face perception

Contextual modulations at primary stages of visual processing depend on the strength of local input. Contextual modulations at high-level stages of (face) processing show a similar dependence to local input strength. Namely, the discriminability of a facial feature determines the amount of influence of the face context on that feature. How high-level contextual modulations emerge from primary mechanisms is unclear due to the scarcity of empirical research systematically addressing the functional link between the two. We tested (62) young adults' ability to process local input independent of the context using contrast detection and (upright and inverted) morphed facial feature matching tasks. We first investigated contextual modulation magnitudes across tasks to address their shared variance. A second analysis focused on the profile of performance across contextual conditions. In upright eye matching and contrast detection tasks, contextual modulations only correlated at the level of their profile (averaged Fisher-Z transformed r = 1.18, BF10 > 100), but not magnitude (r = .15, BF10 = .61), suggesting the functional independence but similar working principles of the mechanisms involved. Both the profile (averaged Fisher-Z transformed r = .32, BF10 = 9.7) and magnitude (r = .28, BF10 = 4.58) of the contextual modulations correlated between inverted eye matching and contrast detection tasks. Our results suggest that non-face-specialized high-level contextual mechanisms (inverted faces) work in connection to primary contextual mechanisms, but that the engagement of face-specialized mechanisms for upright faces obscures this connection. Such combined study of low- and high-level contextual modulations sheds new light on the functional relationship between different levels of the visual processing hierarchy, and thus on its functional organization.

Independent repetition suppression in macaque area V2 and inferotemporal cortex

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.NEW & NOTEWORTHY When a complexly structured natural image is presented twice in rapid succession, neurons in inferotemporal area TE exhibit repetition suppression, responding less strongly to the second than to the first presentation. We have explored whether this phenomenon is confined to high-order areas where neurons respond selectively to such images and thus carry information relevant to recognizing a repeat. We have found surprisingly that repetition suppression occurs even in low-order visual area V2.

Cascaded normalizations for spatial integration in the primary visual cortex of primates

Spatial integration of visual information is an important function in the brain. However, neural computation for spatial integration in the visual cortex remains unclear. In this study, we recorded laminar responses in V1 of awake monkeys driven by visual stimuli with grating patches and annuli of different sizes. We find three important response properties related to spatial integration that are significantly different between input and output layers: neurons in output layers have stronger surround suppression, smaller receptive field (RF), and higher sensitivity to grating annuli partially covering their RFs. These interlaminar differences can be explained by a descriptive model composed of two global divisions (normalization) and a local subtraction. Our results suggest suppressions with cascaded normalizations (CNs) are essential for spatial integration and laminar processing in the visual cortex. Interestingly, the features of spatial integration in convolutional neural networks, especially in lower layers, are different from our findings in V1.

Horizontal connectivity in V1: Prediction of coherence in contour and motion integration

This study demonstrates the functional importance of the Surround context relayed laterally in V1 by the horizontal connectivity, in controlling the latency and the gain of the cortical response to the feedforward visual drive. We report here four main findings: 1) a centripetal apparent motion sequence results in a shortening of the spiking latency of V1 cells, when the orientation of the local inducer and the global motion axis are both co-aligned with the RF orientation preference; 2) this contextual effects grows with visual flow speed, peaking at 150–250°/s when it matches the propagation speed of horizontal connectivity (0.15–0.25 mm/ms); 3) For this speed range, the axial sensitivity of V1 cells is tilted by 90° to become co-aligned with the orientation preference axis; 4) the strength of modulation by the surround context correlates with the spatiotemporal coherence of the apparent motion flow. Our results suggest an internally-generated binding process, linking local (orientation /position) and global (motion/direction) features as early as V1. This long-range diffusion process constitutes a plausible substrate in V1 of the human psychophysical bias in speed estimation for collinear motion. Since it is demonstrated in the anesthetized cat, this novel form of contextual control of the cortical gain and phase is a built-in property in V1, whose expression does not require behavioral attention and top-down control from higher cortical areas. We propose that horizontal connectivity participates in the propagation of an internal “prediction” wave, shaped by visual experience, which links contour co-alignment and global axial motion at an apparent speed in the range of saccade-like eye movements.

The role of lateral modulation in orientation-specific adaptation effect

Center-surround modulation in visual processing reflects a normalization process of contrast gain control in the responsive neurons. Prior adaptation to a clockwise (CW) tilted grating, for example, leads to the percept of counterclockwise tilt in a vertical grating, referred to as the tilt-aftereffect (TAE). We previously reported that the magnitude of the TAE is modulated by adding a same-orientation annular surround to an adapter, suggesting inhibitory lateral modulation. To further examine the property of this lateral modulation effect on the perception of a central target, we here used center-surround sinusoidal patterns as adapters and varied the adapter surround and center orientations independently. The target had the same spatial extent as the adapter center with no physical overlap with the adapter surround. Participants were asked to judge the target orientation as tilted either CW or counterclockwise from vertical after adaptation. Results showed that, when the surround orientation was held constant, the TAE magnitude was determined by the adapter center, peaking between 10° and 20° of tilt. More important, the adapter surround orientation modulated the adaptation effect such that the TAE magnitude first decreased and then increased as the surround orientation became increasingly more different from that of the center, suggesting that the surround modulation effect was indeed orientation specific. Our data can be accounted for by a divisive inhibition model, in which (1) the adaptation effect is represented by increasing the normalizing constant and (2) the surround modulation is captured by two multiplicative sensitivity parameters determined by the adapter surround orientation.

Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey

Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance.

How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.

Population receptive fields of human primary visual cortex organised as DC-balanced bandpass filters

The response to visual stimulation of population receptive fields (pRF) in the human visual cortex has been modelled with a Difference of Gaussians model, yet many aspects of their organisation remain poorly understood. Here, we examined the mathematical basis and signal-processing properties of this model and argue that the DC-balanced Difference of Gaussians (DoG) holds a number of advantages over a DC-biased DoG. Through functional magnetic resonance imaging (fMRI) pRF mapping, we compared performance of DC-balanced and DC-biased models in human primary visual cortex and found that when model complexity is taken into account, the DC-balanced model is preferred. Finally, we present evidence indicating that the BOLD signal DC offset contains information related to the processing of visual stimuli. Taken together, the results indicate that V1 pRFs are at least frequently organised in the exact constellation that allows them to function as bandpass filters, which makes the separation of stimulus contrast and luminance possible. We further speculate that if the DoG models stimulus contrast, the DC offset may reflect stimulus luminance. These findings suggest that it may be possible to separate contrast and luminance processing in fMRI experiments and this could lead to new insights on the haemodynamic response.

What is the dynamical regime of cerebral cortex?

Many studies have shown that the excitation and inhibition received by cortical neurons remain roughly balanced across many conditions. A key question for understanding the dynamical regime of cortex is the nature of this balancing. Theorists have shown that network dynamics can yield systematic cancellation of most of a neuron's excitatory input by inhibition. We review a wide range of evidence pointing to this cancellation occurring in a regime in which the balance is loose, meaning that the net input remaining after cancellation of excitation and inhibition is comparable in size with the factors that cancel, rather than tight, meaning that the net input is very small relative to the canceling factors. This choice of regime has important implications for cortical functional responses, as we describe: loose balance, but not tight balance, can yield many nonlinear population behaviors seen in sensory cortical neurons, allow the presence of correlated variability, and yield decrease of that variability with increasing external stimulus drive as observed across multiple cortical areas.

Opposing effects of selectivity and invariance in peripheral vision

Sensory processing necessitates discarding some information in service of preserving and reformatting more behaviorally relevant information. Sensory neurons seem to achieve this by responding selectively to particular combinations of features in their inputs, while averaging over or ignoring irrelevant combinations. Here, we expose the perceptual implications of this tradeoff between selectivity and invariance, using stimuli and tasks that explicitly reveal their opposing effects on discrimination performance. We generate texture stimuli with statistics derived from natural photographs, and ask observers to perform two different tasks: Discrimination between images drawn from families with different statistics, and discrimination between image samples with identical statistics. For both tasks, the performance of an ideal observer improves with stimulus size. In contrast, humans become better at family discrimination but worse at sample discrimination. We demonstrate through simulations that these behaviors arise naturally in an observer model that relies on a common set of physiologically plausible local statistical measurements for both tasks.

Stimulus Contrast Affects Spatial Integration in the Lateral Geniculate Nucleus of Macaque Monkeys

Gain-control mechanisms adjust neuronal responses to accommodate the wide range of stimulus conditions in the natural environment. Contrast gain control and extraclassical surround suppression are two manifestations of gain control that govern the responses of neurons in the early visual system. Understanding how these two forms of gain control interact has important implications for the detection and discrimination of stimuli across a range of contrast conditions. Here, we report that stimulus contrast affects spatial integration in the lateral geniculate nucleus of alert macaque monkeys (male and female), whereby neurons exhibit a reduction in the strength of extraclassical surround suppression and an expansion in the preferred stimulus size with low-contrast stimuli compared with high-contrast stimuli. Effects were greater for magnocellular neurons than for parvocellular neurons, indicating stream-specific interactions between stimulus contrast and stimulus size. Within the magnocellular pathway, contrast-dependent effects were comparable for ON-center and OFF-center neurons, despite ON neurons having larger receptive fields, less pronounced surround suppression, and more pronounced contrast gain control than OFF neurons. Together, these findings suggest that the parallel streams delivering visual information from retina to primary visual cortex, serve not only to broaden the range of signals delivered to cortex, but also to provide a substrate for differential interactions between stimulus contrast and stimulus size that may serve to improve stimulus detection and stimulus discrimination under pathway-specific lower and higher contrast conditions, respectively.SIGNIFICANCE STATEMENT Stimulus contrast is a salient feature of visual scenes. Here we examine the influence of stimulus contrast on spatial integration in the lateral geniculate nucleus (LGN). Our results demonstrate that increases in contrast generally increase extraclassical suppression and decrease the size of optimal stimuli, indicating a reduction in the extent of visual space from which LGN neurons integrate signals. Differences between magnocellular and parvocellular neurons are noteworthy and further demonstrate that the feedforward parallel pathways to cortex increase the range of information conveyed for downstream cortical processing, a range broadened by diversity in the ON and OFF pathways. These results have important implications for more complex visual processing that underly the detection and discrimination of stimuli under varying natural conditions.

Early recurrence enables figure border ownership

Rubin's face-vase illusion demonstrates how one can switch back and forth between two different interpretations depending on how the figure outlines are assigned. In the primate visual system, assigning ownership along figure borders is encoded by neurons called the border ownership (BO) cells. Studies show that the responses of these neurons not only depend on the local features within their receptive fields, but also on contextual information. Despite two decades of studies on BO neurons, the ownership assignment mechanism in the brain is still unknown. Here, we propose a hierarchical recurrent model grounded on the hypothesis that neurons in the dorsal stream provide the context required for ownership assignment. Our proposed model incorporates early recurrence from the dorsal pathway as well as lateral modulations within the ventral stream. While dorsal modulations initiate the response difference to figure on either side of the border, lateral modulations enhance the difference. We found responses of our dorsally-modulated BO cells, similar to their biological counterparts, are invariant to size, position and solid/outlined figures. Moreover, our model BO cells exhibit comparable levels of reliability in the ownership signal to biological BO neurons. We found dorsal modulations result in high levels of accuracy and robustness for BO assignments in complex scenes compared to previous models based on ventral feedback. Finally, our experiments with illusory contours suggest that BO encoding could explain the perception of such contours in higher processing stages in the brain.

Effects of Increasing Stimulated Area in Spatiotemporally Congruent Unisensory and Multisensory Conditions

Research has shown that the ability to integrate complementary sensory inputs into a unique and coherent percept based on spatiotemporal coincidence can improve perceptual precision, namely multisensory integration. Despite the extensive research on multisensory integration, very little is known about the principal mechanisms responsible for the spatial interaction of multiple sensory stimuli. Furthermore, it is not clear whether the size of spatialized stimulation can affect unisensory and multisensory perception. The present study aims to unravel whether the stimulated area’s increase has a detrimental or beneficial effect on sensory threshold. Sixteen typical adults were asked to discriminate unimodal (visual, auditory, tactile), bimodal (audio-visual, audio-tactile, visuo-tactile) and trimodal (audio-visual-tactile) stimulation produced by one, two, three or four devices positioned on the forearm. Results related to unisensory conditions indicate that the increase of the stimulated area has a detrimental effect on auditory and tactile accuracy and visual reaction times, suggesting that the size of stimulated areas affects these perceptual stimulations. Concerning multisensory stimulation, our findings indicate that integrating auditory and tactile information improves sensory precision only when the stimulation area is augmented to four devices, suggesting that multisensory interaction is occurring for expanded spatial areas.

Laminar Subnetworks of Response Suppression in Macaque Primary Visual Cortex

Cortical inhibition plays an important role in information processing in the brain. However, the mechanisms by which inhibition and excitation are coordinated to generate functions in the six layers of the cortex remain unclear. Here, we measured laminar-specific responses to stimulus orientations in primary visual cortex (V1) of awake monkeys (male, Macaca mulatta). We distinguished inhibitory effects (suppression) from excitation, by taking advantage of the separability of excitation and inhibition in the orientation and time domains. We found two distinct types of suppression governing different layers. Fast suppression (FS) was strongest in input layers (4C and 6), and slow suppression (SS) was 3 times stronger in output layers (2/3 and 5). Interestingly, the two types of suppression were correlated with different functional properties measured with drifting gratings. FS was primarily correlated with orientation selectivity in input layers (r = -0.65, p < 10-9), whereas SS was primarily correlated with surround suppression in output layers (r = 0.61, p < 10-4). The earliest SS in layer 1 indicates the origin of cortical feedback for SS, in contrast to the feedforward/recurrent origin of FS. Our results reveal two V1 laminar subnetworks with different response suppression that may provide a general framework for laminar processing in other sensory cortices.SIGNIFICANCE STATEMENT This study sought to understand inhibitory effects (suppression) and their relationships with functional properties in the six different layers of the cortex. We found that the diversity of neural responses across layers in primary visual cortex (V1) could be fully explained by one excitatory and two suppressive components (fast and slow suppression). The distinct laminar distributions, origins, and functional roles of the two types of suppression provided a simplified representation of the differences between two V1 subnetworks (input network and output network). These results not only help to elucidate computational principles in macaque V1, but also provide a framework for general computation of cortical laminae in other sensory cortices.

Lateral modulation of orientation perception in center-surround sinusoidal stimuli: Divisive inhibition in perceptual filling-in

The perception of a target stimulus may be altered by its context. Perceptual filling-in is thought to be one example of lateral modulation, in which the percept of a central blank area is replaced by that of the surround. We investigated the mechanisms in eccentric vision underlying filling-in by selectively adapting the center (pedestal adapter), surround (annulus adapter), or both (disk adapter) in a sinusoidal grating and observed how the adaptation influences the orientation percept of a subsequently presented Gabor target, located at the same position as the adapter center. In a binary choice task, observers were to judge the orientation (clockwise or counterclockwise) of the target after adaptation. The tilt aftereffect (TAE), corresponding to an illusory tilt of a physically vertical Gabor target, depended both on the adapter orientation and the adapter type. The TAE, peaked between 10 degrees and 20 degrees adapter orientation, was strongest in the pedestal, followed by the disk, and weakest in the annulus adapter conditions. The difference between the disk and pedestal conditions implies lateral inhibition from the surround. Lacking physical overlap with the target, the annulus adapter nonetheless induced a small but significant TAE in the central area. The effect of filling-in on the TAE was estimated by comparing the results from trials with and without subjectively reported filling-in during adaptation to the annulus adapter. The TAE was greater when filling-in occurred during adaptation, suggesting a stronger lateral modulation effect on trials where filling-in was induced. The data were fit by a variant of a divisive inhibition model, in which the adaptation effect is captured by the increase of an additive constant in the denominator of the response function, whereas the surround modulation in the adapter is modeled by an excitatory sensitivity in the numerator.

Deciphering image contrast in object classification deep networks

The ultimate goal of neuroscience is to explain how complex behaviour arises from neuronal activity. A comparable level of complexity also emerges in deep neural networks (DNNs) while exhibiting human-level performance in demanding visual tasks. Unlike in biological systems, all parameters and operations of DNNs are accessible. Therefore, in theory, it should be possible to decipher the exact mechanisms learnt by these artificial networks. Here, we investigate the concept of contrast invariance within the framework of DNNs. We start by discussing how a network can achieve robustness to changes in local and global image contrast. We used a technique from neuroscience-"kernel lesion"-to measure the degree of performance degradation when individual kernels are eliminated from a network. We further compared contrast normalisation, a mechanism used in biological systems, to the strategies that DNNs learn to cope with changes of contrast. The results of our analysis suggest that (i) contrast is a low-level feature for these networks, and it is encoded in the shallow layers; (ii) a handful of kernels appear to have a greater impact on this feature, and their removal causes a substantially larger accuracy loss for low-contrast images; (iii) edges are a distinct visual feature within the internal representation of object classification DNNs.

Anatomy and Physiology of Macaque Visual Cortical Areas V1, V2, and V5/MT: Bases for Biologically Realistic Models

The cerebral cortex of primates encompasses multiple anatomically and physiologically distinct areas processing visual information. Areas V1, V2, and V5/MT are conserved across mammals and are central for visual behavior. To facilitate the generation of biologically accurate computational models of primate early visual processing, here we provide an overview of over 350 published studies of these three areas in the genus Macaca, whose visual system provides the closest model for human vision. The literature reports 14 anatomical connection types from the lateral geniculate nucleus of the thalamus to V1 having distinct layers of origin or termination, and 194 connection types between V1, V2, and V5, forming multiple parallel and interacting visual processing streams. Moreover, within V1, there are reports of 286 and 120 types of intrinsic excitatory and inhibitory connections, respectively. Physiologically, tuning of neuronal responses to 11 types of visual stimulus parameters has been consistently reported. Overall, the optimal spatial frequency (SF) of constituent neurons decreases with cortical hierarchy. Moreover, V5 neurons are distinct from neurons in other areas for their higher direction selectivity, higher contrast sensitivity, higher temporal frequency tuning, and wider SF bandwidth. We also discuss currently unavailable data that could be useful for biologically accurate models.

Spatial contextual effects in primary visual cortex limit feature representation under crowding

Crowding is a profound loss of discriminability of visual features, when a target stimulus is surrounded by distractors. Numerous studies of human perception have characterized how crowding depends on the properties of a visual display. Yet, there is limited understanding of how and where stimulus information is lost in the visual system under crowding. Here, we show that macaque monkeys exhibit perceptual crowding for target orientation that is similar to humans. We then record from neuronal populations in monkey primary visual cortex (V1). These populations show an appreciable loss of information about target orientation in the presence of distractors, due both to divisive and additive modulation of responses to targets by distractors. Our results show that spatial contextual effects in V1 limit the discriminability of visual features and can contribute substantively to crowding.

Visual crowding can strongly limit perceptual discriminability, yet its neural basis remains unclear. Here, the authors show that perceptual crowding is similar in monkeys and humans, and that feature encoding in neuronal populations in primary visual cortex is limited for displays inducing crowding.

Underwater Image Enhancement Using Adaptive Retinal Mechanisms

We propose an underwater image enhancement model inspired by the morphology and function of the teleost fish retina. We aim to solve the problems of underwater image degradation raised by the blurring and nonuniform color biasing. In particular, the feedback from color-sensitive horizontal cells to cones and a red channel compensation are used to correct the nonuniform color bias. The center-surround opponent mechanism of the bipolar cells and the feedback from amacrine cells to interplexiform cells then to horizontal cells serve to enhance the edges and contrasts of the output image. The ganglion cells with color-opponent mechanism are used for color enhancement and color correction. Finally, we adopt a luminance-based fusion strategy to reconstruct the enhanced image from the outputs of ON and OFF pathways of fish retina. Our model utilizes the global statistics (i.e., image contrast) to automatically guide the design of each low-level filter, which realizes the self-adaption of the main parameters. Extensive qualitative and quantitative evaluations on various underwater scenes validate the competitive performance of our technique. Our model also significantly improves the accuracy of transmission map estimation and local feature point matching using the underwater image. Our method is a single image approach that does not require the specialized prior about the underwater condition or scene structure.

Integrating Flexible Normalization into Midlevel Representations of Deep Convolutional Neural Networks

Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.

Lateral effects in pattern vision

There is a large literature on lateral effects in pattern vision but no consensus about them or comprehensive model of them. This paper reviews the literature with a focus on the effects of parallel context in the central fovea. It describes seven experiments that measure detection and discrimination thresholds in annular and Gabor-pattern contexts at different separations. It presents a model of these effects, which is an elaboration of Foley's (1994) model. The model describes the results well, and it shows that lateral context affects the response to the target by both multiplicative excitation and additive inhibition. Both lateral effects extend for several wavelengths beyond the target. They vary in relative strength, producing near suppression and far enhancement of the response to the target. The model describes the detection and discrimination results well, and it also describes the results of experiments on lateral effects on perceived contrast. The model is consistent with the physiology of V1 cells.

The time course of different surround suppression mechanisms

What we see depends on the spatial context in which it appears. Previous work has linked the suppression of perceived contrast by surrounding stimuli to reduced neural responses in early visual cortex. This surround suppression depends on at least two separable neural mechanisms, "low-level" and "higher level," which can be differentiated by their response characteristics. We used electroencephalography to demonstrate for the first time that human occipital neural responses show evidence of these two suppression mechanisms. Eighteen adults (10 women, 8 men) each participated in three experimental sessions, in which they viewed visual stimuli through a mirror stereoscope. The first session was used to identify the C1 component, while the second and third comprised the main experiment. Event-related potentials were measured in response to center gratings either with no surround or with surrounding gratings oriented parallel or orthogonal, and presented in either the same eye (monoptic) or the opposite eye (dichoptic). We found that the earliest component of an event-related potential (C1; ∼60 ms) was suppressed by surrounding stimuli, but that suppression did not depend on surround configuration. This suggests a suppression mechanism that is not tuned for relative orientation acting on the earliest cortical response to the target. A later response component (N1; ∼160 ms) showed stronger suppression for parallel and monoptic surrounds, consistent with our earlier psychophysical results and a second form of suppression that is binocular and orientation tuned. We conclude that these two forms of surround suppression have distinct response time courses in the human visual system, which can be differentiated using electrophysiology.

Layer 3 Dynamically Coordinates Columnar Activity According to Spatial Context

To reduce statistical redundancy of natural inputs and increase the sparseness of coding, neurons in primary visual cortex (V1) show tuning for stimulus size and surround suppression. This integration of spatial information is a fundamental, context-dependent neural operation involving extensive neural circuits that span across all cortical layers of a V1 column, and reflects both feedforward and feedback processing. However, how spatial integration is dynamically coordinated across cortical layers remains poorly understood. We recorded single- and multiunit activity and local field potentials across V1 layers of awake mice (both sexes) while they viewed stimuli of varying size and used dynamic Bayesian model comparisons to identify when laminar activity and interlaminar functional interactions showed surround suppression, the hallmark of spatial integration. We found that surround suppression is strongest in layer 3 (L3) and L4 activity, where suppression is established within ∼10 ms after response onset, and receptive fields dynamically sharpen while suppression strength increases. Importantly, we also found that specific directed functional connections were strongest for intermediate stimulus sizes and suppressed for larger ones, particularly for connections from L3 targeting L5 and L1. Together, the results shed light on the different functional roles of cortical layers in spatial integration and on how L3 dynamically coordinates activity across a cortical column depending on spatial context.SIGNIFICANCE STATEMENT Neurons in primary visual cortex (V1) show tuning for stimulus size, where responses to stimuli exceeding the receptive field can be suppressed (surround suppression). We demonstrate that functional connectivity between V1 layers can also have a surround-suppressed profile. A particularly prominent role seems to have layer 3, the functional connections to layers 5 and 1 of which are strongest for stimuli of optimal size and decreased for large stimuli. Our results therefore point toward a key role of layer 3 in coordinating activity across the cortical column according to spatial context.

Distinct Laminar Processing of Local and Global Context in Primate Primary Visual Cortex

Visual perception is affected by spatial context. In visual cortex, neuronal responses to stimuli inside the receptive field (RF) are suppressed by stimuli in the RF surround. To understand the circuits and cortical layers processing spatial context, we simultaneously recorded across all layers of macaque primary visual cortex while presenting stimuli at increasing distances from the recorded cells' RF. We find that near versus far-surround stimuli activate distinct layers, thus revealing unique laminar contributions to the processing of local and global spatial context. Stimuli in the near-surround evoke the earliest subthreshold responses in superficial and upper-deep layers, and earliest suppression of spiking responses in superficial layers. Conversely, far-surround stimuli evoke the earliest subthreshold responses in feedback-recipient layer 1 and lower-deep layers, and earliest suppression of spiking responses almost simultaneously in all layers, except 4C, where suppression emerges last. Our results suggest distinct circuits for local and global signal integration.

Neural Correlate of Visual Familiarity in Macaque Area V2

Neurons in macaque inferotemporal cortex (ITC) respond less strongly to familiar than to novel images. It is commonly assumed that this effect arises within ITC because its neurons respond selectively to complex images and thus encode in an explicit form information sufficient for identifying a particular image as familiar. However, no prior study has examined whether neurons in low-order visual areas selective for local features also exhibit familiarity suppression. To address this issue, we recorded from neurons in macaque area V2 with semichronic microelectrode arrays while monkeys repeatedly viewed a set of large complex natural images. We report here that V2 neurons exhibit familiarity suppression. The effect develops over several days with a trajectory well fitted by an exponential function with a rate constant of ∼100 exposures. Suppression occurs in V2 at a latency following image onset shorter than its reported latency in ITC.SIGNIFICANCE STATEMENT Familiarity suppression, the tendency for neurons to respond less strongly to familiar than novel images, is well known in monkey inferotemporal cortex. Suppression has been thought to arise in inferotemporal cortex because its neurons respond selectively to large complex images and thus explicitly to encode information sufficient for identifying a particular image as familiar. No previous study has explored the possibility that familiarity suppression occurs even in early-stage visual areas where neurons are selective for simple features in confined receptive fields. We now report that neurons in area V2 exhibit familiarity suppression. This finding challenges our current understanding of information processing in V2 as well as our understanding of the mechanisms that underlie familiarity suppression.

Colour Constancy Beyond the Classical Receptive Field

The problem of removing illuminant variations to preserve the colours of objects (colour constancy) has already been solved by the human brain using mechanisms that rely largely on centre-surround computations of local contrast. In this paper we adopt some of these biological solutions described by long known physiological findings into a simple, fully automatic, functional model (termed Adaptive Surround Modulation or ASM). In ASM, the size of a visual neuron's receptive field (RF) as well as the relationship with its surround varies according to the local contrast within the stimulus, which in turn determines the nature of the centre-surround normalisation of cortical neurons higher up in the processing chain. We modelled colour constancy by means of two overlapping asymmetric Gaussian kernels whose sizes are adapted based on the contrast of the surround pixels, resembling the change of RF size. We simulated the contrast-dependent surround modulation by weighting the contribution of each Gaussian according to the centre-surround contrast. In the end, we obtained an estimation of the illuminant from the set of the most activated RFs' outputs. Our results on three single-illuminant and one multi-illuminant benchmark datasets show that ASM is highly competitive against the state-of-the-art and it even outperforms learning-based algorithms in one case. Moreover, the robustness of our model is more tangible if we consider that our results were obtained using the same parameters for all datasets, that is, mimicking how the human visual system operates. These results suggest a dynamical adaptation mechanisms contribute to achieving higher accuracy in computational colour constancy.

Comparison of Decision-Related Signals in Sensory and Motor Preparatory Responses of Neurons in Area LIP

Neurons in the lateral intraparietal (LIP) area of Macaques exhibit both sensory and oculomotor preparatory responses. During perceptual decision making, the preparatory responses have been shown to track the state of the evolving evidence leading to the decision. The sensory responses are known to reflect categorical properties of visual stimuli, but it is not known whether these responses also track evolving evidence. We recorded neural responses from lateral intraparietal area of 2 female rhesus monkeys during a direction discrimination task. We compared sensory and oculomotor-preparatory responses in the same neurons when either the discriminandum (random dot motion) or an eye movement choice-target was in the neuron's response field. The neural responses in both configurations reflected the strength and direction of motion and were correlated with the animal's choice, albeit more prominently when the choice-target was in the response field. However, the variance and autocorrelation pattern of only the motor preparatory responses reflected the process of evidence accumulation. Simulations suggest that the task related activity of sensory responses could be inherited through lateral interactions with neurons that are carrying evidence accumulation signals in their motor-preparatory responses. The results are consistent with the proposal that evolving decision processes are supported by persistent neural activity in the service of actions or intentions, as opposed to high-order representations of stimulus properties.SIGNIFICANCE STATEMENT Perceptual decision making is the process of choosing an appropriate motor action based on perceived sensory information. Association areas of the cortex play an important role in this sensory-motor transformation. The neurons in these areas show both sensory- and motor-related activity. We show here that, in the macaque parietal association area LIP, signatures of the process of evidence accumulation that underlies the decisions are predominantly reflected in the motor-related activity. This finding supports the proposal that perceptual decision making is implemented in the brain as a process of choosing between available motor actions rather than as a process of representing the properties of the sensory stimulus.

Top-down feedback controls spatial summation and response amplitude in primate visual cortex

Sensory information travels along feedforward connections through a hierarchy of cortical areas, which, in turn, send feedback connections to lower-order areas. Feedback has been implicated in attention, expectation, and sensory context, but the mechanisms underlying these diverse feedback functions are unknown. Using specific optogenetic inactivation of feedback connections from the secondary visual area (V2), we show how feedback affects neural responses in the primate primary visual cortex (V1). Reducing feedback activity increases V1 cells' receptive field (RF) size, decreases their responses to stimuli confined to the RF, and increases their responses to stimuli extending into the proximal surround, therefore reducing surround suppression. Moreover, stronger reduction of V2 feedback activity leads to progressive increase in RF size and decrease in response amplitude, an effect predicted by a recurrent network model. Our results indicate that feedback modulates RF size, surround suppression and response amplitude, similar to the modulatory effects of visual spatial attention.

Near- and Far-Surround Suppression in Human Motion Discrimination

The spatial context has strong effects on visual processing. Psychophysics and modeling studies have provided evidence that the surround context can systematically modulate the perception of center stimuli. For motion direction, these center-surround interactions are considered to come from spatio-directional interactions between direction of motion tuned neurons, which are attributed to the middle temporal (MT) area. Here, we investigated through psychophysics experiments on human subjects changes with spatial separation in center-surround inhibition and motion direction interactions. Center-surround motion repulsion effects were measured under near-and far-surround conditions. Using a simple physiological model of the repulsion effect we extracted theoretical population parameters of surround inhibition strength and tuning widths with spatial distance. All 11 subjects showed clear motion repulsion effects under the near-surround condition, while only 10 subjects showed clear motion repulsion effects under the far-surround condition. The model predicted human performance well. Surround inhibition under the near-surround condition was significantly stronger than that under the far-surround condition, and the tuning widths were smaller under the near-surround condition. These results demonstrate that spatial separation can both modulate the surround inhibition strength and surround to center tuning width.

Contextual modulation of sensitivity to naturalistic image structure in macaque V2

The stimulus selectivity of neurons in V1 is well known, as is the finding that their responses can be affected by visual input to areas outside of the classical receptive field. Less well understood are the ways selectivity is modified as signals propagate to visual areas beyond V1, such as V2. We recently proposed a role for V2 neurons in representing the higher order statistical dependencies found in images of naturally occurring visual texture. V2 neurons, but not V1 neurons, respond more vigorously to "naturalistic" images that contain these dependencies than to "noise" images that lack them. In this work, we examine the dependency of these effects on stimulus size. For most V2 neurons, the preference for naturalistic over noise stimuli was modest when presented in small patches and gradually strengthened with increasing size, suggesting that the mechanisms responsible for this enhanced sensitivity operate over regions of the visual field that are larger than the classical receptive field. Indeed, we found that surround suppression was stronger for noise than for naturalistic stimuli and that the preference for large naturalistic stimuli developed over a delayed time course consistent with lateral or feedback connections. These findings are compatible with a spatially broad facilitatory mechanism that is absent in V1 and suggest that a distinct role for the receptive field surround emerges in V2 along with sensitivity for more complex image structure. NEW & NOTEWORTHY The responses of neurons in visual cortex are often affected by visual input delivered to regions of the visual field outside of the conventionally defined receptive field, but the significance of such contextual modulations are not well understood outside of area V1. We studied the importance of regions beyond the receptive field in establishing a novel form of selectivity for the statistical dependencies contained in natural visual textures that first emerges in area V2.

Phase-specific Surround suppression in Mouse Primary Visual Cortex Correlates with Figure Detection Behavior Based on Phase Discontinuity

In the primary visual cortex (V1), neuronal responses to stimuli within the receptive field (RF) are modulated by stimuli in the RF surround. A common effect of surround modulation is surround suppression, which is dependent on the feature difference between stimuli within and surround the RF and is suggested to be involved in the perceptual phenomenon of figure-ground segregation. In this study, we examined the relationship between feature-specific surround suppression of V1 neurons and figure detection behavior based on figure-ground feature difference. We trained freely moving mice to perform a figure detection task using figure and ground gratings that differed in spatial phase. The performance of figure detection increased with the figure-ground phase difference, and was modulated by stimulus contrast. Electrophysiological recordings from V1 in head-fixed mice showed that the increase in phase difference between stimuli within and surround the RF caused a reduction in surround suppression, which was associated with an increase in V1 neural discrimination between stimuli with and without RF-surround phase difference. Consistent with the behavioral performance, the sensitivity of V1 neurons to RF-surround phase difference could be influenced by stimulus contrast. Furthermore, inhibiting V1 by optogenetically activating either parvalbumin (PV)- or somatostatin (SOM)-expressing inhibitory neurons both decreased the behavioral performance of figure detection. Thus, the phase-specific surround suppression in V1 represents a neural correlate of figure detection behavior based on figure-ground phase discontinuity.

Nonuniform surround suppression of visual responses in mouse V1

Complex receptive field characteristics, distributed across a population of neurons, are thought to be critical for solving perceptual inference problems that arise during motion and image segmentation. For example, in a class of neurons referred to as "end-stopped," increasing the length of stimuli outside of the bar-responsive region into the surround suppresses responsiveness. It is unknown whether these properties exist for receptive field surrounds in the mouse. We examined surround modulation in layer 2/3 neurons of the primary visual cortex in mice using two-photon calcium imaging. We found that surround suppression was significantly asymmetric in 17% of the visually responsive neurons examined. Furthermore, the magnitude of asymmetry was correlated with orientation selectivity. Our results demonstrate that neurons in mouse primary visual cortex are differentially sensitive to the addition of elements in the surround and that individual neurons can be described as being either uniformly suppressed by the surround, end-stopped, or side-stopped. NEW & NOTEWORTHY Perception of visual scenes requires active integration of both local and global features to successfully segment objects from the background. Although the underlying circuitry and development of perceptual inference is not well understood, converging evidence indicates that asymmetry and diversity in surround modulation are likely fundamental for these computations. We determined that these key features are present in the mouse. Our results support the mouse as a model to explore the neural basis and development of surround modulation as it relates to perceptual inference.

Characterizing the effects of feature salience and top-down attention in the early visual system

The visual system employs a sophisticated balance of attentional mechanisms: salient stimuli are prioritized for visual processing, yet observers can also ignore such stimuli when their goals require directing attention elsewhere. A powerful determinant of visual salience is local feature contrast: if a local region differs from its immediate surround along one or more feature dimensions, it will appear more salient. We used high-resolution functional MRI (fMRI) at 7T to characterize the modulatory effects of bottom-up salience and top-down voluntary attention within multiple sites along the early visual pathway, including visual areas V1-V4 and the lateral geniculate nucleus (LGN). Observers viewed arrays of spatially distributed gratings, where one of the gratings immediately to the left or right of fixation differed from all other items in orientation or motion direction, making it salient. To investigate the effects of directed attention, observers were cued to attend to the grating to the left or right of fixation, which was either salient or nonsalient. Results revealed reliable additive effects of top-down attention and stimulus-driven salience throughout visual areas V1-hV4. In comparison, the LGN exhibited significant attentional enhancement but was not reliably modulated by orientation- or motion-defined salience. Our findings indicate that top-down effects of spatial attention can influence visual processing at the earliest possible site along the visual pathway, including the LGN, whereas the processing of orientation- and motion-driven salience primarily involves feature-selective interactions that take place in early cortical visual areas.NEW & NOTEWORTHY While spatial attention allows for specific, goal-driven enhancement of stimuli, salient items outside of the current focus of attention must also be prioritized. We used 7T fMRI to compare salience and spatial attentional enhancement along the early visual hierarchy. We report additive effects of attention and bottom-up salience in early visual areas, suggesting that salience enhancement is not contingent on the observer's attentional state.

Circuits and Mechanisms for Surround Modulation in Visual Cortex

Surround modulation (SM) is a fundamental property of sensory neurons in many species and sensory modalities. SM is the ability of stimuli in the surround of a neuron's receptive field (RF) to modulate (typically suppress) the neuron's response to stimuli simultaneously presented inside the RF, a property thought to underlie optimal coding of sensory information and important perceptual functions. Understanding the circuit and mechanisms for SM can reveal fundamental principles of computations in sensory cortices, from mouse to human. Current debate is centered over whether feedforward or intracortical circuits generate SM, and whether this results from increased inhibition or reduced excitation. Here we present a working hypothesis, based on theoretical and experimental evidence, that SM results from feedforward, horizontal, and feedback interactions with local recurrent connections, via synaptic mechanisms involving both increased inhibition and reduced recurrent excitation. In particular, strong and balanced recurrent excitatory and inhibitory circuits play a crucial role in the computation of SM.

Systematic variation of population receptive field properties across cortical depth in human visual cortex

Receptive fields (RFs) in visual cortex are organized in antagonistic, center-surround, configurations. RF properties change systematically across eccentricity and between visual field maps. However, it is unknown how center-surround configurations are organized in human visual cortex across lamina. We use sub-millimeter resolution functional MRI at 7Tesla and population receptive field (pRF) modeling to investigate the pRF properties in primary visual cortex (V1) across cortical depth. pRF size varies according to a U-shaped function, indicating smaller pRF center size in the middle compared to superficial and deeper intra-cortical portions of V1, consistent with non-human primate neurophysiological measurements. Moreover, a similar U-shaped function is also observed for pRF surround size. However, pRF center-surround ratio remains constant across cortical depth. Simulations suggest that this pattern of results can be directly linked to the flow of signals across cortical depth, with the visual input reaching the middle of cortical depth and then spreading towards superficial and deeper layers of V1. Conversely, blood-oxygenation-level-dependent (BOLD) signal amplitude increases monotonically towards the pial surface, in line with the known vascular organization across cortical depth. Independent estimates of the haemodynamic response function (HRF) across cortical depth show that the center-surround pRF size estimates across cortical depth cannot be explained by variations in the full-width half maximum (FWHM) of the HRF.

Electrophysiology Alterations in Primary Visual Cortex Neurons of Retinal Degeneration (S334ter-line-3) Rats

The dynamic nature of the brain is critical for the success of treatments aimed at restoring vision at the retinal level. The success of these treatments relies highly on the functionality of the surviving neurons along the entire visual pathway. Electrophysiological properties at the retina level have been investigated during the progression of retinal degeneration; however, little is known about the changes in electrophysiological properties that occur in the primary visual cortex (V1) during the course of retinal degeneration. By conducting extracellular recording, we examined the electrophysiological properties of V1 in S334ter-line-3 rats (a transgenic model of retinal degeneration developed to express a rhodopsin mutation similar to that found in human retinitis pigmentosa patients). We measured the orientation tuning, spatial and temporal frequency tunings and the receptive field (RF) size for 127 V1 neurons from 11 S334ter-3 rats and 10 Long-Evans (LE) rats. V1 neurons in the S334ter-3 rats showed weaker orientation selectivity, lower optimal spatial and temporal frequency values and a smaller receptive field size compared to the LE rats. These results suggest that the visual cognitive ability significantly changes during retinal degeneration.

Selectivity and tolerance for visual texture in macaque V2

As information propagates along the ventral visual hierarchy, neuronal responses become both more specific for particular image features and more tolerant of image transformations that preserve those features. Here, we present evidence that neurons in area V2 are selective for local statistics that occur in natural visual textures, and tolerant of manipulations that preserve these statistics. Texture stimuli were generated by sampling from a statistical model, with parameters chosen to match the parameters of a set of visually distinct natural texture images. Stimuli generated with the same statistics are perceptually similar to each other despite differences, arising from the sampling process, in the precise spatial location of features. We assessed the accuracy with which these textures could be classified based on the responses of V1 and V2 neurons recorded individually in anesthetized macaque monkeys. We also assessed the accuracy with which particular samples could be identified, relative to other statistically matched samples. For populations of up to 100 cells, V1 neurons supported better performance in the sample identification task, whereas V2 neurons exhibited better performance in texture classification. Relative to V1, the responses of V2 show greater selectivity and tolerance for the representation of texture statistics.

Identifying separate components of surround suppression

Surround suppression is a well-known phenomenon in which the response to a visual stimulus is diminished by the presence of neighboring stimuli. This effect is observed in neural responses in areas such as primary visual cortex, and also manifests in visual contrast perception. Studies in animal models have identified at least two separate mechanisms that may contribute to surround suppression: one that is monocular and resistant to contrast adaptation, and another that is binocular and strongly diminished by adaptation. The current study was designed to investigate whether these two mechanisms exist in humans and if they can be identified psychophysically using eye-of-origin and contrast adaptation manipulations. In addition, we examined the prediction that the monocular suppression component is broadly tuned for orientation, while suppression between eyes is narrowly tuned. Our results confirmed that when center and surrounding stimuli were presented dichoptically (in opposite eyes), suppression was orientation-tuned. Following adaptation in the surrounding region, no dichoptic suppression was observed, and monoptic suppression no longer showed orientation selectivity. These results are consistent with a model of surround suppression that depends on both low-level and higher level components. This work provides a method to assess the separate contributions of these components during spatial context processing in human vision.

The Effects of Context and Attention on Spiking Activity in Human Early Visual Cortex

Here we report the first quantitative analysis of spiking activity in human early visual cortex. We recorded multi-unit activity from two electrodes in area V2/V3 of a human patient implanted with depth electrodes as part of her treatment for epilepsy. We observed well-localized multi-unit receptive fields with tunings for contrast, orientation, spatial frequency, and size, similar to those reported in the macaque. We also observed pronounced gamma oscillations in the local-field potential that could be used to estimate the underlying spiking response properties. Spiking responses were modulated by visual context and attention. We observed orientation-tuned surround suppression: responses were suppressed by image regions with a uniform orientation and enhanced by orientation contrast. Additionally, responses were enhanced on regions that perceptually segregated from the background, indicating that neurons in the human visual cortex are sensitive to figure-ground structure. Spiking responses were also modulated by object-based attention. When the patient mentally traced a curve through the neurons' receptive fields, the accompanying shift of attention enhanced neuronal activity. These results demonstrate that the tuning properties of cells in the human early visual cortex are similar to those in the macaque and that responses can be modulated by both contextual factors and behavioral relevance. Our results, therefore, imply that the macaque visual system is an excellent model for the human visual cortex.

Canonical computations of cerebral cortex

The idea that there is a fundamental cortical circuit that performs canonical computations remains compelling though far from proven. Here we review evidence for two canonical operations within sensory cortical areas: a feedforward computation of selectivity; and a recurrent computation of gain in which, given sufficiently strong external input, perhaps from multiple sources, intracortical input largely, but not completely, cancels this external input. This operation leads to many characteristic cortical nonlinearities in integrating multiple stimuli. The cortical computation must combine such local processing with hierarchical processing across areas. We point to important changes in moving from sensory cortex to motor and frontal cortex and the possibility of substantial differences between cortex in rodents vs. species with columnar organization of selectivity.

A Hierarchical Statistical Model of Natural Images Explains Tuning Properties in V2

Previous theoretical and experimental studies have demonstrated tight relationships between natural image statistics and neural representations in V1. In particular, receptive field properties similar to simple and complex cells have been shown to be inferable from sparse coding of natural images. However, whether such a relationship exists in higher areas has not been clarified. To address this question for V2, we trained a sparse coding model that took as input the output of a fixed V1-like model, which was in its turn fed a large variety of natural image patches as input. After the training, the model exhibited response properties that were qualitatively and quantitatively compatible with three major neurophysiological results on macaque V2, as follows: (1) homogeneous and heterogeneous integration of local orientations (Anzai et al., 2007); (2) a wide range of angle selectivities with biased sensitivities to one component orientation (Ito and Komatsu, 2004); and (3) exclusive length and width suppression (Schmid et al., 2014). The reproducibility was stable across variations in several model parameters. Further, a formal classification of the internal representations of the model units offered detailed interpretations of the experimental data, emphasizing that a novel type of model cell that could detect a combination of local orientations converging toward a single spatial point (potentially related to corner-like features) played an important role in reproducing tuning properties compatible with V2. These results are consistent with the idea that V2 uses a sparse code of natural images. Significance statement: Sparse coding theory has successfully explained a number of receptive field properties in V1; but how about in V2? This question has recently become important since a variety of properties distinct from V1 have been discovered in V2, and thus a more integrative understanding is called for. Our study shows that a hierarchical sparse coding model of natural images explains three major response properties known in the macaque V2. We further provide a detailed analysis revealing the roles of different kinds of model cells in explaining the V2-specific properties. Our results thus offer the first sparse coding account for receptive field properties in V2 that has extensive biological relevance.

Early monocular defocus disrupts the normal development of receptive-field structure in V2 neurons of macaque monkeys

Experiencing different quality images in the two eyes soon after birth can cause amblyopia, a developmental vision disorder. Amblyopic humans show the reduced capacity for judging the relative position of a visual target in reference to nearby stimulus elements (position uncertainty) and often experience visual image distortion. Although abnormal pooling of local stimulus information by neurons beyond striate cortex (V1) is often suggested as a neural basis of these deficits, extrastriate neurons in the amblyopic brain have rarely been studied using microelectrode recording methods. The receptive field (RF) of neurons in visual area V2 in normal monkeys is made up of multiple subfields that are thought to reflect V1 inputs and are capable of encoding the spatial relationship between local stimulus features. We created primate models of anisometropic amblyopia and analyzed the RF subfield maps for multiple nearby V2 neurons of anesthetized monkeys by using dynamic two-dimensional noise stimuli and reverse correlation methods. Unlike in normal monkeys, the subfield maps of V2 neurons in amblyopic monkeys were severely disorganized: subfield maps showed higher heterogeneity within each neuron as well as across nearby neurons. Amblyopic V2 neurons exhibited robust binocular suppression and the strength of the suppression was positively correlated with the degree of hereogeneity and the severity of amblyopia in individual monkeys. Our results suggest that the disorganized subfield maps and robust binocular suppression of amblyopic V2 neurons are likely to adversely affect the higher stages of cortical processing resulting in position uncertainty and image distortion.