Surround suppression supports second-order feature encoding by macaque V1 and V2 neurons

A filter-rectify-filter model of the tactile perception of 3D-printed, texture-defined form

We show that a two-stage filter-rectify-filter (FRF) model, previously used to explain the visual perception of texture-defined form, can also account for the tactile perception of texture-defined form. This result is interesting because, first, relatively little is known about the neural mechanisms of tactile form perception, and second, the generalization of the model may reflect a canonical computation at work in both visual and somatosensory cortex. We 3D-printed test objects comprising a regular, rectangular array of raised, oriented bars measuring 0.75 × 0.75 × 3 mm (width × height × length) that were centre-to-centre spaced by 4 mm. Bars on the left-hand-side of a test object were horizontal, and those on the right were vertical, thus defining a texture boundary. We independently jittered the orientations of bars by drawing random numbers from a uniform distribution; across trials, we systematically increased jitter from 0° (i.e., no jitter) to ±90° (i.e., no boundary). Blindfolded participants (n = 25) used the preferred index finger pad to actively scan objects for 10 seconds before reporting the texture boundary's orientation (vertical or horizontal; randomised across trials). Results showed a threshold jitter of ±52.7° (i.e., the jitter at which the boundary orientation was only just discriminable). Computational modelling indicated that the first stage of the FRF model is a Gabor function tuned to spatial frequency = 0.23 cycles per mm with extent = 2.53 mm (full-width at half-maximum). We discuss this result with regard to neuronal receptive field structure in non-human primate somatosensory cortex.

Macaque V1 responses to 2nd-order contrast-modulated stimuli and the possible subcortical and cortical contributions

Natural images comprise contours and boundaries defined by 1st-order luminance-modulated (LM) cues that are readily encoded by V1 neurons, and 2nd-order contrast-modulated (CM) cues that carry local, but not over-the-space, luminance changes. The neurophysiological foundations for CM processing remain unsolved. Here we used two-photon calcium imaging to demonstrate that V1 superficial-layer neurons respond to both LM and CM gratings in awake, fixating, macaques, with overall LM responses stronger than CM responses. Furthermore, adaptation experiments revealed that LM responses were similarly suppressed by LM and CM adaptation, with moderately larger effects by iso-orientation adaptation than by orthogonal adaptation, suggesting that LM and CM orientation responses likely share a strong orientation-non-selective subcortical origin. In contrast, CM responses were substantially more suppressed by iso-orientation than by orthogonal LM and CM adaptation, likely suggesting stronger orientation-specific intracortical influences for CM responses than for LM responses, besides shared orientation-non-selective subcortical influences. These results thus may indicate a subcortical-to-V1 filter-rectify-filter mechanism for CM processing: Local luminance changes in CM stimuli are initially encoded by orientation-non-selective subcortical neurons, and the outputs are half-wave rectified, and then summed by V1 neurons to signal CM orientation, which may be further substantially refined by intracortical influences.

The Mechanism of Macular Sparing

Patients with homonymous hemianopia sometimes show preservation of the central visual fields, ranging up to 10°. This phenomenon, known as macular sparing, has sparked perpetual controversy. Two main theories have been offered to explain it. The first theory proposes a dual representation of the macula in each hemisphere. After loss of one occipital lobe, the back-up representation in the remaining occipital lobe is postulated to sustain ipsilateral central vision in the blind hemifield. This theory is supported by studies showing that some midline retinal ganglion cells project to the wrong hemisphere, presumably driving neurons in striate cortex that have ipsilateral receptive fields. However, more recent electrophysiological recordings and neuroimaging studies have cast doubt on this theory by showing only a minuscule ipsilateral field representation in early visual cortical areas. The second theory holds that macular sparing arises because the occipital pole, where the macula is represented, remains perfused after occlusion of the posterior cerebral artery because it receives collateral flow from the middle cerebral artery. An objection to this theory is that it cannot account for reports of macular sparing in patients after loss of an entire occipital lobe. On close scrutiny, such reports turn out to be erroneous, arising from inadequate control of fixation during visual field testing. Patients seem able to detect test stimuli on their blind side within the macula or along the vertical meridian because they make surveillance saccades. A purported treatment for hemianopia, called vision restoration therapy, is based on this error. The dual perfusion theory is supported by anatomical studies showing that the middle cerebral artery perfuses the occipital pole in many individuals. In patients with hemianopia from stroke, neuroimaging shows preservation of the occipital pole when macular sparing is present. The frontier dividing the infarcted territory of the posterior cerebral artery and the preserved territory of the middle cerebral artery is variable, but always falls within the representation of the macula, because the macula is so highly magnified. For physicians, macular sparing was an important neurological sign in acute hemianopia because it signified a posterior cerebral artery occlusion. Modern neuroimaging has supplanted the importance of that clinical sign but at the same time confirmed its validity. For patients, macular sparing remains important because it mitigates the impact of hemianopia and preserves the ability to read fluently. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Distinct spatiotemporal mechanisms underlie extra-classical receptive field modulation in macaque V1 microcircuits

Complex scene perception depends upon the interaction between signals from the classical receptive field (CRF) and the extra-classical receptive field (eCRF) in primary visual cortex (V1) neurons. Although much is known about V1 eCRF properties, we do not yet know how the underlying mechanisms map onto the cortical microcircuit. We probed the spatio-temporal dynamics of eCRF modulation using a reverse correlation paradigm, and found three principal eCRF mechanisms: tuned-facilitation, untuned-suppression, and tuned-suppression. Each mechanism had a distinct timing and spatial profile. Laminar analysis showed that the timing, orientation-tuning, and strength of eCRF mechanisms had distinct signatures within magnocellular and parvocellular processing streams in the V1 microcircuit. The existence of multiple eCRF mechanisms provides new insights into how V1 responds to spatial context. Modeling revealed that the differences in timing and scale of these mechanisms predicted distinct patterns of net modulation, reconciling many previous disparate physiological and psychophysical findings.

Contingent adaptation in masking and surround suppression

Adaptation is the process that changes a neuron's response based on recent inputs. In the traditional model, a neuron's state of adaptation depends on the recent input to that neuron alone, whereas in a recently introduced model (Hebbian normalization), adaptation depends on the structure of neural correlated firing. In particular, increased response products between pairs of neurons leads to increased mutual suppression. We test a psychophysical prediction of this model: adaptation should depend on 2nd-order statistics of input stimuli. That is, if two stimuli excite two distinct sub-populations of neurons, then presenting those stimuli simultaneously during adaptation should strengthen mutual suppression between those subpopulations. We confirm this prediction in two experiments. In the first, pairing two gratings synchronously during adaptation (i.e., a plaid) rather than asynchronously (interleaving the two gratings in time) leads to increased effectiveness of one pattern for masking the other. In the second, pairing the gratings in a center-surround configuration results in reduced apparent contrast for the central grating when paired with the same surround (as compared with a condition in which the central grating appears with a different surround at test than during adaptation). These results are consistent with the prediction that an increase in response covariance leads to greater mutual suppression between neurons. This effect is detectable both at threshold (masking) and well above threshold (apparent contrast).

Second-order visual sensitivity in the aging population

Most visual and cognitive functions are affected by aging over the lifespan. In this study, our aim was to investigate the loss in sensitivity to different classes of second-order stimuli-a class of stimuli supposed to be mainly processed in extrastriate cortex-in the aging population. These stimuli will then allow one to identify specific cortical deficit independently of visibility losses in upstream parts of the visual pathway. For this purpose, we measured the sensitivity to first-order stimuli and second-order stimuli: orientation-modulated, motion-modulated or contrast-modulated as a function of spatial frequency in 50 aged participants. Overall, we observed a sensitivity loss for all classes of stimuli, but this loss differentially affects the three classes of second-order stimuli tested. It involves all modulation spatial frequencies in the case of motion modulation, but just high modulation spatial frequencies in the case of contrast- and orientation modulations. These observations imply that aging selectively affects the sensitivity to second-order stimuli depending on their type. Since there is evidence that these different second-order stimuli are processed in different regions of extrastriate cortex, this result may suggest that some visual cortical areas are more susceptible to aging effects than others.

Stimulus vignetting and orientation selectivity in human visual cortex

Neural selectivity to orientation is one of the simplest and most thoroughly-studied cortical sensory features. Here, we show that a large body of research that purported to measure orientation tuning may have in fact been inadvertently measuring sensitivity to second-order changes in luminance, a phenomenon we term 'vignetting'. Using a computational model of neural responses in primary visual cortex (V1), we demonstrate the impact of vignetting on simulated V1 responses. We then used the model to generate a set of predictions, which we confirmed with functional MRI experiments in human observers. Our results demonstrate that stimulus vignetting can wholly determine the orientation selectivity of responses in visual cortex measured at a macroscopic scale, and suggest a reinterpretation of a well-established literature on orientation processing in visual cortex.

Contextual modulation of sensitivity to naturalistic image structure in macaque V2

The stimulus selectivity of neurons in V1 is well known, as is the finding that their responses can be affected by visual input to areas outside of the classical receptive field. Less well understood are the ways selectivity is modified as signals propagate to visual areas beyond V1, such as V2. We recently proposed a role for V2 neurons in representing the higher order statistical dependencies found in images of naturally occurring visual texture. V2 neurons, but not V1 neurons, respond more vigorously to "naturalistic" images that contain these dependencies than to "noise" images that lack them. In this work, we examine the dependency of these effects on stimulus size. For most V2 neurons, the preference for naturalistic over noise stimuli was modest when presented in small patches and gradually strengthened with increasing size, suggesting that the mechanisms responsible for this enhanced sensitivity operate over regions of the visual field that are larger than the classical receptive field. Indeed, we found that surround suppression was stronger for noise than for naturalistic stimuli and that the preference for large naturalistic stimuli developed over a delayed time course consistent with lateral or feedback connections. These findings are compatible with a spatially broad facilitatory mechanism that is absent in V1 and suggest that a distinct role for the receptive field surround emerges in V2 along with sensitivity for more complex image structure. NEW & NOTEWORTHY The responses of neurons in visual cortex are often affected by visual input delivered to regions of the visual field outside of the conventionally defined receptive field, but the significance of such contextual modulations are not well understood outside of area V1. We studied the importance of regions beyond the receptive field in establishing a novel form of selectivity for the statistical dependencies contained in natural visual textures that first emerges in area V2.

Altered Balance of Receptive Field Excitation and Suppression in Visual Cortex of Amblyopic Macaque Monkeys

In amblyopia, a visual disorder caused by abnormal visual experience during development, the amblyopic eye (AE) loses visual sensitivity whereas the fellow eye (FE) is largely unaffected. Binocular vision in amblyopes is often disrupted by interocular suppression. We used 96-electrode arrays to record neurons and neuronal groups in areas V1 and V2 of six female macaque monkeys (Macaca nemestrina) made amblyopic by artificial strabismus or anisometropia in early life, as well as two visually normal female controls. To measure suppressive binocular interactions directly, we recorded neuronal responses to dichoptic stimulation. We stimulated both eyes simultaneously with large sinusoidal gratings, controlling their contrast independently with raised-cosine modulators of different orientations and spatial frequencies. We modeled each eye's receptive field at each cortical site using a difference of Gaussian envelopes and derived estimates of the strength of central excitation and surround suppression. We used these estimates to calculate ocular dominance separately for excitation and suppression. Excitatory drive from the FE dominated amblyopic visual cortex, especially in more severe amblyopes, but suppression from both the FE and AEs was prevalent in all animals. This imbalance created strong interocular suppression in deep amblyopes: increasing contrast in the AE decreased responses at binocular cortical sites. These response patterns reveal mechanisms that likely contribute to the interocular suppression that disrupts vision in amblyopes.SIGNIFICANCE STATEMENT Amblyopia is a developmental visual disorder that alters both monocular vision and binocular interaction. Using microelectrode arrays, we examined binocular interaction in primary visual cortex and V2 of six amblyopic macaque monkeys (Macaca nemestrina) and two visually normal controls. By stimulating the eyes dichoptically, we showed that, in amblyopic cortex, the binocular combination of signals is altered. The excitatory influence of the two eyes is imbalanced to a degree that can be predicted from the severity of amblyopia, whereas suppression from both eyes is prevalent in all animals. This altered balance of excitation and suppression reflects mechanisms that may contribute to the interocular perceptual suppression that disrupts vision in amblyopes.

Cross-orientation suppression in visual area V2

Object recognition relies on a series of transformations among which only the first cortical stage is relatively well understood. Already at the second stage, the visual area V2, the complexity of the transformation precludes a clear understanding of what specifically this area computes. Previous work has found multiple types of V2 neurons, with neurons of each type selective for multi-edge features. Here we analyse responses of V2 neurons to natural stimuli and find three organizing principles. First, the relevant edges for V2 neurons can be grouped into quadrature pairs, indicating invariance to local translation. Second, the excitatory edges have nearby suppressive edges with orthogonal orientations. Third, the resulting multi-edge patterns are repeated in space to form textures or texture boundaries. The cross-orientation suppression increases the sparseness of responses to natural images based on these complex forms of feature selectivity while allowing for multiple scales of position invariance.

V2 neurons exhibit complex and diverse selectivity for visual features. Here the authors use a statistical analytical framework to model V2 responses to natural stimuli and find three organizing principles, chief among them is the cross-orientation suppression that increases response selectivity.

A Mechanosensory Circuit that Mixes Opponent Channels to Produce Selectivity for Complex Stimulus Features

Johnston's organ is the largest mechanosensory organ in Drosophila; it analyzes movements of the antenna due to sound, wind, gravity, and touch. Different Johnston's organ neurons (JONs) encode distinct stimulus features. Certain JONs respond in a sustained manner to steady displacements, and these JONs subdivide into opponent populations that prefer push or pull displacements. Here, we describe neurons in the brain (aPN3 neurons) that combine excitation and inhibition from push/pull JONs in different ratios. Consequently, different aPN3 neurons are sensitive to movement in different parts of the antenna's range, at different frequencies, or at different amplitude modulation rates. We use a model to show how the tuning of aPN3 neurons can arise from rectification and temporal filtering in JONs, followed by mixing of JON signals in different proportions. These results illustrate how several canonical neural circuit components-rectification, opponency, and filtering-can combine to produce selectivity for complex stimulus features.

Binocular functional architecture for detection of contrast-modulated gratings

Combination of signals from the two eyes is the gateway to stereo vision. To gain insight into binocular signal processing, we studied binocular summation for luminance-modulated gratings (L or LM) and contrast-modulated gratings (CM). We measured 2AFC detection thresholds for a signal grating (0.75c/deg, 216ms) shown to one eye, both eyes, or both eyes out-of-phase. For LM and CM, the carrier noise was in both eyes, even when the signal was monocular. Mean binocular thresholds for luminance gratings (L) were 5.4dB better than monocular thresholds - close to perfect linear summation (6dB). For LM and CM the binocular advantage was again 5-6dB, even when the carrier noise was uncorrelated, anti-correlated, or at orthogonal orientations in the two eyes. Binocular combination for CM probably arises from summation of envelope responses, and not from summation of these conflicting carrier patterns. Antiphase signals produced no binocular advantage, but thresholds were about 1-3dB higher than monocular ones. This is not consistent with simple linear summation, which should give complete cancellation and unmeasurably high thresholds. We propose a three-channel model in which noisy monocular responses to the envelope are binocularly combined in a contrast-weighted sum, but also remain separately available to perception via a max operator. Vision selects the largest of the three responses. With in-phase gratings the binocular channel dominates, but antiphase gratings cancel in the binocular channel and the monocular channels mediate detection. The small antiphase disadvantage might be explained by a subtle influence of background responses on binocular and monocular detection.

Mice Can Use Second-Order, Contrast-Modulated Stimuli to Guide Visual Perception

Visual processing along the primate ventral stream takes place in a hierarchy of areas, characterized by an increase in both complexity of neuronal preferences and invariance to changes of low-level stimulus attributes. A basic type of invariance is form-cue invariance, where neurons have similar preferences in response to first-order stimuli, defined by changes in luminance, and global features of second-order stimuli, defined by changes in texture or contrast. Whether in mice, a now popular model system for early visual processing, visual perception can be guided by second-order stimuli is currently unknown. Here, we probed mouse visual perception and neural responses in areas V1 and LM using various types of second-order, contrast-modulated gratings with static noise carriers. These gratings differ in their spatial frequency composition and thus in their ability to invoke first-order mechanisms exploiting local luminance features. We show that mice can transfer learning of a coarse orientation discrimination task involving first-order, luminance-modulated gratings to the contrast-modulated gratings, albeit with markedly reduced discrimination performance. Consistent with these behavioral results, we demonstrate that neurons in area V1 and LM are less responsive and less selective to contrast-modulated than to luminance-modulated gratings, but respond with broadly similar preferred orientations. We conclude that mice can, at least in a rudimentary form, use second-order stimuli to guide visual perception.

SIGNIFICANCE STATEMENT

To extract object boundaries in natural scenes, the primate visual system does not only rely on differences in local luminance but can also take into account differences in texture or contrast. Whether the mouse, which has a much simpler visual system, can use such second-order information to guide visual perception is unknown. Here we tested mouse perception of second-order, contrast-defined stimuli and measured their neural representations in two areas of visual cortex. We find that mice can use contrast-defined stimuli to guide visual perception, although behavioral performance and neural representations were less robust than for luminance-defined stimuli. These findings shed light on basic steps of feature extraction along the mouse visual cortical hierarchy, which may ultimately lead to object recognition.

Early suppression effect in human primary visual cortex during Kanizsa illusion processing: A magnetoencephalographic evidence

Detection of illusory contours (ICs) such as Kanizsa figures is known to depend primarily upon the lateral occipital complex. Yet there is no universal agreement on the role of the primary visual cortex in this process; some existing evidence hints that an early stage of the visual response in V1 may involve relative suppression to Kanizsa figures compared with controls. Iso-oriented luminance borders, which are responsible for Kanizsa illusion, may evoke surround suppression in V1 and adjacent areas leading to the reduction in the initial response to Kanizsa figures. We attempted to test the existence, as well as to find localization and timing of the early suppression effect produced by Kanizsa figures in adult nonclinical human participants. We used two sizes of visual stimuli (4.5 and 9.0°) in order to probe the effect at two different levels of eccentricity; the stimuli were presented centrally in passive viewing conditions. We recorded magnetoencephalogram, which is more sensitive than electroencephalogram to activity originating from V1 and V2 areas. We restricted our analysis to the medial occipital area and the occipital pole, and to a 40-120 ms time window after the stimulus onset. By applying threshold-free cluster enhancement technique in combination with permutation statistics, we were able to detect the inverted IC effect-a relative suppression of the response to the Kanizsa figures compared with the control stimuli. The current finding is highly compatible with the explanation involving surround suppression evoked by iso-oriented collinear borders. The effect may be related to the principle of sparse coding, according to which V1 suppresses representations of inner parts of collinear assemblies as being informationally redundant. Such a mechanism is likely to be an important preliminary step preceding object contour detection.

Chronic recordings reveal tactile stimuli can suppress spontaneous activity of neurons in somatosensory cortex of awake and anesthetized primates

In somatosensory cortex, tactile stimulation within the neuronal receptive field (RF) typically evokes a transient excitatory response with or without postexcitatory inhibition. Here, we describe neuronal responses in which stimulation on the hand is followed by suppression of the ongoing discharge. With the use of 16-channel microelectrode arrays implanted in the hand representation of primary somatosensory cortex of New World monkeys and prosimian galagos, we recorded neuronal responses from single units and neuron clusters. In 66% of our sample, neuron activity tended to display suppression of firing when regions of skin outside of the excitatory RF were stimulated. In a small proportion of neurons, single-site indentations suppressed firing without initial increases in response to any of the tested sites on the hand. Latencies of suppressive responses to skin indentation (usually 12-34 ms) were similar to excitatory response latencies. The duration of inhibition varied across neurons. Although most observations were from anesthetized animals, we also found similar neuron response properties in one awake galago. Notably, suppression of ongoing neuronal activity did not require conditioning stimuli or multi-site stimulation. The suppressive effects were generally seen following single-site skin indentations outside of the neuron's minimal RF and typically on different digits and palm pads, which have not often been studied in this context. Overall, the characteristics of widespread suppressive or inhibitory response properties with and without initial facilitative or excitatory responses add to the growing evidence that neurons in primary somatosensory cortex provide essential processing for integrating sensory stimulation from across the hand.

Neural coding of image structure and contrast polarity of Cartesian, hyperbolic, and polar gratings in the primary and secondary visual cortex of the tree shrew

We comprehensively characterize spiking and visual evoked potential (VEP) activity in tree shrew V1 and V2 using Cartesian, hyperbolic, and polar gratings. Neural selectivity to structure of Cartesian gratings was higher than other grating classes in both visual areas. From V1 to V2, structure selectivity of spiking activity increased, whereas corresponding VEP values tended to decrease, suggesting that single-neuron coding of Cartesian grating attributes improved while the cortical columnar organization of these neurons became less precise from V1 to V2. We observed that neurons in V2 generally exhibited similar selectivity for polar and Cartesian gratings, suggesting that structure of polar-like stimuli might be encoded as early as in V2. This hypothesis is supported by the preference shift from V1 to V2 toward polar gratings of higher spatial frequency, consistent with the notion that V2 neurons encode visual scene borders and contours. Neural sensitivity to modulations of polarity of hyperbolic gratings was highest among all grating classes and closely related to the visual receptive field (RF) organization of ON- and OFF-dominated subregions. We show that spatial RF reconstructions depend strongly on grating class, suggesting that intracortical contributions to RF structure are strongest for Cartesian and polar gratings. Hyperbolic gratings tend to recruit least cortical elaboration such that the RF maps are similar to those generated by sparse noise, which most closely approximate feedforward inputs. Our findings complement previous literature in primates, rodents, and carnivores and highlight novel aspects of shape representation and coding occurring in mammalian early visual cortex.

A Hierarchical Statistical Model of Natural Images Explains Tuning Properties in V2

Previous theoretical and experimental studies have demonstrated tight relationships between natural image statistics and neural representations in V1. In particular, receptive field properties similar to simple and complex cells have been shown to be inferable from sparse coding of natural images. However, whether such a relationship exists in higher areas has not been clarified. To address this question for V2, we trained a sparse coding model that took as input the output of a fixed V1-like model, which was in its turn fed a large variety of natural image patches as input. After the training, the model exhibited response properties that were qualitatively and quantitatively compatible with three major neurophysiological results on macaque V2, as follows: (1) homogeneous and heterogeneous integration of local orientations (Anzai et al., 2007); (2) a wide range of angle selectivities with biased sensitivities to one component orientation (Ito and Komatsu, 2004); and (3) exclusive length and width suppression (Schmid et al., 2014). The reproducibility was stable across variations in several model parameters. Further, a formal classification of the internal representations of the model units offered detailed interpretations of the experimental data, emphasizing that a novel type of model cell that could detect a combination of local orientations converging toward a single spatial point (potentially related to corner-like features) played an important role in reproducing tuning properties compatible with V2. These results are consistent with the idea that V2 uses a sparse code of natural images. Significance statement: Sparse coding theory has successfully explained a number of receptive field properties in V1; but how about in V2? This question has recently become important since a variety of properties distinct from V1 have been discovered in V2, and thus a more integrative understanding is called for. Our study shows that a hierarchical sparse coding model of natural images explains three major response properties known in the macaque V2. We further provide a detailed analysis revealing the roles of different kinds of model cells in explaining the V2-specific properties. Our results thus offer the first sparse coding account for receptive field properties in V2 that has extensive biological relevance.