Phase sensitivity of complex cells in primary visual cortex

Mechanisms of Feature Selectivity and Invariance in Primary Visual Cortex

Visual object identification requires both selectivity for specific visual features that are important to the object's identity and invariance to feature manipulations. For example, a hand can be shifted in position, rotated, or contracted but still be recognized as a hand. How are the competing requirements of selectivity and invariance built into the early stages of visual processing? Typically, cells in the primary visual cortex are classified as either simple or complex. They both show selectivity for edge-orientation but complex cells develop invariance to edge position within the receptive field (spatial phase). Using a data-driven model that extracts the spatial structures and nonlinearities associated with neuronal computation, we quantitatively describe the balance between selectivity and invariance in complex cells. Phase invariance is frequently partial, while invariance to orientation and spatial frequency are more extensive than expected. The invariance arises due to two independent factors: (1) the structure and number of filters and (2) the form of nonlinearities that act upon the filter outputs. Both vary more than previously considered, so primary visual cortex forms an elaborate set of generic feature sensitivities, providing the foundation for more sophisticated object processing.

Comparison of contrast-dependent phase sensitivity in primary visual cortex of mouse, cat and macaque

Neurones in the primary visual cortex (V1) are classified into simple and complex types. Simple cells are phase-sensitive, that is, they modulate their responses according to the position and brightness polarity of edges in their receptive fields. Complex cells are phase invariant, that is, they respond to edges in their receptive fields regardless of location or brightness polarity. Simple and complex cells are quantified by the degree of sensitivity to the spatial phases of drifting sinusoidal gratings. Some V1 complex cells become more phase-sensitive at low contrasts. Here we use a standardized analysis method for data derived from grating stimuli developed for macaques to reanalyse data previously collected from cats, and also collect and analyse the responses of 73 mouse V1 neurons. The analysis provides the first consistent comparative study of contrast-dependent phase sensitivity in V1 of mouse, cat and macaque monkey.

Synaptic Basis for Contrast-Dependent Shifts in Functional Identity in Mouse V1

A central transformation that occurs within mammalian visual cortex is the change from linear, polarity-sensitive responses to nonlinear, polarity-insensitive responses. These neurons are classically labelled as either simple or complex, respectively, on the basis of their response linearity (Skottun et al., 1991). While the difference between cell classes is clear when the stimulus strength is high, reducing stimulus strength diminishes the differences between the cell types and causes some complex cells to respond as simple cells (Crowder et al., 2007; van Kleef et al., 2010; Hietanen et al., 2013). To understand the synaptic basis for this shift in behavior, we used in vivo whole-cell recordings while systematically shifting stimulus contrast. We find systematic shifts in the degree of complex cell responses in mouse primary visual cortex (V1) at the subthreshold level, demonstrating that synaptic inputs change in concert with the shifts in response linearity and that the change in response linearity is not simply due to the threshold nonlinearity. These shifts are consistent with a visual cortex model in which the recurrent amplification acts as a critical component in the generation of complex cell responses (Chance et al., 1999).

Contrast-dependent phase sensitivity in area MT of macaque visual cortex

In primate visual cortex (V1), about half the neurons are sensitive to the spatial phases of grating stimuli and generate highly modulated responses to drifting gratings (simple cells). The remaining cells show far less phase sensitivity and relatively unmodulated responses to moving gratings (complex cells). In the second visual area (V2) and the motion processing area MT (or V5), the majority of cells have unmodulated responses to drifting gratings - they are phase invariant. At just-detectable contrasts, 44% of V1 complex cells show highly modulated responses, but this contrast-dependent phase sensitivity is found in only 7% of V2 complex cells. We recorded from 149 cells in macaque MT - 142 classed as complex cells at high contrast. Approximately 14% (20/142) of MT complex cells showed significantly modulated responses to drifting gratings at just-detectable contrasts. A general feature of MT cells is that they can be divided into pattern and component selective types, but we found no correlation between this classification and contrast-dependent phase sensitivity. Phase sensitivity in MT is discussed in relation to MT's input structure.

The divisive normalization model of V1 neurons: a comprehensive comparison of physiological data and model predictions

The physiological responses of simple and complex cells in the primary visual cortex (V1) have been studied extensively and modeled at different levels. At the functional level, the divisive normalization model (DNM; Heeger DJ. Vis Neurosci 9: 181-197, 1992) has accounted for a wide range of single-cell recordings in terms of a combination of linear filtering, nonlinear rectification, and divisive normalization. We propose standardizing the formulation of the DNM and implementing it in software that takes static grayscale images as inputs and produces firing rate responses as outputs. We also review a comprehensive suite of 30 empirical phenomena and report a series of simulation experiments that qualitatively replicate dozens of key experiments with a standard parameter set consistent with physiological measurements. This systematic approach identifies novel falsifiable predictions of the DNM. We show how the model simultaneously satisfies the conflicting desiderata of flexibility and falsifiability. Our key idea is that, while adjustable parameters are needed to accommodate the diversity across neurons, they must be fixed for a given individual neuron. This requirement introduces falsifiable constraints when this single neuron is probed with multiple stimuli. We also present mathematical analyses and simulation experiments that explicate some of these constraints.

Neural coding of image structure and contrast polarity of Cartesian, hyperbolic, and polar gratings in the primary and secondary visual cortex of the tree shrew

We comprehensively characterize spiking and visual evoked potential (VEP) activity in tree shrew V1 and V2 using Cartesian, hyperbolic, and polar gratings. Neural selectivity to structure of Cartesian gratings was higher than other grating classes in both visual areas. From V1 to V2, structure selectivity of spiking activity increased, whereas corresponding VEP values tended to decrease, suggesting that single-neuron coding of Cartesian grating attributes improved while the cortical columnar organization of these neurons became less precise from V1 to V2. We observed that neurons in V2 generally exhibited similar selectivity for polar and Cartesian gratings, suggesting that structure of polar-like stimuli might be encoded as early as in V2. This hypothesis is supported by the preference shift from V1 to V2 toward polar gratings of higher spatial frequency, consistent with the notion that V2 neurons encode visual scene borders and contours. Neural sensitivity to modulations of polarity of hyperbolic gratings was highest among all grating classes and closely related to the visual receptive field (RF) organization of ON- and OFF-dominated subregions. We show that spatial RF reconstructions depend strongly on grating class, suggesting that intracortical contributions to RF structure are strongest for Cartesian and polar gratings. Hyperbolic gratings tend to recruit least cortical elaboration such that the RF maps are similar to those generated by sparse noise, which most closely approximate feedforward inputs. Our findings complement previous literature in primates, rodents, and carnivores and highlight novel aspects of shape representation and coding occurring in mammalian early visual cortex.

Functional characterization of spikelet activity in the primary visual cortex

KEY POINTS

In vivo whole-cell patch-clamp recordings in cat visual cortex revealed small deflections in the membrane potential of neurons, termed spikelets. Spikelet statistics and functional properties suggest these deflections originate from a single, nearby cell. Spikelets shared a number sensory selectivities with the principal neuron including orientation selectivity, receptive field location and eye preference. Principal neurons and spikelets did not, however, generally share preferences for depth (binocular disparity). Cross-correlation of spikelet activity and membrane potential revealed direct effects on the membrane potential of some principal neurons, suggesting that these cells were synaptically coupled or received common input from the cortical network. Other spikelet-neuron pairs revealed indirect effects, likely to be the result of correlated network events.

ABSTRACT

Intracellular recordings in the neocortex reveal not only the membrane potential of neurons, but small unipolar or bipolar deflections that are termed spikelets. Spikelets have been proposed to originate from various sources, including active dendritic mechanisms, gap junctions and extracellular signals. Here we examined the functional characteristics of spikelets measured in neurons from cat primary visual cortex in vivo. Spiking statistics and our functional characterization of spikelet activity indicate that spikelets originate from a separate, nearby cell. Spikelet kinetics and lack of a direct effect on spikelet activity from hyperpolarizing current injection suggest they do not arise from electrical coupling to the principal neuron being recorded. Spikelets exhibited matched orientation tuning preference and ocular dominance to the principal neuron. In contrast, binocular disparity preferences of spikelets and the principal neuron were unrelated. Finally, we examined the impact of spikelets on the principal neuron's membrane potential; we did observe some records for which spikelets were correlated with the membrane potential of the principal neuron, suggesting that these neurons were synaptically coupled or received common input from the cortical network.

Spatial phase sensitivity of complex cells in primary visual cortex depends on stimulus contrast

Neurons in primary visual cortex are classified as simple, which are phase sensitive, or complex, which are significantly less phase sensitive. Previously, we have used drifting gratings to show that the phase sensitivity of complex cells increases at low contrast and after contrast adaptation while that of simple cells remains the same at all contrasts (Cloherty SL, Ibbotson MR. J Neurophysiol 113: 434-444, 2015; Crowder NA, van Kleef J, Dreher B, Ibbotson MR. J Neurophysiol 98: 1155-1166, 2007; van Kleef JP, Cloherty SL, Ibbotson MR. J Physiol 588: 3457-3470, 2010). However, drifting gratings confound the influence of spatial and temporal summation, so here we have stimulated complex cells with gratings that are spatially stationary but continuously reverse the polarity of the contrast over time (contrast-reversing gratings). By varying the spatial phase and contrast of the gratings we aimed to establish whether the contrast-dependent phase sensitivity of complex cells results from changes in spatial or temporal processing or both. We found that most of the increase in phase sensitivity at low contrasts could be attributed to changes in the spatial phase sensitivities of complex cells. However, at low contrasts the complex cells did not develop the spatiotemporal response characteristics of simple cells, in which paired response peaks occur 180° out of phase in time and space. Complex cells that increased their spatial phase sensitivity at low contrasts were significantly overrepresented in the supragranular layers of cortex. We conclude that complex cells in supragranular layers of cat cortex have dynamic spatial summation properties and that the mechanisms underlying complex cell receptive fields differ between cortical layers.

Representation of higher-order statistical structures in natural scenes via spatial phase distributions

Natural scenes contain richer perceptual information in their spatial phase structure than their amplitudes. Modeling phase structure of natural scenes may explain higher-order structure inherent to the natural scenes, which is neglected in most classical models of redundancy reduction. Only recently, a few models have represented images using a complex form of receptive fields (RFs) and analyze their complex responses in terms of amplitude and phase. However, these complex representation models often tacitly assume a uniform phase distribution without empirical support. The structure of spatial phase distributions of natural scenes in the form of relative contributions of paired responses of RFs in quadrature has not been explored statistically until now. Here, we investigate the spatial phase structure of natural scenes using complex forms of various Gabor-like RFs. To analyze distributions of the spatial phase responses, we constructed a mixture model that accounts for multi-modal circular distributions, and the EM algorithm for estimation of the model parameters. Based on the likelihood, we report presence of both uniform and structured bimodal phase distributions in natural scenes. The latter bimodal distributions were symmetric with two peaks separated by about 180°. Thus, the redundancy in the natural scenes can be further removed by using the bimodal phase distributions obtained from these RFs in the complex representation models. These results predict that both phase invariant and phase sensitive complex cells are required to represent the regularities of natural scenes in visual systems.

On the role of spatial phase and phase correlation in vision, illusion, and cognition

Numerous findings indicate that spatial phase bears an important cognitive information. Distortion of phase affects topology of edge structures and makes images unrecognizable. In turn, appropriately phase-structured patterns give rise to various illusions of virtual image content and apparent motion. Despite a large body of phenomenological evidence not much is known yet about the role of phase information in neural mechanisms of visual perception and cognition. Here, we are concerned with analysis of the role of spatial phase in computational and biological vision, emergence of visual illusions and pattern recognition. We hypothesize that fundamental importance of phase information for invariant retrieval of structural image features and motion detection promoted development of phase-based mechanisms of neural image processing in course of evolution of biological vision. Using an extension of Fourier phase correlation technique, we show that the core functions of visual system such as motion detection and pattern recognition can be facilitated by the same basic mechanism. Our analysis suggests that emergence of visual illusions can be attributed to presence of coherently phase-shifted repetitive patterns as well as the effects of acuity compensation by saccadic eye movements. We speculate that biological vision relies on perceptual mechanisms effectively similar to phase correlation, and predict neural features of visual pattern (dis)similarity that can be used for experimental validation of our hypothesis of “cognition by phase correlation.”

Contrast-dependent phase sensitivity in V1 but not V2 of macaque visual cortex

Some neurons in early visual cortex are highly selective for the position of oriented edges in their receptive fields (simple cells), whereas others are largely position insensitive (complex cells). These characteristics are reflected in their sensitivity to the spatial phase of moving sine-wave gratings: simple cell responses oscillate at the fundamental frequency of the stimulus, whereas this is less so for complex cells. In primates, when assessed at high stimulus contrast, simple cells and complex cells are roughly equally represented in the first visual cortical area, V1, whereas in the second visual area, V2, the majority of cells are complex. Recent evidence has shown that phase sensitivity of complex cells is contrast dependent. This has led to speculation that reduced contrast may lead to changes in the spatial structure of receptive fields, perhaps due to changes in how feedforward and recurrent signals interact. Given the substantial interconnections between V1 and V2 and recent evidence for the emergence of unique functional capacity in V2, we assess the relationship between contrast and phase sensitivity in the two brain regions. We show that a substantial proportion of complex cells in macaque V1 exhibit significant increases in phase sensitivity at low contrast, whereas this is rarely observed in V2. Our results support a degree of hierarchical processing from V1 to V2 with the differences possibly relating to the fact that V1 combines both subcortical and cortical input, whereas V2 receives input purely from cortical circuits.

Spectral characteristics of phase sensitivity and discharge rate of neurons in the ascending tectofugal visual system

Drifting gratings can modulate the activity of visual neurons at the temporal frequency of the stimulus. In order to characterize the temporal frequency modulation in the cat’s ascending tectofugal visual system, we recorded the activity of single neurons in the superior colliculus, the suprageniculate nucleus, and the anterior ectosylvian cortex during visual stimulation with drifting sine-wave gratings. In response to such stimuli, neurons in each structure showed an increase in firing rate and/or oscillatory modulated firing at the temporal frequency of the stimulus (phase sensitivity). To obtain a more complete characterization of the neural responses in spatiotemporal frequency domain, we analyzed the mean firing rate and the strength of the oscillatory modulations measured by the standardized Fourier component of the response at the temporal frequency of the stimulus. We show that the spatiotemporal stimulus parameters that elicit maximal oscillations often differ from those that elicit a maximal discharge rate. Furthermore, the temporal modulation and discharge-rate spectral receptive fields often do not overlap, suggesting that the detection range for visual stimuli provided jointly by modulated and unmodulated response components is larger than the range provided by a one response component.

Slow feature analysis on retinal waves leads to V1 complex cells

The developing visual system of many mammalian species is partially structured and organized even before the onset of vision. Spontaneous neural activity, which spreads in waves across the retina, has been suggested to play a major role in these prenatal structuring processes. Recently, it has been shown that when employing an efficient coding strategy, such as sparse coding, these retinal activity patterns lead to basis functions that resemble optimal stimuli of simple cells in primary visual cortex (V1). Here we present the results of applying a coding strategy that optimizes for temporal slowness, namely Slow Feature Analysis (SFA), to a biologically plausible model of retinal waves. Previously, SFA has been successfully applied to model parts of the visual system, most notably in reproducing a rich set of complex-cell features by training SFA with quasi-natural image sequences. In the present work, we obtain SFA units that share a number of properties with cortical complex-cells by training on simulated retinal waves. The emergence of two distinct properties of the SFA units (phase invariance and orientation tuning) is thoroughly investigated via control experiments and mathematical analysis of the input-output functions found by SFA. The results support the idea that retinal waves share relevant temporal and spatial properties with natural visual input. Hence, retinal waves seem suitable training stimuli to learn invariances and thereby shape the developing early visual system such that it is best prepared for coding input from the natural world.

Spatiotemporal specificity of contrast adaptation in mouse primary visual cortex

Prolonged viewing of high contrast gratings alters perceived stimulus contrast, and produces characteristic changes in the contrast response functions of neurons in the primary visual cortex (V1). This is referred to as contrast adaptation. Although contrast adaptation has been well-studied, its underlying neural mechanisms are not well-understood. Therefore, we investigated contrast adaptation in mouse V1 with the goal of establishing a quantitative description of this phenomenon in a genetically manipulable animal model. One interesting aspect of contrast adaptation that has been observed both perceptually and in single unit studies is its specificity for the spatial and temporal characteristics of the stimulus. Therefore, in the present work we determined if the magnitude of contrast adaptation in mouse V1 neurons was dependent on the spatial frequency and temporal frequency of the adapting grating. We used protocols that were readily comparable with previous studies in cats and primates, and also a novel contrast ramp stimulus that characterized the spatial and temporal specificity of contrast adaptation simultaneously. Similar to previous work in higher mammals, we found that contrast adaptation was strongest when the spatial frequency and temporal frequency of the adapting grating matched the test stimulus. This suggests similar mechanisms underlying contrast adaptation across animal models and indicates that the rapidly advancing genetic tools available in mice could be used to provide insights into this phenomenon.

On the relation between receptive field structure and stimulus selectivity in the tree shrew primary visual cortex

There are notable differences in functional properties of primary visual cortex (V1) neurons among mammalian species, particularly those concerning the occurrence of simple and complex cells and the generation of orientation selectivity. Here, we present quantitative data on receptive field (RF) structure, response modulation, and orientation tuning for single neurons in V1 of the tree shrew, a close relative of primates. We find that spatial RF subfield segregation, a criterion for identifying simple cells, was exceedingly small in the tree shrew V1. In contrast, many neurons exhibited elevated F1/F0 modulation that is often used as a simple cell marker. This apparent discrepancy can be explained by the robust stimulus polarity preference in tree shrew V1, which inflates F1/F0 ratio values. RF structure mapped with sparse-noise-which is spatially restricted and emphasizes thalamo-cortical feed-forward inputs-appeared unrelated to orientation selectivity. However, RF structure mapped using the Hartley subspace stimulus-which covers a large area of the visual field and recruits considerable intracortical processing-did predict orientation preference. Our findings reveal a number of striking similarities in V1 functional organization between tree shrews and primates, emphasizing the important role of intracortical recurrent processing in shaping V1 response properties in these species.