Representation of spatial frequency and orientation in the visual cortex

A neural network model of the effect of prior experience with regularities on subsequent category learning

Categories are often structured by the similarities of instances within the category defined across dimensions or features. Researchers typically assume that there is a direct, linear relationship between the physical input dimensions across which category exemplars are defined and the psychological representation of these dimensions. However, this assumption is not always warranted. Through a set of simulations, we demonstrate that the psychological representations of input dimensions developed through long-term prior experience can place very strong constraints on category learning. We compare the model's behavior to auditory, visual, and cross-modal human category learning and make conclusions regarding the nature of the psychological representations of the dimensions in those studies. These simulations support the conclusion that the nature of psychological representations of input dimensions is a critical aspect to understanding the mechanisms underlying category learning.

Functional organization of spatial frequency tuning in macaque V1 revealed with two-photon calcium imaging

V1 neurons are functionally organized in orientation columns in primates. Whether spatial frequency (SF) columns also exist is less clear because mixed results have been reported. A definitive solution would be SF functional maps at single-neuron resolution. Here we used two-photon calcium imaging to construct first cellular SF maps in V1 superficial layers of five awake fixating macaques, and studied SF functional organization properties and neuronal tuning characteristics. The SF maps (850 × 850 μm²) showed weak horizontal SF clustering (median clustering index = 1.43 vs. unity baseline), about one sixth as strong as orientation clustering in the same sets of neurons, which argues against a meaningful orthogonal relationship between orientation and SF functional maps. These maps also displayed nearly absent vertical SF clustering between two cortical depths (150 & 300 μm), indicating a lack of SF columnar structures within the superficial layers. The underlying causes might be that most neurons were tuned to a narrow two-octave range of medium frequencies, and many neurons with different SF preferences were often spatially mixed, which disallowed finer grouping of SF tuning. In addition, individual SF tuning functions were often asymmetric, having wider lower frequency branches, which may help encode low SF information for later decoding.

Cultural Differences in Face Recognition and Potential Underlying Mechanisms

The ability to recognize a face is crucial for the success of social interactions. Understanding the visual processes underlying this ability has been the focus of a long tradition of research. Recent advances in the field have revealed that individuals having different cultural backgrounds differ in the type of visual information they use for face processing. However, the mechanisms that underpin these differences remain unknown. Here, we revisit recent findings highlighting group differences in face processing. Then, we integrate these results in a model of visual categorization developed in the field of psychophysics: the RAP framework. On the basis of this framework, we discuss potential mechanisms, whether face-specific or not, that may underlie cross-cultural differences in face perception.

The perception threshold of the panda illusion, a particular form of 2D pulse-width-modulated halftone, correlates with visual acuity

To call attention to the danger of extinction of the panda bear, the Lithuanian artist Ilja Klemencov created the artwork “They can disappear”. The illustration is composed of black-and-white zigzagged lines, which form the famous panda logo of the World Wild Fund For Nature (WWF) when seen from a distance. If one is too close to the artwork, it is difficult to spot the bear, however, if one steps back or takes off one’s glasses the panda suddenly appears. This led us to ask if the ability to see the panda is related to the visual acuity of the observer and if therefore, the panda illusion can be used to assess the spatial resolution of the eye. Here we present the results of the comparison between visual acuity determined using the Landolt C and that predicted from the panda illusion in 23 healthy volunteers with artificially reduced visual acuity. Furthermore, we demonstrate that the panda illusion is based on a 2D pulse-width modulation, explain its technical history, and provide the equations required to create the illusion. Finally, we explain why the illusion indeed can be used to predict visual acuity and discuss the neural causes of its perception with best-corrected visual acuity.

Local organization of spatial frequency tuning dynamics in the cat visual areas 17 and 18

Spatial frequency (SF) is a prominent feature to which most neurons in cat areas 17 and 18 (area 17/18) exhibit tuning selectivity. Previous studies have shown that neurons with similar SF tunings are locally clustered into SF preference domains. However, the functional organization of SF tuning remains not fully understood. Neurons in these areas show a variety of SF tuning dynamics; however, it is unknown how neurons with diverse dynamics are locally organized to form the population dynamics of the domains. The laminar organization of SF dynamics is also unknown, knowledge of which may be useful for determining how SF tuning dynamics of cat area 17/18 neurons arise in cortical circuits. To address these issues, we recorded the activities of multiple neurons in the cat area 17/18 using microelectrode arrays and characterized the time courses of the SF tunings of these neurons by a subspace reverse correlation. A wide range of SF dynamics was already present in the input layer, suggesting that intracortical mechanisms contribute to generating SF dynamics inside this layer but do not further shape it outside this layer. Local neuronal pools with similar SF tunings contained diverse SF dynamics. The average preferred SF of a pool similarly increased with response time. Moreover, the range of single-neuron preferred SFs in a pool tended to increase with time. Our results suggest that, in the presence of organized tuning diversity within an SF domain, the cortical SF organization remains stable during response time in cat area 17/18.NEW & NOTEWORTHY In cat area 17/18, we found that a local pool of neurons with similar spatial frequency (SF) tunings shows diverse but organized dynamics. Our results suggest that, in the presence of organized tuning diversity within an SF domain, the cortical SF organization remains stable over response time in these areas. Laminar analysis suggests that intracortical mechanisms contribute to generating SF dynamics inside the input layer but do not further shape it outside this layer.

Population spatial frequency tuning in human early visual cortex

Neurons within early visual cortex are selective for basic image statistics, including spatial frequency. However, these neurons are thought to act as band-pass filters, with the window of spatial frequency sensitivity varying across the visual field and across visual areas. Although a handful of previous functional (f)MRI studies have examined human spatial frequency sensitivity using conventional designs and analysis methods, these measurements are time consuming and fail to capture the precision of spatial frequency tuning (bandwidth). In this study, we introduce a model-driven approach to fMRI analyses that allows for fast and efficient estimation of population spatial frequency tuning (pSFT) for individual voxels. Blood oxygen level-dependent (BOLD) responses within early visual cortex were acquired while subjects viewed a series of full-field stimuli that swept through a large range of spatial frequency content. Each stimulus was generated by band-pass filtering white noise with a central frequency that changed periodically between a minimum of 0.5 cycles/degree (cpd) and a maximum of 12 cpd. To estimate the underlying frequency tuning of each voxel, we assumed a log-Gaussian pSFT and optimized the parameters of this function by comparing our model output against the measured BOLD time series. Consistent with previous studies, our results show that an increase in eccentricity within each visual area is accompanied by a drop in the peak spatial frequency of the pSFT. Moreover, we found that pSFT bandwidth depends on eccentricity and is correlated with the pSFT peak; populations with lower peaks possess broader bandwidths in logarithmic scale, whereas in linear scale this relationship is reversed.NEW & NOTEWORTHY Spatial frequency selectivity is a hallmark property of early visuocortical neurons, and mapping these sensitivities gives us crucial insight into the hierarchical organization of information within visual areas. Due to technical obstacles, we lack a comprehensive picture of the properties of this sensitivity in humans. Here, we introduce a new method, coined population spatial frequency tuning mapping, which circumvents the limitations of the conventional neuroimaging methods, yielding a fuller visuocortical map of spatial frequency sensitivity.

Organization of spatial frequency in cat striate cortex

Primary visual cortex, the first cortical stage of visual information processing, is represented by diverse functional maps that demonstrate the selectivity for specific visual features such as spatial frequency (SF). Although the local organization of SF maps in cat area 17 (A17) has been largely investigated, the global arrangement remains elusive. To address this unclear aspect, we evaluated the organization of SF maps within A17 by intrinsic signal optical imaging and extracellular electrophysiological recording. Our results explicitly showed that SF organization in cat A17 displayed a global asymmetrical unimodal distribution. In particular, we found the highest SF preference within the global distribution concentrated around the horizontal meridian. These results significantly contribute to a more comprehensive understanding of the SF organization in visual cortex.

A Multitaper, Causal Decomposition for Stochastic, Multivariate Time Series: Application to High-Frequency Calcium Imaging Data

Neural data analysis has increasingly incorporated causal information to study circuit connectivity. Dimensional reduction forms the basis of most analyses of large multivariate time series. Here, we present a new, multitaper-based decomposition for stochastic, multivariate time series that acts on the covariance of the time series at all lags, C(τ), as opposed to standard methods that decompose the time series, X(t), using only information at zero-lag. In both simulated and neural imaging examples, we demonstrate that methods that neglect the full causal structure may be discarding important dynamical information in a time series.

Differences in spatial and temporal frequency interactions between central and peripheral parts of the feline area 18

The visual system demonstrates significant differences in information processing abilities between the central and peripheral parts of the visual field. Optical imaging based on intrinsic signals was used to investigate the difference in stimulus spatial and temporal frequency interactions related to receptive field eccentricity in the cat area 18. Changing either the spatial or the temporal frequency of grating stimuli had a significant impact on responses in the cortical areas corresponding to the centre of the visual field and more peripheral parts at 10 degrees eccentricity. The cortical region corresponding to the centre of the gaze was tuned to 0.4 cycles per degree (c/deg) for spatial frequency and 2 Hz for temporal frequency. In contrast, the cortical region corresponding to the periphery of the visual field was tuned to a lower spatial frequency of 0.15 c/deg and a higher temporal frequency of 4 Hz. Interestingly, when we simultaneously changed both the spatial frequency and the temporal frequency of the grating stimuli, the responses were significantly different from those estimated with an assumption of independence between the spatial and temporal frequency in the cortical region corresponding to the periphery of the visual field. However, in the cortical area corresponding to the centre of the gaze, spatial frequency showed significant independence from temporal frequency. These properties support the notion of relative specialization of visual information processing for peripheral representations in cortical areas.

Dimensional reduction of a V1 ring model with simple and complex cells

In this paper, we extend a framework for constructing low-dimensional dynamical systems models of mammalian primary visual cortex to a cortical network model that incorporates the full nonlinear effects of complex cells. The procedure consists of capturing the essential dynamics in a low-dimensional subspace using empirical methods, then recasting the equations in the reduced vector space. Previously, we considered visual cortical network models consisting of only simple cells with nearly linear responses to external stimuli. Here we show that fully nonlinear effects can be incorporated by examining the dimensional reduction of an idealized ring model of V1 with both simple and complex cells. We found it expedient to divide the subspace into four separate neuronal populations: excitatory simple, excitatory complex, inhibitory simple and inhibitory complex. In order to reproduce the fluctuation-driven dynamics in this reduced space, we incorporated (1) white noises with different intensities into individual neuronal populations, and (2) firing rate estimates to capture the probability of firing due to subthreshold fluctuations. With a more accurate, fitted connectivity, our modified dimensional reduced models can reproduce the firing rates, circular variances and modulation ratios observed in the original ring model.

The neural bases of spatial frequency processing during scene perception

Theories on visual perception agree that scenes are processed in terms of spatial frequencies. Low spatial frequencies (LSF) carry coarse information whereas high spatial frequencies (HSF) carry fine details of the scene. However, how and where spatial frequencies are processed within the brain remain unresolved questions. The present review addresses these issues and aims to identify the cerebral regions differentially involved in low and high spatial frequency processing, and to clarify their attributes during scene perception. Results from a number of behavioral and neuroimaging studies suggest that spatial frequency processing is lateralized in both hemispheres, with the right and left hemispheres predominantly involved in the categorization of LSF and HSF scenes, respectively. There is also evidence that spatial frequency processing is retinotopically mapped in the visual cortex. HSF scenes (as opposed to LSF) activate occipital areas in relation to foveal representations, while categorization of LSF scenes (as opposed to HSF) activates occipital areas in relation to more peripheral representations. Concomitantly, a number of studies have demonstrated that LSF information may reach high-order areas rapidly, allowing an initial coarse parsing of the visual scene, which could then be sent back through feedback into the occipito-temporal cortex to guide finer HSF-based analysis. Finally, the review addresses spatial frequency processing within scene-selective regions areas of the occipito-temporal cortex.

An adaptable neuromorphic model of orientation selectivity based on floating gate dynamics

The biggest challenge that the neuromorphic community faces today is to build systems that can be considered truly cognitive. Adaptation and self-organization are the two basic principles that underlie any cognitive function that the brain performs. If we can replicate this behavior in hardware, we move a step closer to our goal of having cognitive neuromorphic systems. Adaptive feature selectivity is a mechanism by which nature optimizes resources so as to have greater acuity for more abundant features. Developing neuromorphic feature maps can help design generic machines that can emulate this adaptive behavior. Most neuromorphic models that have attempted to build self-organizing systems, follow the approach of modeling abstract theoretical frameworks in hardware. While this is good from a modeling and analysis perspective, it may not lead to the most efficient hardware. On the other hand, exploiting hardware dynamics to build adaptive systems rather than forcing the hardware to behave like mathematical equations, seems to be a more robust methodology when it comes to developing actual hardware for real world applications. In this paper we use a novel time-staggered Winner Take All circuit, that exploits the adaptation dynamics of floating gate transistors, to model an adaptive cortical cell that demonstrates Orientation Selectivity, a well-known biological phenomenon observed in the visual cortex. The cell performs competitive learning, refining its weights in response to input patterns resembling different oriented bars, becoming selective to a particular oriented pattern. Different analysis performed on the cell such as orientation tuning, application of abnormal inputs, response to spatial frequency and periodic patterns reveal close similarity between our cell and its biological counterpart. Embedded in a RC grid, these cells interact diffusively exhibiting cluster formation, making way for adaptively building orientation selective maps in silicon.

Genomic data and disease forecasting: application to type 2 diabetes (T2D)

A general approach is presented for the extraction of a classifier of disease risk that is latent in large scale disease/control databases. Novel features are the following: (1) a data reorganization into a regularized standard form that emphasizes individual alleles instead of the single nucleotide polymorphism (Snp) allele pair to which they belong; (2) from this a procedure that significantly enhances the discovery of high value genomic loci; (3) an investigative analysis based on the hypothesis that disease represents a very small signal (small signal-to-noise) that is latent in the data. The resulting analyses applied to the FUSION T2D database leads to the polling of thousands of genomic loci to classify disease. This large genomic kernel of loci is shared by non-diabetics at nearly the same high level; but a small well defined separation exists and it is speculated that this might be due to unconventional disease mechanisms. Another analysis demonstrates that the FUSION database size limits its disease predictability, and only one third of the resulting classifier loci are estimated to relate to T2D. The remainder is associated with hidden features that might contrast the disease and control populations and that more data would eliminate.

Organization and origin of spatial frequency maps in cat visual cortex

It remains controversial whether and how spatial frequency (SF) is represented tangentially in cat visual cortex. Several models were proposed, but there is no consensus. Worse still, some data indicate that the SF organization previously revealed by optical imaging techniques simply reflects non-stimulus-specific responses. Instead, stimulus-specific responses arise from the homogeneous distribution of geniculo-cortical afferents representing X and Y pathways. To clarify this, we developed a new imaging method allowing rapid stimulation with a wide range of SFs covering more than 6 octaves with only 0.2 octave resolution. A benefit of this method is to avoid error of high-pass filtering methods which systematically under-represent dominant selectivity features near pinwheel centers. We show unequivocally that SF is organized into maps in cat area 17 (A17) and area 18 (A18). The SF organization in each area displays a global anteroposterior SF gradient and local patches. Its layout is constrained to that of the orientation map, and it is suggested that both maps share a common functional architecture. A17 and A18 are bound at the transition zone by another SF gradient involving the geniculo-cortical and the callosal pathways. A model based on principal component analysis shows that SF maps integrate three different SF-dependent channels. Two of these reflect the segregated excitatory input from X and Y geniculate cells to A17 and A18. The third one conveys a specific combination of excitatory and suppressive inputs to the visual cortex. In a manner coherent with anatomical and electrophysiological data, it is interpreted as originating from a subtype of Y geniculate cells.

Temporally-structured acquisition of multidimensional optical imaging data facilitates visualization of elusive cortical representations in the behaving monkey

Fundamental understanding of higher cognitive functions can greatly benefit from imaging of cortical activity with high spatiotemporal resolution in the behaving non-human primate. To achieve rapid imaging of high-resolution dynamics of cortical representations of spontaneous and evoked activity, we designed a novel data acquisition protocol for sensory stimulation by rapidly interleaving multiple stimuli in continuous sessions of optical imaging with voltage-sensitive dyes. We also tested a new algorithm for the "temporally structured component analysis" (TSCA) of a multidimensional time series that was developed for our new data acquisition protocol, but was tested only on simulated data (Blumenfeld, 2010). In addition to the raw data, the algorithm incorporates prior knowledge about the temporal structure of the data as well as input from other information. Here we showed that TSCA can successfully separate functional signal components from other signals referred to as noise. Imaging of responses to multiple visual stimuli, utilizing voltage-sensitive dyes, was performed on the visual cortex of awake monkeys. Multiple cortical representations, including orientation and ocular dominance maps as well as the hitherto elusive retinotopic representation of orientation stimuli, were extracted in only 10s of imaging, approximately two orders of magnitude faster than accomplished by conventional methods. Since the approach is rather general, other imaging techniques may also benefit from the same stimulation protocol. This methodology can thus facilitate rapid optical imaging explorations in monkeys, rodents and other species with a versatility and speed that were not feasible before.

Retinotopic and lateralized processing of spatial frequencies in human visual cortex during scene categorization

Using large natural scenes filtered in spatial frequencies, we aimed to demonstrate that spatial frequency processing could not only be retinotopically mapped but could also be lateralized in both hemispheres. For this purpose, participants performed a categorization task using large black and white photographs of natural scenes (indoors vs. outdoors, with a visual angle of 24° × 18°) filtered in low spatial frequencies (LSF), high spatial frequencies (HSF), and nonfiltered scenes, in block-designed fMRI recording sessions. At the group level, the comparison between the spatial frequency content of scenes revealed first that, compared with HSF, LSF scene categorization elicited activation in the anterior half of the calcarine fissures linked to the peripheral visual field, whereas, compared with LSF, HSF scene categorization elicited activation in the posterior part of the occipital lobes, which are linked to the fovea, according to the retinotopic property of visual areas. At the individual level, functional activations projected on retinotopic maps revealed that LSF processing was mapped in the anterior part of V1, whereas HSF processing was mapped in the posterior and ventral part of V2, V3, and V4. Moreover, at the group level, direct interhemispheric comparisons performed on the same fMRI data highlighted a right-sided occipito-temporal predominance for LSF processing and a left-sided temporal cortex predominance for HSF processing, in accordance with hemispheric specialization theories. By using suitable method of analysis on the same data, our results enabled us to demonstrate for the first time that spatial frequencies processing is mapped retinotopically and lateralized in human occipital cortex.

Spontaneous oscillations in intrinsic signals reveal the structure of cerebral vasculature

Functional imaging of intrinsic signals allows minimally invasive spatiotemporal mapping of stimulus representations in the cortex, but representations are often corrupted by stimulus-independent spatial artifacts, especially those originating from the blood vessels. In this paper, we present novel algorithms for unsupervised identification of cerebral vascularization, allowing blind separation of stimulus representations from noise. These algorithms commonly take advantage of the temporal fluctuations in global reflectance to extract anatomic information. More specifically, the phase of low-frequency oscillations relative to global fluctuations reveals local vascular identity. Arterioles can be reconstructed using their characteristically high power in those frequencies corresponding to respiration, heartbeat, and vasomotion signals. By treating the vasculature as a dynamic flow network, we finally demonstrate that direction of blood perfusion can be quantitatively visualized. Application of these methods for removal of stimulus-independent changes in reflectance permits isolation of stimulus-evoked representations even if the representation spatially overlaps with blood vessels. The algorithms can be expanded further to extract temporal information on blood flow, monitor revascularization following a focal stroke, and distinguish arterioles from venules and parenchyma.

Orthogonal micro-organization of orientation and spatial frequency in primate primary visual cortex

Orientation and spatial frequency tuning are highly salient properties of neurons in primary visual cortex (V1). The combined organization of these particular tuning properties in the cortical space will strongly shape the V1 population response to different visual inputs, yet it is poorly understood. In this study, we used two-photon imaging in macaque monkey V1 to demonstrate the three-dimensional cell-by-cell layout of both spatial frequency and orientation tuning. We first found that spatial frequency tuning was organized into highly structured maps that remained consistent across the depth of layer II/III, similarly to orientation tuning. Next, we found that orientation and spatial frequency maps were intimately related at the fine spatial scale observed with two-photon imaging. Not only did the map gradients tend notably toward orthogonality, but they also co-varied negatively from cell to cell at the spatial scale of cortical columns.

Real-time simulation of large-scale neural architectures for visual features computation based on GPU

The intrinsic parallelism of visual neural architectures based on distributed hierarchical layers is well suited to be implemented on the multi-core architectures of modern graphics cards. The design strategies that allow us to optimally take advantage of such parallelism, in order to efficiently map on GPU the hierarchy of layers and the canonical neural computations, are proposed. Specifically, the advantages of a cortical map-like representation of the data are exploited. Moreover, a GPU implementation of a novel neural architecture for the computation of binocular disparity from stereo image pairs, based on populations of binocular energy neurons, is presented. The implemented neural model achieves good performances in terms of reliability of the disparity estimates and a near real-time execution speed, thus demonstrating the effectiveness of the devised design strategies. The proposed approach is valid in general, since the neural building blocks we implemented are a common basis for the modeling of visual neural functionalities.

Parallel development of orientation maps and spatial frequency selectivity in cat visual cortex

In an early stage of the postnatal development of cats, orientation maps mature and spatial frequency selectivity is consolidated. To investigate the time course of orientation map maturation associated with the consolidation of spatial frequency selectivity, we performed optical imaging of intrinsic signals in areas 17 and 18 of cats under the stimulation of drifting square-wave gratings with different orientations and spatial frequencies. First, orientation maps for lower spatial frequencies emerged in the entire part of the lateral gyrus, which includes areas 17 and 18, and then these orientation maps in the posterior part of the lateral gyrus disappeared as orientation maps for higher spatial frequencies matured. Independent of age, an anteroposterior gradient of response strengths from lower to higher spatial frequencies was observed. This indicates that the regional distribution of spatial frequencies is innately determined. The size of iso-orientation domains tended to decrease as the stimulus spatial frequency increased at every age examined. In contrast, orientation representation bias changed with age. In cats younger than 3 months, the cardinal (vertical and horizontal) orientations were represented predominantly over the oblique orientations. However, in young adult cats from 3 to 9 months old, the representation bias switched to predominantly oblique orientations. These age-dependent changes in the orientation representation bias imply that orientation maps continue to elaborate within postnatal 1 year with the consolidation of spatial frequency selectivity. We conclude that both intrinsic and mutual factors lead to the development of orientation maps and spatial frequency selectivity.

A multivariate, multitaper approach to detecting and estimating harmonic response in cortical optical imaging data

The efficiency and accuracy of cortical maps from optical imaging experiments have been improved using periodic stimulation protocols. The resulting data analysis requires the detection and estimation of periodic information in a multivariate dataset. To date, these analyses have relied on discrete Fourier transform (DFT) sinusoid estimates. Multitaper methods have become common statistical tools in the analysis of univariate time series that can give improved estimates. Here, we extend univariate multitaper harmonic analysis methods to the multivariate, imaging context. Given the hypothesis that a coherent oscillation across many pixels exists within a specified bandwidth, we investigate the problem of its detection and estimation in noisy data by constructing Hotelling's generalized T(2)-test. We then extend the investigation of this problem in two contexts, that of standard canonical variate analysis (CVA) and that of generalized indicator function analysis (GIFA) which is often more robust in extracting a signal in spatially correlated noise. We provide detailed information on the fidelities of the mean estimates found with our methods and comparison with DFT estimates. Our results indicate that GIFA provides particularly good estimates of harmonic signals in spatially correlated noise and is useful for detecting small amplitude harmonic signals in applications such as biological imaging measurements where spatially correlated noise is common. We demonstrate the power of our methods with an optical imaging dataset of the periodic response to a periodically rotating oriented drifting grating stimulus experiment in cat visual cortex.

Denoising two-photon calcium imaging data

Two-photon calcium imaging is now an important tool for in vivo imaging of biological systems. By enabling neuronal population imaging with subcellular resolution, this modality offers an approach for gaining a fundamental understanding of brain anatomy and physiology. Proper analysis of calcium imaging data requires denoising, that is separating the signal from complex physiological noise. To analyze two-photon brain imaging data, we present a signal plus colored noise model in which the signal is represented as harmonic regression and the correlated noise is represented as an order autoregressive process. We provide an efficient cyclic descent algorithm to compute approximate maximum likelihood parameter estimates by combing a weighted least-squares procedure with the Burg algorithm. We use Akaike information criterion to guide selection of the harmonic regression and the autoregressive model orders. Our flexible yet parsimonious modeling approach reliably separates stimulus-evoked fluorescence response from background activity and noise, assesses goodness of fit, and estimates confidence intervals and signal-to-noise ratio. This refined separation leads to appreciably enhanced image contrast for individual cells including clear delineation of subcellular details and network activity. The application of our approach to in vivo imaging data recorded in the ferret primary visual cortex demonstrates that our method yields substantially denoised signal estimates. We also provide a general Volterra series framework for deriving this and other signal plus correlated noise models for imaging. This approach to analyzing two-photon calcium imaging data may be readily adapted to other computational biology problems which apply correlated noise models.

Statistics of natural scenes and cortical color processing

We investigate the spatial correlations of orientation and color information in natural images. We find that the correlation of orientation information falls off rapidly with increasing distance, while color information is more highly correlated over longer distances. We show that orientation and color information are statistically independent in natural images and that the spatial correlation of jointly encoded orientation and color information decays faster than that of color alone. Our findings suggest that: (a) orientation and color information should be processed in separate channels and (b) the organization of cortical color and orientation selectivity at low spatial frequencies is a reflection of the cortical adaptation to the statistical structure of the visual world. These findings are in agreement with biological observations, as form and color are thought to be represented by different classes of neurons in the primary visual cortex, and the receptive fields of color-selective neurons are larger than those of orientation-selective neurons. The agreement between our findings and biological observations supports the ecological theory of perception.

Neural computation of visual imaging based on Kronecker product in the primary visual cortex

BACKGROUND

What kind of neural computation is actually performed by the primary visual cortex and how is this represented mathematically at the system level? It is an important problem in the visual information processing, but has not been well answered. In this paper, according to our understanding of retinal organization and parallel multi-channel topographical mapping between retina and primary visual cortex V1, we divide an image into orthogonal and orderly array of image primitives (or patches), in which each patch will evoke activities of simple cells in V1. From viewpoint of information processing, this activated process, essentially, involves optimal detection and optimal matching of receptive fields of simple cells with features contained in image patches. For the reconstruction of the visual image in the visual cortex V1 based on the principle of minimum mean squares error, it is natural to use the inner product expression in neural computation, which then is transformed into matrix form.

RESULTS

The inner product is carried out by using Kronecker product between patches and function architecture (or functional column) in localized and oriented neural computing. Compared with Fourier Transform, the mathematical description of Kronecker product is simple and intuitive, so is the algorithm more suitable for neural computation of visual cortex V1. Results of computer simulation based on two-dimensional Gabor pyramid wavelets show that the theoretical analysis and the proposed model are reasonable.

CONCLUSIONS

Our results are: 1. The neural computation of the retinal image in cortex V1 can be expressed to Kronecker product operation and its matrix form, this algorithm is implemented by the inner operation between retinal image primitives and primary visual cortex's column. It has simple, efficient and robust features, which is, therefore, such a neural algorithm, which can be completed by biological vision. 2. It is more suitable that the function of cortical column in cortex V1 is considered as the basic unit of visual image processing (such unit can implement basic multiplication of visual primitives, such as contour, line, and edge), rather than a set of tiled array filter. Fourier Transformation is replaced with Kronecker product, which greatly reduces the computational complexity. The neurobiological basis of this idea is that a visual image can be represented as a linear combination of orderly orthogonal primitive image containing some local feature. In the visual pathway, the image patches are topographically mapped onto cortex V1 through parallel multi-channels and then are processed independently by functional columns. Clearly, the above new perspective has some reference significance to exploring the neural mechanisms on the human visual information processing.

Symmetry, probability, and recognition in face space

The essential midline symmetry of human faces is shown to play a key role in facial coding and recognition. This also has deep and important connections with recent explorations of the organization of primate cortex, as well as human psychophysical experiments. Evidence is presented that the dimension of face recognition space for human faces is dramatically lower than previous estimates. One result of the present development is the construction of a probability distribution in face space that produces an interesting and realistic range of (synthetic) faces. Another is a recognition algorithm that by reasonable criteria is nearly 100% accurate.

Quantification of optical images of cortical responses for inferring functional maps

Optical imaging of cortical signals enables the mapping of functional organization across large patches of cortex with good spatial resolution. But techniques for the quantitative analysis and interpretation of these images are limited. Frequently the functional architecture of the cortex is inferred from the visible topography of cortical reflectance images averaged or differenced across stimulus conditions and scaled or color-coded for presentation. Such qualitative assessments have sometimes led to divergent conclusions particularly about the organization of spatial and temporal frequency preferences in the primary visual cortex. We applied quantitative methods derived from signal detection theory to objectively interpret optical images. The differential response to any two arbitrary stimuli was represented at each pixel as the probability of discriminating between the two stimuli given the reflectance values at that pixel. These probability maps reduced false alarms and provided better signal-to-noise ratio in fewer trials than difference maps. We applied these methods to optical images of primate primary visual area (V1) obtained in response to sinusoidal gratings of different orientations and spatiotemporal frequencies. Clustering by orientation preference was stronger than that for spatial frequency, whereas clustering by temporal frequency preference was the weakest, largely in agreement with a previous electrophysiological study that quantified the degree of clustering of neurons for various response properties using uniform, quantitative criterion. We suggest that probability maps can extend the applicability of optical imaging to investigations of finer aspects of cortical functional organization through better signal-to-noise ratio and uniform, quantitative criteria for interpretation.

Extrapolative delay compensation through facilitating synapses and its relation to the flash-lag effect

Neural conduction delay is a serious issue for organisms that need to act in real time. Various forms of flash-lag effect (FLE) suggest that the nervous system may perform extrapolation to compensate for delay. For example, in motion FLE, the position of a moving object is perceived to be ahead of a brief flash when they are actually colocalized. However, the precise mechanism for extrapolation at a single-neuron level has not been fully investigated. Our hypothesis is that facilitating synapses, with their dynamic sensitivity to the rate of change in the input, can serve as a neural basis for extrapolation. To test this hypothesis, we constructed and tested models of facilitating dynamics. First, we derived a spiking neuron model of facilitating dynamics at a single-neuron level, and tested it in the luminance FLE domain. Second, the spiking neuron model was extended to include multiple neurons and spike-timing-dependent plasticity (STDP), and was tested with orientation FLE. The results showed a strong relationship between delay compensation, FLE, and facilitating synapses/STDP. The results are expected to shed new light on real time and predictive processing in the brain, at the single neuron level.

Functional organization of temporal frequency selectivity in primate visual cortex

Several studies have shown that neurons with similar response properties are arranged together in domains across primary visual cortex (V1). An orderly pattern of domains has been described for preferences to ocular dominance, orientation, and spatial frequency. Temporal frequency preference, another important attribute of the visual scene, also might be expected to map into different domains. Using optical imaging and a variety of quantitative methods, we examined how temporal frequency selectivity is mapped in V1 of the prosimian primate, bush baby (Otolemur garnetti). We found that unlike other attribute maps, selectivity for different temporal frequencies is arranged uniformly across V1 with no evidence of local clustering. Global tuning for temporal frequency, based on magnitude of response, showed a good match to previous tuning curves for single neurons. A peak response was found around 2.0 Hz, with smaller attenuation at lower temporal frequencies than at higher frequencies. We also examined whether the peak temporal frequency response differed between anatomical compartments defined by cytochrome oxidase (CO). No significant differences in the preference for temporal frequency were found between these CO compartments. Our findings show that key sensory attributes that are linked in perception can be organized in quite distinct ways in V1 of primates.

The organization of spatial frequency maps measured by cortical flavoprotein autofluorescence

To determine the organization of spatial frequency (SF) preference within cat Area 17, we imaged responses to stimuli with different SFs using optical intrinsic signals (ISI) and flavoprotein autofluorescence (AFI). Previous studies have suggested that neurons cluster based on SF preference, but a recent report argued that SF maps measured with ISI were artifacts of the vascular bed. Because AFI derives from a non-hemodynamic signal, it is less contaminated by vasculature. The two independent imaging methods produced similar SF preference maps in the same animals, suggesting that the patchy organization of SF preference is a genuine feature of Area 17.

Models and Measurements of Functional Maps in V1

The organization of primary visual cortex has been heavily studied for nearly 50 years, and in the last 20 years functional imaging has provided high-resolution maps of its tangential organization. Recently, however, the usefulness of maps like those of orientation and spatial frequency (SF) preference has been called into question because they do not, by themselves, predict how moving images are represented in V1. In this review, we discuss a model for cortical responses (the spatiotemporal filtering model) that specifies the types of cortical maps needed to predict distributed activity within V1. We then review the structure and interrelationships of several of these maps, including those of orientation, SF, and temporal frequency preference. Finally, we discuss tests of the model and the sufficiency of the requisite maps in predicting distributed cortical responses. Although the spatiotemporal filtering model does not account for all responses within V1, it does, with reasonable accuracy, predict population responses to a variety of complex stimuli.

Adaptation changes the spatial frequency tuning of adult cat visual cortex neurons

The modular layout of striate cortex is arguably a hallmark of all cortical organization. Neurons of a given module or domain respond optimally to very few specific properties, such as orientation or direction. However, it is possible, under appropriate conditions, to compel a neuron to respond preferentially to a different optimal property. In anesthetized cats, prepared for electrophysiological recordings in the visual cortex, we applied a spatial frequency (SF) that differs (by 0.25-3.0 octaves) from the optimal one for 7-13 min without interruption. This application shifted the tuning curve of the cell mainly in the direction of the imposed SF. Indeed, results indicate an attractive push occurring more frequently (50%) than a repulsive (30%) shift in cortical cells. The increase of responsivity is band-limited and is around the imposed SF, while flanked responses remained unmodified in all conditions. We hypothesize that the observed reversible plasticity is obtained by a modulation of the balance between the strengths of the respective synaptic inputs. These changes in preferred original optimal spatial frequencies may allow a dynamic reaction of cortex to a new environment and particularly to ''zoom'' cellular activity toward persistent stimuli in spite of the tuning inherited from genetic programming of response properties and environmental conditions during critical periods in new born animals.

Kinetic theory for neuronal networks with fast and slow excitatory conductances driven by the same spike train

We present a kinetic theory for all-to-all coupled networks of identical, linear, integrate-and-fire, excitatory point neurons in which a fast and a slow excitatory conductance are driven by the same spike train in the presence of synaptic failure. The maximal-entropy principle guides us in deriving a set of three (1+1) -dimensional kinetic moment equations from a Boltzmann-like equation describing the evolution of the one-neuron probability density function. We explain the emergence of correlation terms in the kinetic moment and Boltzmann-like equations as a consequence of simultaneous activation of both the fast and slow excitatory conductances and furnish numerical evidence for their importance in correctly describing the coarse-grained dynamics of the underlying neuronal network.

Higher order visual processing in macaque extrastriate cortex

The extrastriate cortex of primates encompasses a substantial portion of the cerebral cortex and is devoted to the higher order processing of visual signals and their dispatch to other parts of the brain. A first step towards the understanding of the function of this cortical tissue is a description of the selectivities of the various neuronal populations for higher order aspects of the image. These selectivities present in the various extrastriate areas support many diverse representations of the scene before the subject. The list of the known selectivities includes that for pattern direction and speed gradients in middle temporal/V5 area; for heading in medial superior temporal visual area, dorsal part; for orientation of nonluminance contours in V2 and V4; for curved boundary fragments in V4 and shape parts in infero-temporal area (IT); and for curvature and orientation in depth from disparity in IT and CIP. The most common putative mechanism for generating such emergent selectivity is the pattern of excitatory and inhibitory linear inputs from the afferent area combined with nonlinear mechanisms in the afferent and receiving area.

Neural model of disinhibitory interactions in the modified Poggendorff illusion

Visual illusions can be strengthened or weakened with the addition of extra visual elements. For example, in the Poggendorff illusion, with an additional bar added, the illusory skew in the perceived angle can be enlarged or reduced. In this paper, we show that a nontrivial interaction between lateral inhibitory processes in the early visual system (i.e., disinhibition) can explain such an enhancement or degradation of the illusory effect. The computational model we derived successfully predicted the perceived angle in the Poggendorff illusion task that was modified to include an extra thick bar. The concept of disinhibition employed in the model is general enough that we expect it can be further extended to account for other classes of geometric illusions.

The representation of complex images in spatial frequency domains of primary visual cortex

The organization of cat primary visual cortex has been well mapped using simple stimuli such as sinusoidal gratings, revealing superimposed maps of orientation and spatial frequency preferences. However, it is not yet understood how complex images are represented across these maps. In this study, we ask whether a linear filter model can explain how cortical spatial frequency domains are activated by complex images. The model assumes that the response to a stimulus at any point on the cortical surface can be predicted by its individual orientation, spatial frequency, and temporal frequency tuning curves. To test this model, we imaged the pattern of activity within cat area 17 in response to stimuli composed of multiple spatial frequencies. Consistent with the predictions of the model, the stimuli activated low and high spatial frequency domains differently: at low stimulus drift speeds, both domains were strongly activated, but activity fell off in high spatial frequency domains as drift speed increased. To determine whether the filter model quantitatively predicted the activity patterns, we measured the spatiotemporal tuning properties of the functional domains in vivo and calculated expected response amplitudes from the model. The model accurately predicted cortical response patterns for two types of complex stimuli drifting at a variety of speeds. These results suggest that the distributed activity of primary visual cortex can be predicted from cortical maps like those of orientation and SF preference generated using simple, sinusoidal stimuli, and that dynamic visual acuity is degraded at or before the level of area 17.

Spatial frequency characteristics of nearby neurons in cats’ visual cortex

Various methods have allowed mapping of responses to several stimulus features on the cortical surface, particularly edge orientation and motion direction. The cortical mapping of spatial frequencies (SF), which is the basic property that leads to perception of spatial details of visual objects, is still controversial. We recorded simultaneously extracellular action potentials from neighboring cells in superficial layers of the area 17-18 border region of anesthetized cats. Responses of nearby cells to sine-wave gratings of varying SF were analyzed. Spatial frequency tuning curves were cross-correlated to establish the degree of similarity between the curves and optimal SFs were compared for each pair of neurons. The investigation showed that only about a half of nearby neurons exhibited close optimal SFs and similar tuning curves. The results suggest that SF channels do not show a clear clustering within a small pool of neurons. Such organization may contribute to the perception of spatial details at all orientations and motion directions.

How do functional maps in primary visual cortex vary with eccentricity?

It is important to understand whether functional maps of primary visual cortex (V1) are organized differently at the representation of different eccentricities. By using optical imaging of intrinsic signals, we compared the maps of orientation and spatial frequency (SF) preference between central (0-3 degrees ) and paracentral (4-8 degrees ) V1 in the prosimian bush baby (Otolemur garnetti). No differences related to eccentricity were found for orientation selectivity or magnitude between central and paracentral V1. We found, however, that cardinal orientations were overrepresented in central but not in paracentral V1 and that isoorientation domain size tended to be smaller in the central representation. We demonstrated that spatial frequency was represented continuously across V1, and that the map of SF preference exhibited eccentricity-dependent variations, with more territory devoted to higher SFs in central than in paracentral V1. Although there were no spatial relationships between orientation domains and cytochrome oxidase (CO) blobs or interblobs, CO blobs tended to prefer lower SFs than interblobs. Taken together with previous research, our data indicate that functional domains in V1 show eccentricity-related differences in organization and also support the idea that different maps (with or without specific geometrical relationships) are organized for adequate coverage of each feature in visual space.

Maximum-entropy closures for kinetic theories of neuronal network dynamics

We analyze (1 + 1)D kinetic equations for neuronal network dynamics, which are derived via an intuitive closure from a Boltzmann-like equation governing the evolution of a one-particle (i.e., one-neuron) probability density function. We demonstrate that this intuitive closure is a generalization of moment closures based on the maximum-entropy principle. By invoking maximum-entropy closures, we show how to systematically extend this kinetic theory to obtain higher-order, kinetic equations and to include coupled networks of both excitatory and inhibitory neurons.

The neural mechanisms of perceptual filling-in

Filling-in is a perceptual phenomenon in which a visual attribute such as colour, brightness, texture or motion is perceived in a region of the visual field even though such an attribute exists only in the surround. Filling-in dramatically reveals the dissociation between the retinal input and the percept, and raises fundamental questions about how these two relate to each other. Filling-in is observed in various situations, and is an essential part of our normal surface perception. Here, I review recent experiments examining brain activities associated with filling-in, and discuss possible neural mechanisms underlying this remarkable perceptual phenomenon. The evidence shows that neuronal activities in early visual cortical areas are involved in filling-in, providing new insights into visual cortical functions.

Cortical cartography revisited: a frequency perspective on the functional architecture of visual cortex

Viewed in the plane of the cortical surface, the visual cortex is composed of overlapping functional maps that represent stimulus features such as edge orientation, direction of motion, and spatial frequency. Spatial relationships between these maps are thought to ensure that all combinations of stimulus features are represented uniformly across the visual field. Implicit in this view is the assumption that feature combinations are represented in the form of a place code such that a given pattern of activity uniquely signifies a specific combination of stimulus features. Here we review results of experiments that challenge the place code model for the representation of feature combinations. Rather than overlapping maps of stimulus features, we suggest that patterns of activity evoked by complex stimuli are best understood in the context of a single map of spatiotemporal energy.

The cortical column: a structure without a function

This year, the field of neuroscience celebrates the 50th anniversary of Mountcastle's discovery of the cortical column. In this review, we summarize half a century of research and come to the disappointing realization that the column may have no function. Originally, it was described as a discrete structure, spanning the layers of the somatosensory cortex, which contains cells responsive to only a single modality, such as deep joint receptors or cutaneous receptors. Subsequently, examples of columns have been uncovered in numerous cortical areas, expanding the original concept to embrace a variety of different structures and principles. A "column" now refers to cells in any vertical cluster that share the same tuning for any given receptive field attribute. In striate cortex, for example, cells with the same eye preference are grouped into ocular dominance columns. Unaccountably, ocular dominance columns are present in some species, but not others. In principle, it should be possible to determine their function by searching for species differences in visual performance that correlate with their presence or absence. Unfortunately, this approach has been to no avail; no visual faculty has emerged that appears to require ocular dominance columns. Moreover, recent evidence has shown that the expression of ocular dominance columns can be highly variable among members of the same species, or even in different portions of the visual cortex in the same individual. These observations deal a fatal blow to the idea that ocular dominance columns serve a purpose. More broadly, the term "column" also denotes the periodic termination of anatomical projections within or between cortical areas. In many instances, periodic projections have a consistent relationship with some architectural feature, such as the cytochrome oxidase patches in V1 or the stripes in V2. These tissue compartments appear to divide cells with different receptive field properties into distinct processing streams. However, it is unclear what advantage, if any, is conveyed by this form of columnar segregation. Although the column is an attractive concept, it has failed as a unifying principle for understanding cortical function. Unravelling the organization of the cerebral cortex will require a painstaking description of the circuits, projections and response properties peculiar to cells in each of its various areas.

Extraction of the average and differential dynamical response in stimulus-locked experimental data

In optical imaging experiments of primary visual cortex, visual stimuli evoke a complicated dynamics. Typically, any stimulus with sufficient contrast evokes a response. Much of the response is the same regardless of which stimulus is presented. For instance, when oriented drifting gratings are presented to the visual system, over 90% of the response is the same from orientation to orientation. Small differences may be seen, however, between the responses to different orientations. A problem in the analysis of optical measurements of the response to stimulus in cortical tissue is the distinction of the 'global' or 'non-specific' response from the 'differential' or 'stimulus-specific' response. This problem arises whenever the signal of interest is the difference in response to various stimuli and is evident in many kinds of uni- and multivariate data. To this end, we present enhancements to a frequency-based method that we previously introduced called the periodic stacking method. These enhancements allow us to separately estimate the dynamics of both the average signal across all stimuli (the 'global' response) and deviations from the average amongst the various stimuli (the 'stimulus-specific' response) evoked in response to a set of stimuli. We also discuss improvements in the signal-to-noise ratio, relative to standard trial averaging methods, that result from the data-adaptive smoothing in our method.

Cortical maps of separable tuning properties predict population responses to complex visual stimuli

In the earliest cortical stages of visual processing, a scene is represented in different functional domains selective for specific features. Maps of orientation and spatial frequency preference have been described in the primary visual cortex using simple sinusoidal grating stimuli. However, recent imaging experiments suggest that the maps of these two spatial parameters are not sufficient to describe patterns of activity in different orientation domains generated in response to complex, moving stimuli. A model of cortical organization is presented in which cortical temporal frequency tuning is superimposed on the maps of orientation and spatial frequency tuning. The maps of these three tuning properties are sufficient to describe the activity in orientation domains that have been measured in response to drifting complex images. The model also makes specific predictions about how moving images are represented in different spatial frequency domains. These results suggest that the tangential organization of primary visual cortex can be described by a set of maps of separable neuronal receptive field features including maps of orientation, spatial frequency, and temporal frequency tuning properties.

Orientation and direction selectivity of neurons in V1 of alert monkeys: functional relationships and laminar distributions

We studied orientation selectivity in V1 of alert monkeys and its relationship to other physiological parameters and to anatomical organization. Single neurons were stimulated with drifting bars or with sinusoidal gratings while compensating for eye position. Orientation selectivity based on spike counts was quantified by circular variance and by the bandwidth of the orientation tuning curve. The circular variance distribution was bimodal, suggesting groups with low and with high selectivity. Orientation selectivity was clearly correlated with spontaneous activity, classical receptive field (CRF) size and the strength of surround suppression. Laminar distributions of neuronal properties were distinct. Neurons in the output layers 2/3, 4B and 5 had low spontaneous activity, small CRFs and high orientation selectivity, while the input layers had greater diversity. Direction-selective cells were among the neurons most selective for orientation and most had small CRFs. A narrow band of direction- and orientation-selective cells with small CRFs was located in the middle of layer 4C, indicating appearance of very selective cells at an early stage of cortical processing. We suggest that these results reflect interactions between excitatory and inhibitory mechanisms specific to each sublamina. Regions with less inhibition have higher spontaneous activity, larger CRFs and broader orientation tuning. Where inhibition is stronger, spontaneous activity almost disappears, CRFs shrink, and orientation selectivity is high.