How does the striate cortex begin the reconstruction of the visual world?

Synaptic Origins of the Complex Receptive Field Structure in Primate Smooth Monostratified Retinal Ganglion Cells

Considerable progress has been made in studying the receptive fields of the most common primate retinal ganglion cell (RGC) types, such as parasol RGCs. Much less is known about the rarer primate RGC types and the circuitry that gives rise to noncanonical receptive field structures. The goal of this study was to analyze synaptic inputs to smooth monostratified RGCs to determine the origins of their complex spatial receptive fields, which contain isolated regions of high sensitivity called "hotspots." Interestingly, smooth monostratified RGCs co-stratify with the well-studied parasol RGCs and are thus constrained to receiving input from bipolar and amacrine cells with processes sharing the same layer, raising the question of how their functional differences originate. Through 3D reconstructions of circuitry and synapses onto ON smooth monostratified and ON parasol RGCs from central macaque retina, we identified four distinct sampling strategies employed by smooth and parasol RGCs to extract diverse response properties from co-stratifying bipolar and amacrine cells. The two RGC types differed in the proportion of amacrine cell input, relative contributions of co-stratifying bipolar cell types, amount of synaptic input per bipolar cell, and spatial distribution of bipolar cell synapses. Our results indicate that the smooth RGC's complex receptive field structure arises through spatial asymmetries in excitatory bipolar cell input which formed several discrete clusters comparable with physiologically measured hotspots. Taken together, our results demonstrate how the striking differences between ON parasol and ON smooth monostratified RGCs arise from distinct strategies for sampling a common set of synaptic inputs.

Spatiotemporal flicker detector model of motion silencing

Motion can impair the perception of other visual changes. Suchow and Alvarez (2011a, Current Biology, 21, 140-143) recently demonstrated a striking 'motion silencing' illusion, in which the salient changes among a group of objects' luminances (or colors, etc) appear to cease in the presence of large, coherent object motion. To understand why the visual system might be insensitive to changes in object luminances ('flicker') in the presence of object motion, we constructed similar stimuli and did a systematic spectral analysis of them. We conducted human psychophysical experiments to examine motion silencing as a function of stimulus velocity, flicker frequency, and spacing; and we created a simple filter-based model as a working hypothesis of motion silencing. From the results, we found that the threshold of silencing occurs when the log frequency of object replacement is roughly one quarter of the log flicker frequency (the mean slope is approximately 0.27). The dependence of silencing on object spacing may be explained as a phenomenon of temporal sampling of the stimuli by the visual system. Our proposed model successfully captures the psychophysical data over a wide range of velocities and flicker frequencies.

Theoretical role of square wavelet analysis in stereopsis and binocular rivalry: a mathematical model and computer simulation

To determine stereopsis, a mathematical model and computer simulation specify that square wavelet coefficients be independently estimated for the images in each eye. Then, by comparing these independently determined coefficients, the model theorizes that the visual system both identifies those parts of the two images which lie across corresponding points and identifies those parts of the two images which lie across noncorresponding or disparate points. Given the latter occurrence, the retinal disparity is evaluated as being either stereopsis or binocular rivalry. If stereopsis is indicated, the wavelet coefficients provide a reference plane by which the depth of a disparate area is located either nearer than or more distant than the reference plane. In the model also it is proposed that the determination of depth with respect to the reference plane provides a signal for vergence of the eyes. If binocular rivalry is indicated, the model's computer simulation identifies the locus of the background fringes associated with binocular rivalry. In sum, it seems theoretically tenable that stereopsis or binocular rivalry occurs as a consequence of the comparison of one eye's wavelet coefficients with those of the other eye.

Visual image analysis by square wavelets: empirical evidence supporting a theoretical agreement between wavelet analysis and receptive field organization of visual cortical neurons

It was proposed that the human visual system analyzes images into square wavelets. To test this view, comparisons were made between the perceived similarity-dissimilarity of alphabet letters and the wavelet analyses of those same letters. For the proposal to be considered tenable, the coefficients of the wavelet analysis of similar letters must be similar, and the coefficients of the wavelet analysis of dissimilar letters must be dissimilar. From a selection of 12 letters, four pairs of letters had been reported by Van der Heijden, Mathas, and Van den Roovaart as very similar, and four other pairs of letters dissimilar. Each of the 12 letters was separately depicted in 8 x 8 matrices, and the signal represented by each of the matrices was analyzed into square wavelets using a new and original procedure which yielded a single set of coefficients for each matrix. Correlations between sets of coefficients were high (r ranged from .88 to .58) for those letter pairs judged high in similarity; correlations were low (r ranged from -.02 to .29) for those letter pairs judged low in similarity. When the correlations between the coefficients of wavelets of all eight-letter pairs were compared with the judged similarity-dissimilarity of all eight-letter pairs, the linear agreement was statistically significant. Agreement was found between the neurophysiological mapping of receptive fields of visual cortical neurons and the vectors or the pattern of pluses and minuses which characterized the wavelet analysis. Furthermore, regeneration of the visual image, or the pattern of neural activity representing the image, could be described by a tree-like flow of information among visual cortical neurons which received response data from visual receptive fields, the response data being wavelet coefficients. Results indicate the analysis accurately produces reliable transformations of visual patterns and may be a process used by the visual system.

Spatial frequency discrimination in schizophrenia

Pathways within the visual system can be distinguished on the basis of selectivity for low or high spatial frequencies. Spatial frequency discrimination was evaluated in 17 medicated male patients with schizophrenia and 19 male control subjects. Subjects were required to discriminate whether pairs of high contrast, sinusoidally modulated gratings were the same or different in spatial frequency. Accuracy performance was compared at high, medium, and low spatial frequencies on tasks matched for control performance. Patients showed a greater performance decrement of 12% on low as compared with 4% on high spatial frequencies. These findings suggest a disturbance of right hemisphere mechanisms involved in spatial perception and attention in schizophrenia.

Spatial frequency discrimination in schizophrenia

Pathways within the visual system can be distinguished on the basis of selectivity for low or high spatial frequencies. Spatial frequency discrimination was evaluated in 17 medicated male patients with schizophrenia and 19 male control subjects. Subjects were required to discriminate whether pairs of high contrast, sinusoidally modulated gratings were the same or different in spatial frequency. Accuracy performance was compared at high, medium, and low spatial frequencies on tasks matched for control performance. Patients showed a greater performance decrement of 12% on low as compared with 4% on high spatial frequencies. These findings suggest a disturbance of right hemisphere mechanisms involved in spatial perception and attention in schizophrenia.

The problem of the missing wavelet

A block face was generated by dividing a drawn face into a matrix of rectangular cells and then making the brightness of each cell equal to the average of the brightnesses within the facial area circumscribed by a cell. A wavelet analysis was performed on the numbers representing the brightnesses of the cells in each row of the matrix, and from each row, the wavelet was deleted which corresponded to the fundamental or the lowest frequency sine wave of a complex wave. The appearance of the block face was not substantially altered by the deletion of the wavelet corresponding to the fundamental.

Evolving concepts of spatial channels in vision: from independence to nonlinear interactions

By the 1960s it was evident from neuroanatomy that there were extensive recurrent interactions, both excitatory and inhibitory, among visual cortical neurons. Nevertheless, the psychophysical discovery of 'spatial-frequency channels' gave rise to a decade in which parallel, independent channels were thought to subserve early spatial vision. Recent work, however, has clearly demonstrated that early visual channels do not perform a Fourier or wavelet decomposition of the image. Instead, they interact through a variety of nonlinear pooling mechanisms. Such nonlinear interactions perform important computations in texture perception, stereopsis, and motion and form vision.

Non-linear Feature Extraction by Redundancy Reduction in an Unsupervised Stochastic Neural Network

Unsupervised feature extraction by a stochastic neural network can be defined as a minimization of the redundancy between the elements of the output layer, given complete information transfer from input to output. Redundancy minimization can be achieved by minimization of the mutual information between the units of the output layer. Complete information transfer is enforced by maximizing the mutual information of the input and output. With these two conditions we define a novel learning algorithm for stochastic recurrent networks. The minimum of redundancy corresponds to the extraction of statistically independent features, leading to a factorial representation of the environment. The resulting learning rule includes Hebbian and anti-Hebbian learning terms. These two terms are weighted by the amount of information transmitted in the learning synapse minus the grade of redundant information in the corresponding output neuron, giving thus, an information-theoretic interpretation of the proportionality constant of Hebb's biological rule. Simulations demonstrate the performance of this method. When a retina is simulated, the learning algorithm forms decorrelated receptive fields. This represents the first experiment that extends the results of the linear principle component analysis to the nonlinear case by a direct implementation of Barlow's principle of redundancy reduction for unsupervised features extraction by receptive fields formation in a retina model. Copyright 1997 Elsevier Science Ltd.

Spatio-temporal frequency domains and their relation to cytochrome oxidase staining in cat visual cortex

Spatial and temporal frequencies are important attributes of the visual scene. It is a long-standing question whether these attributes are represented in a spatially organized way in cat primary visual cortex. Using optical imaging of intrinsic signals, we show here that grating stimuli of different spatial frequencies drifting at various speeds produce distinct activity patterns. Rather than observing a map of continuously changing spatial frequency preference across the cortical surface, we found only two distinct sets of domains, one preferring low spatial frequency and high speed, and the other high spatial frequency and low speed. We compared the arrangement of these spatio-temporal frequency domains with the cytochrome oxidase staining pattern, which, based on work in primate striate cortex, is thought to reflect the partition of the visual cortex into different processing streams. We found that the cytochrome oxidase blobs in cat striate cortex coincide with domains engaged in the processing of low spatial and high temporal frequency contents of the visual scene. Together with other recent results, our data suggest that spatiotemporal frequency domains are a manifestation of parallel streams in cat visual cortex, with distinct patterns of thalamic inputs and extrastriate projections.

Similarity between Fourier transforms of objects predicts their experimental confusions

Two sets of stimuli were presented tachistoscopically to 4 subjects. On each trial, a single stimulus was presented, and the subject was required to identify the stimulus by verbal response. An exposure duration was chosen such that the subject's identification performance fell within a range from faultless identification to chance guessing. The object-identification data of each subject obtained for all stimulus exposures were pooled to form an object confusion matrix. A model of visual processing based on two-dimensional spatial frequency content (Fourier transforms) was used to predict confusions among stimulus pairs. The model properties that appear to be the most essential are those that allow it (1) to account for the obvious dependence of the Fourier transform on the choice of an origin point; and (2) choose the point of origin for each object separately, irrespective of other objects of the set. The point of origin of the reference frame, in which Fourier transforms are performed, is chosen so as to minimize the low-frequency phase component for each object. A high correlation (up to .96) between confusion matrices and model interobject distances was attained. The results demonstrate that such a distance measure gives a good prediction of object confusability.

Feature detection in human vision: a phase-dependent energy model

This paper presents a simple and biologically plausible model of how mammalian visual systems could detect and identify features in an image. We suggest that the points in a waveform that have unique perceptual significance as 'lines' and 'edges' are the points where the Fourier components of the waveform come into phase with each other. At these points 'local energy' is maximal. Local energy is defined as the square root of the sum of the squared response of sets of matched filters, of identical amplitude spectrum but differing in phase spectrum by 90 degrees: one filter type has an even-symmetric line-spread function, the other an odd-symmetric line-spread function. For a line the main contribution to the local energy peak is in the output of the even-symmetric filters, whereas for edges it is in the output of the odd-symmetric filters. If both filter types respond at the peak of local energy, both edges and lines are seen, either simultaneously or alternating in time. The model was tested with a series of images, and shown to predict well the position of perceived features and the organization of the images.

Modulation of rhythmic function in the posterior midbrain

Recordings of single unit activity in the posterior midbrain of the cat were carried out in the "fictive spontaneous locomotion" preparation. Neuronal activity was studied in relation to the onset, alternation and termination of cyclic hindlimb neurographic activity in the precollicular-postmammillary transected animal. Histochemical identification of pedunculopontine (nicotinamide adenine dinuceotide phosphate-diaphorase positive) neurons allowed the localization of recording sites in relation to this nucleus. Neurons located in the area of the cuneiform nucleus dorsal to the pedunculopontine nucleus were found to be related preferentially to cyclic (bursting) neurographic activity, while neurons in the area of the pedunculopontine were found to be related preferentially to the onset ("on") or termination ("off") of cycling episodes. Different populations of cells in the area appeared to be related to the frequency of alternation (bursting) compared with the duration of the cyclic episodes (on/off). While the area of the cuneiform-pedunculopontine nucleus has been found to be equivalent to the mesencephalic locomotor region, the same area has been found to be related to other rhythmic activities (e.g. respiratory, masticatory, sleep cycle, pressor, vesico-motor, etc.). A hypothesis is proposed to account for the weight of evidence implicating the same region in a host of distinct rhythmic activities. This hypothesis suggest that an oscillatory reverberation between cholinergic (pedunculopontine, laterodorsal tegmental nuclei) and aminergic (locus coeruleus, substantia nigra) centers is responsible for generating the various function-related "frequencies" (bursting) or "states" (on/off) of activity.

Spatial contrast sensitivity in benign intracranial hypertension

Spatial Contrast Sensitivity (CS) was studied in 20 patients with benign intracranial hypertension (BIH). At presentation CS loss was found in 43% of the eyes, and impairment of visual acuity attributed to BIH in only 16%. Nine patients had blurred vision or visual obscurations, all of whom had abnormal CS. The clinical application of CS measurement in BIH for monitoring the progression or regression of the disease is illustrated by serial measurements in 11 patients. Progressive visual loss in longstanding papilloedema and improvement of visual function in subsiding papilloedema can occur without any change in Snellen acuity or visual field charting.

Spatiotemporal receptive fields: a dynamical model derived from cortical architectonics

We assume that the mammalian neocortex is built up out of some six layers which differ in their morphology and their external connections. Intrinsic connectivity is largely excitatory, leading to a considerable amount of positive feedback. The majority of cortical neurons can be divided into two main classes: the pyramidal cells, which are said to be excitatory, and local cells (most notably the non-spiny stellate cells), which are said to be inhibitory. The form of the dendritic and axonal arborizations of both groups is discussed in detail. This results in a simplified model of the cortex as a stack of six layers with mutual connections determined by the principles of fibre anatomy. This stack can be treated as a multi-input-multi-output system by means of the linear systems theory of homogeneous layers. The detailed equations for the simulation are derived in the Appendix. The results of the simulations show that the temporal and spatial behaviour of an excitation distribution cannot be treated separately. Further, they indicate specific processing in the different layers and some independence from details of wiring. Finally, the simulation results are applied to the theory of visual receptive fields. This yields some insight into the mechanisms possibly underlying hypercomplexity, putative nonlinearities, lateral inhibition, oscillating cell responses, and velocity-dependent tuning curves.

The development of basic mechanisms of pattern vision: spatial frequency channels

The mature visual system possesses mechanisms that analyze visual inputs into bands of spatial frequency. This analysis appears to be important to several visual capabilities. We have investigated the development of these spatial-frequency channels in young infants. Experiment 1 used a masking paradigm to test 6-week-olds, 12-week-olds, and adults. The detectability of sine wave gratings of different spatial frequencies was measured in the presence and the absence of a narrowband noise masker. The 12-week data showed that at least two spatial-frequency channels with adultlike specificity are present at 12 weeks. The 6-week data did not reveal the presence of narrowband spatial-frequency channels. Experiment 2 used a different paradigm to investigate the same issue. The detectability of gratings composed of two sine wave components was measured in 6-week-olds and adults. The results were entirely consistent with those of experiment 1. The 12-week and adult data indicated the presence of narrowband spatial-frequency channels. The 6-week data did not. The results of these experiments suggest that the manner in which pattern information is processed changes fundamentally between 6 and 12 weeks of age.

The development of contrast constancy

The mature visual system possesses mechanisms that enable invariant perception of the contrast of an object and its features as the object undergoes changes in distance. This phenomenon, which has been called contrast constancy, obtains at suprathreshold contrasts only. Some models of contrast constancy assume the presence of narrowband spatial-frequency channels. An implication of M.S. Banks, B.R. Stephens, and E.E. Hartmann (1985, Journal of Experimental Child Psychology, 40, 501-527) is that contrast constancy should not be observed at 6 weeks but may be observed at 12 weeks. We examined this implication by investigating the development of contrast constancy in 6- and 12-week-old infants. Two sine wave gratings, differing in spatial frequency by a factor of 3, were presented side-by-side. The contrast of one grating was varied in order to estimate the contrast at which preference for the two gratings was equal. The equal preference points for 6-week-olds were predictable from their contrast thresholds. The 12-week-olds' equal preference points for low-contrast stimuli were predictable from their contrast thresholds, but those for intermediate and high-contrast stimuli were not. Thus, if one accepts the assumption that equal preference in infants is analogous to apparent contrast matches in adults, these data imply that contrast constancy is observed at 12 weeks but not 6 weeks. The perceptual consequences of this developmental transition are discussed.

Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters

Two-dimensional spatial linear filters are constrained by general uncertainty relations that limit their attainable information resolution for orientation, spatial frequency, and two-dimensional (2D) spatial position. The theoretical lower limit for the joint entropy, or uncertainty, of these variables is achieved by an optimal 2D filter family whose spatial weighting functions are generated by exponentiated bivariate second-order polynomials with complex coefficients, the elliptic generalization of the one-dimensional elementary functions proposed in Gabor's famous theory of communication [J. Inst. Electr. Eng. 93, 429 (1946)]. The set includes filters with various orientation bandwidths, spatial-frequency bandwidths, and spatial dimensions, favoring the extraction of various kinds of information from an image. Each such filter occupies an irreducible quantal volume (corresponding to an independent datum) in a four-dimensional information hyperspace whose axes are interpretable as 2D visual space, orientation, and spatial frequency, and thus such a filter set could subserve an optimally efficient sampling of these variables. Evidence is presented that the 2D receptive-field profiles of simple cells in mammalian visual cortex are well described by members of this optimal 2D filter family, and thus such visual neurons could be said to optimize the general uncertainty relations for joint 2D-spatial-2D-spectral information resolution. The variety of their receptive-field dimensions and orientation and spatial-frequency bandwidths, and the correlations among these, reveal several underlying constraints, particularly in width/length aspect ratio and principal axis organization, suggesting a polar division of labor in occupying the quantal volumes of information hyperspace.(ABSTRACT TRUNCATED AT 250 WORDS)

Theory of shape-invariant imaging systems

The human visual system exhibits two properties that may be useful in other pattern-recognition systems: images do not change their shape with changes in image size and resolution declines rapidly with distance from the center of fixation. We show here that, in general, shape invariance requires inhomogeneous resolution over image space in a manner similar to that of the human visual system. Thus shape-invariant systems must process less information when compared with uniform-resolution systems. Although shape-invariant systems can be rotationally invariant, they cannot, in general, be translationally invariant. The properties of shape-invariant systems are explored in the spatial-frequency domain using a modified Fourier transform called a scaled transform. The features of scaled transforms are discussed and their behavior illustrated in the image domain by using them to filter various images, including the dot, the line, and the edge. It is shown that the filtered profile of an edge is preserved when it passes through the origin of a scaled transform. This result suggests that scaled transforms may be useful in edge-detection algorithms.

Spatial visual channels in the Fourier plane

Properties of human spatial visual channels were studied in two-dimensional form by a signal detection masking paradigm. Tuning surfaces of contrast threshold elevation induced by a sinusoidal mask were generated for four Subjects, interpolated from an 11 X 11 Cartesian grid over the Fourier plane, and numerically Fourier transformed in two dimensions to infer putative filter profiles in the 2D space domain. Among the main findings in the 2D frequency domain were: (1) Threshold elevation surfaces are highly polar nonseparable--they cannot be described as the product of a spatial frequency tuning curve times an orientation tuning curve. (2) Iso-half-amplitude contours of the spectral tuning surfaces have a length/width elongation ratio of about 2:1. (3) Necessarily, resolution for spatial frequency and for orientation are in fundamental competition with 2D spatial resolution. By calculating the occupied area of the inferred filters both in the 2D space domain and in the 2D frequency domain, it was estimated that these mechanisms approach within a factor of 2.5 of the theoretical limit of joint resolution in the two 2D domains that can be derived by 2D generalization of Gabor's famous Theory of Communication (1946). Other classes of 2D filters, such as an ideal 2D bandpass filter, have joint 2D entropies which are suboptimal by a factor of 13 or more. Subject to the inherent constraints on inference from these 2D masking experiments, the evidence suggests that 2D spatial frequency channels can be described as elongated 2D spatial wave-packets which crudely resemble optimal forms for joint information resolution in the 2D spatial and 2D frequency domains.

Visual illusions without low spatial frequencies

It has been suggested that many geometrical illusions may be caused by the lower spatial frequencies in the Fourier spectra of these images. We have tested this hypothesis by constructing classical illusory figures out of a new dot stimulus, which is free of visible low spatial frequencies. In every case, not only does the illusion persist, but for the Müller-Lyer figures, which were measured quantitatively, the illusion magnitude is not significantly changed. We conclude that geometrical illusions are not primarily a consequence of low spatial frequencies in the illusory figures.

A model for the economical encoding of the visual image in cerebral cortex

We propose a model for the first stage of the cortical transformation of the visual image based on the principle that the cortex encodes the information with the minimum number of channels mathematically needed. We restrict our model to be consistent with the data on size adaptations, the known relationships of acuity and the inverse of magnification factor with eccentricity, and the electrophysiological findings on the physiological uniformity of the striate cortex. Assuming that each hypercolumn analyzes a limited spatial domain, we apply the sampling theorem to show that only 16 channels, composed of 4 sizes, are needed for dimension. The extension to 2 dimensions leads to a possible scheme for the number, spacing, and orientational disposition of the elements, together with predictions about the number of inputs from the eyes and the total number of hypercolumns. Since all these predictions are consistent with physical and neural estimates, we conclude that the cortex may analyze the image along the lines we suggest.

Spatial computation performed by simple and complex cells in the visual cortex of the cat

Simple and complex cells have been tested with drifting sine-wave and square-wave gratings. Despite the known differences in the response pattern of each cell type to drifting sine-wave gratings, the tuning curves for square-wave gratings for both cell types show a similar secondary response band peaking at one-third the preferred spatial frequency as determined from sine-wave studies. These results establish that both cell types respond predominantly to the third harmonic of square-wave gratings in this frequency range. At the simple cell stage, all the information required to specify the amplitude and phase for channel at a given orientation, direction and spatial frequency can be conveyed by four cells for a given subsection of visual space. At the complex cell stage, the cell's mean firing rate appears to represent the amplitude of a local Fourier coefficient, but phase information is not conveyed in the action potential code.

Phase relationships between adjacent simple cells in the visual cortex

Adjacent simple cells recorded and "isolated" simultaneously from the same microelectrode placement were usually tuned to the same orientation and spatial frequency. The responses of the members of these "spatial frequency pairs" to drifting sine-wave gratings were cross-correlates. Within the middle range of the spatial frequency selectivity curves, the responses of the paired cells differed in phase by approximately 90 percent. This phase relationship suggests that adjacent simple cells tuned to the same spatial frequency and orientation represent paired sine and cosine filters in terms of their processing of afferent spatial inputs and truncated sine and cosine filters in terms of the output of simple cells.

Tactile pattern perception

Because efforts to use the cutaneous sense for conveying speech and visual information have met with only partial success, it would be useful to understand better the pattern-sensing capabilities of touch. This paper is an account of the sensory and perceptual factors known or hypothesized to limit the tactile perception of simple two-dimensional patterns, with special attention to the limited spatial bandwidth of touch.