Change Blindness Is Influenced by Both Contrast Energy and Subjective Importance within Local Regions of the Image

Our visual system receives an enormous amount of information, but not all information is retained. This is exemplified by the fact that subjects fail to detect large changes in a visual scene, i.e., change-blindness. Current theories propose that our ability to detect these changes is influenced by the gist or interpretation of an image. On the other hand, stimulus-driven image features such as contrast energy dominate the representation in early visual cortex (De Valois and De Valois, 1988; Boynton et al., 1999; Olman et al., 2004; Mante and Carandini, 2005; Dumoulin et al., 2008). Here we investigated whether contrast energy contributes to our ability to detect changes within a visual scene. We compared the ability to detect changes in contrast energy together with changes to a measure of the interpretation of an image. We used subjective important aspects of the image as a measure of the interpretation of an image. We measured reaction times while manipulating the contrast energy and subjective important properties using the change blindness paradigm. Our results suggest that our ability to detect changes in a visual scene is not only influenced by the subjective importance, but also by contrast energy. Also, we find that contrast energy and subjective importance interact. We speculate that contrast energy and subjective important properties are not independently represented in the visual system. Thus, our results suggest that the information that is retained of a visual scene is both influenced by stimulus-driven information as well as the interpretation of a scene.

Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex

Neural responses in the visual cortex are variable, and there is now an abundance of data characterizing how the magnitude and structure of this variability depends on the stimulus. Current theories of cortical computation fail to account for these data; they either ignore variability altogether or only model its unstructured Poisson-like aspects. We develop a theory in which the cortex performs probabilistic inference such that population activity patterns represent statistical samples from the inferred probability distribution. Our main prediction is that perceptual uncertainty is directly encoded by the variability, rather than the average, of cortical responses. Through direct comparisons to previously published data as well as original data analyses, we show that a sampling-based probabilistic representation accounts for the structure of noise, signal, and spontaneous response variability and correlations in the primary visual cortex. These results suggest a novel role for neural variability in cortical dynamics and computations.

A step toward understanding the human ventral visual pathway

The human ventral visual pathway is implicated in higher order form processing, but the organizational principles within this region are not yet well understood. Recently, Lafer-Sousa, Conway, and Kanwisher (J Neurosci 36: 1682-1697, 2016) used functional magnetic resonance imaging to demonstrate that functional responses in the human ventral visual pathway share a broad homology with the those in macaque inferior temporal cortex, providing new evidence supporting the validity of the macaque as a model of the human visual system in this region. In addition, these results give new clues for understanding the organizational principles within the ventral visual pathway and the processing of higher order color and form, suggesting new avenues for research into this cortical region.

Low-Level Contrast Statistics of Natural Images Can Modulate the Frequency of Event-Related Potentials (ERP) in Humans

Humans are fast and accurate in categorizing complex natural images. It is, however, unclear what features of visual information are exploited by brain to perceive the images with such speed and accuracy. It has been shown that low-level contrast statistics of natural scenes can explain the variance of amplitude of event-related potentials (ERP) in response to rapidly presented images. In this study, we investigated the effect of these statistics on frequency content of ERPs. We recorded ERPs from human subjects, while they viewed natural images each presented for 70 ms. Our results showed that Weibull contrast statistics, as a biologically plausible model, explained the variance of ERPs the best, compared to other image statistics that we assessed. Our time-frequency analysis revealed a significant correlation between these statistics and ERPs' power within theta frequency band (~3–7 Hz). This is interesting, as theta band is believed to be involved in context updating and semantic encoding. This correlation became significant at ~110 ms after stimulus onset, and peaked at 138 ms. Our results show that not only the amplitude but also the frequency of neural responses can be modulated with low-level contrast statistics of natural images and highlights their potential role in scene perception.

A tilt after-effect for images of buildings: evidence of selectivity for the orientation of everyday scenes

The tilt after-effect (TAE) is thought to be a manifestation of gain control in mechanisms selective for spatial orientation in visual stimuli. It has been demonstrated with luminance-defined stripes, contrast-defined stripes, orientation-defined stripes and even with natural images. Of course, all images can be decomposed into a sum of stripes, so it should not be surprising to find a TAE when adapting and test images contain stripes that differ by 15° or so. We show this latter condition is not necessary for the TAE with natural images: adaptation to slightly tilted and vertically filtered houses produced a 'repulsive' bias in the perceived orientation of horizontally filtered houses. These results suggest gain control in mechanisms selective for spatial orientation in natural images.

Decoding the Semantic Content of Natural Movies from Human Brain Activity

One crucial test for any quantitative model of the brain is to show that the model can be used to accurately decode information from evoked brain activity. Several recent neuroimaging studies have decoded the structure or semantic content of static visual images from human brain activity. Here we present a decoding algorithm that makes it possible to decode detailed information about the object and action categories present in natural movies from human brain activity signals measured by functional MRI. Decoding is accomplished using a hierarchical logistic regression (HLR) model that is based on labels that were manually assigned from the WordNet semantic taxonomy. This model makes it possible to simultaneously decode information about both specific and general categories, while respecting the relationships between them. Our results show that we can decode the presence of many object and action categories from averaged blood-oxygen level-dependent (BOLD) responses with a high degree of accuracy (area under the ROC curve > 0.9). Furthermore, we used this framework to test whether semantic relationships defined in the WordNet taxonomy are represented the same way in the human brain. This analysis showed that hierarchical relationships between general categories and atypical examples, such as organism and plant, did not seem to be reflected in representations measured by BOLD fMRI.

Categorically distinct types of receptive fields in early visual cortex

In the visual cortex, distinct types of neurons have been identified based on cellular morphology, response to injected current, or expression of specific markers, but neurophysiological studies have revealed visual receptive field (RF) properties that appear to be on a continuum, with only two generally recognized classes: simple and complex. Most previous studies have characterized visual responses of neurons using stereotyped stimuli such as bars, gratings, or white noise and simple system identification approaches (e.g., reverse correlation). Here we estimate visual RF models of cortical neurons using visually rich natural image stimuli and regularized regression system identification methods and characterize their spatial tuning, temporal dynamics, spatiotemporal behavior, and spiking properties. We quantitatively demonstrate the existence of three functionally distinct categories of simple cells, distinguished by their degree of orientation selectivity (isotropic or oriented) and the nature of their output nonlinearity (expansive or compressive). In addition, these three types have differing average values of several other properties. Cells with nonoriented RFs tend to have smaller RFs, shorter response durations, no direction selectivity, and high reliability. Orientation-selective neurons with an expansive output nonlinearity have Gabor-like RFs, lower spontaneous activity and responsivity, and spiking responses with higher sparseness. Oriented RFs with a compressive nonlinearity are spatially nondescript and tend to show longer response latency. Our findings indicate multiple physiologically defined types of RFs beyond the simple/complex dichotomy, suggesting that cortical neurons may have more specialized functional roles rather than lying on a multidimensional continuum.

Spatial structure of neuronal receptive field in awake monkey secondary visual cortex (V2)

Visual processing depends critically on the receptive field (RF) properties of visual neurons. However, comprehensive characterization of RFs beyond the primary visual cortex (V1) remains a challenge. Here we report fine RF structures in secondary visual cortex (V2) of awake macaque monkeys, identified through a projection pursuit regression analysis of neuronal responses to natural images. We found that V2 RFs could be broadly classified as V1-like (typical Gabor-shaped subunits), ultralong (subunits with high aspect ratios), or complex-shaped (subunits with multiple oriented components). Furthermore, single-unit recordings from functional domains identified by intrinsic optical imaging showed that neurons with ultralong RFs were primarily localized within pale stripes, whereas neurons with complex-shaped RFs were more concentrated in thin stripes. Thus, by combining single-unit recording with optical imaging and a computational approach, we identified RF subunits underlying spatial feature selectivity of V2 neurons and demonstrated the functional organization of these RF properties.

Rapid categorization of natural face images in the infant right hemisphere

Human performance at categorizing natural visual images surpasses automatic algorithms, but how and when this function arises and develops remain unanswered. We recorded scalp electrical brain activity in 4–6 months infants viewing images of objects in their natural background at a rapid rate of 6 images/second (6 Hz). Widely variable face images appearing every 5 stimuli generate an electrophysiological response over the right hemisphere exactly at 1.2 Hz (6 Hz/5). This face-selective response is absent for phase-scrambled images and therefore not due to low-level information. These findings indicate that right lateralized face-selective processes emerge well before reading acquisition in the infant brain, which can perform figure-ground segregation and generalize face-selective responses across changes in size, viewpoint, illumination as well as expression, age and gender. These observations made with a highly sensitive and objective approach open an avenue for clarifying the developmental course of natural image categorization in the human brain.

DOI:http://dx.doi.org/10.7554/eLife.06564.001

Putting names to faces can sometimes be challenging, but humans are generally extremely good at recognising faces. Computers, on the other hand, often find it difficult to categorize a face as a face. Indeed, a major challenge in face recognition arises because faces come in many different shapes and sizes. Moreover, both the lighting conditions and the orientation of the head can change, which makes the challenge even more difficult.

Young infants also show a preference for pictures of human faces over nonsense images, which suggests that the ability to recognise faces is at least partly hard-wired. Neuroimaging studies have revealed that face recognition depends on activity in specific regions of the right hemisphere of the brain, and adults who sustain damage to these regions lose their face recognition skills.

De Heering and Rossion have now provided the first evidence that the right hemisphere is specialized for distinguishing between natural images of faces and ‘non-face objects’ in infants as young as 4 to 6 months. By using scalp electrodes to record electrical activity in the brain as the infants viewed images on a screen, De Heering and Rossion showed that photographs of human faces triggered a distinct pattern of electrical activity in the right hemisphere: this pattern was clearly different to the patterns triggered by photographs of animals or objects.

A consistent response was triggered by faces of different genders and expressions, and by faces presented from various viewpoints and under different lighting conditions. In a control experiment, De Heering and Rossion demonstrated that low-level visual features such as differences in luminance or contrast do not contribute to this selective response to faces.

These results argue against the idea that face perception only becomes assigned to the right hemisphere of the brain when children learn to read (that is, when language processing begins to occupy parts of the left hemisphere). By generating significant responses in a short period of time (just five minutes or less), the protocol developed by De Heering and Rossion has the potential to prove very useful to researchers investigating developmental changes to the perception of visual images during childhood.

DOI:http://dx.doi.org/10.7554/eLife.06564.002

Uncomfortable images produce non-sparse responses in a model of primary visual cortex

The processing of visual information by the nervous system requires significant metabolic resources. To minimize the energy needed, our visual system appears to be optimized to encode typical natural images as efficiently as possible. One consequence of this is that some atypical images will produce inefficient, non-optimal responses. Here, we show that images that are reported to be uncomfortable to view, and that can trigger migraine attacks and epileptic seizures, produce relatively non-sparse responses in a model of the primary visual cortex. In comparison with the responses to typical inputs, responses to aversive images were larger and less sparse. We propose that this difference in the neural population response may be one cause of visual discomfort in the general population, and can produce more extreme responses in clinical populations such as migraine and epilepsy sufferers.

Fast periodic presentation of natural images reveals a robust face-selective electrophysiological response in the human brain

We designed a fast periodic visual stimulation approach to identify an objective signature of face categorization incorporating both visual discrimination (from nonface objects) and generalization (across widely variable face exemplars). Scalp electroencephalographic (EEG) data were recorded in 12 human observers viewing natural images of objects at a rapid frequency of 5.88 images/s for 60 s. Natural images of faces were interleaved every five stimuli, i.e., at 1.18 Hz (5.88/5). Face categorization was indexed by a high signal-to-noise ratio response, specifically at an oddball face stimulation frequency of 1.18 Hz and its harmonics. This face-selective periodic EEG response was highly significant for every participant, even for a single 60-s sequence, and was generally localized over the right occipitotemporal cortex. The periodicity constraint and the large selection of stimuli ensured that this selective response to natural face images was free of low-level visual confounds, as confirmed by the absence of any oddball response for phase-scrambled stimuli. Without any subtraction procedure, time-domain analysis revealed a sequence of differential face-selective EEG components between 120 and 400 ms after oddball face image onset, progressing from medial occipital (P1-faces) to occipitotemporal (N1-faces) and anterior temporal (P2-faces) regions. Overall, this fast periodic visual stimulation approach provides a direct signature of natural face categorization and opens an avenue for efficiently measuring categorization responses of complex visual stimuli in the human brain.

Retina-V1 model of detectability across the visual field

A practical model is proposed for predicting the detectability of targets at arbitrary locations in the visual field, in arbitrary gray scale backgrounds, and under photopic viewing conditions. The major factors incorporated into the model include (a) the optical point spread function of the eye, (b) local luminance gain control (Weber's law), (c) the sampling array of retinal ganglion cells, (d) orientation and spatial frequency-dependent contrast masking, (e) broadband contrast masking, and (f) efficient response pooling. The model is tested against previously reported threshold measurements on uniform backgrounds (the ModelFest data set and data from Foley, Varadharajan, Koh, & Farias, 2007) and against new measurements reported here for several ModelFest targets presented on uniform, 1/f noise, and natural backgrounds at retinal eccentricities ranging from 0° to 10°. Although the model has few free parameters, it is able to account quite well for all the threshold measurements.

Local masking in natural images: a database and analysis

Studies of visual masking have provided a wide range of important insights into the processes involved in visual coding. However, very few of these studies have employed natural scenes as masks. Little is known on how the particular features found in natural scenes affect visual detection thresholds and how the results obtained using unnatural masks relate to the results obtained using natural masks. To address this issue, this paper describes a psychophysical study designed to obtain local contrast detection thresholds for a database of natural images. Via a three-alternative forced-choice experiment, we measured thresholds for detecting 3.7 cycles/° vertically oriented log-Gabor noise targets placed within an 85 × 85-pixels patch (1.9° patch) drawn from 30 natural images from the CSIQ image database (Larson & Chandler, Journal of Electronic Imaging, 2010). Thus, for each image, we obtained a masking map in which each entry in the map denotes the root mean squared contrast threshold for detecting the log-Gabor noise target at the corresponding spatial location in the image. From qualitative observations we found that detection thresholds were affected by several patch properties such as visual complexity, fineness of textures, sharpness, and overall luminance. Our quantitative analysis shows that except for the sharpness measure (correlation coefficient of 0.7), the other tested low-level mask features showed a weak correlation (correlation coefficients less than or equal to 0.52) with the detection thresholds. Furthermore, we evaluated the performance of a computational contrast gain control model that performed fairly well with an average correlation coefficient of 0.79 in predicting the local contrast detection thresholds. We also describe specific choices of parameters for the gain control model. The objective of this database is to provide researchers with a large ground-truth dataset in order to further investigate the properties of the human visual system using natural masks.

Simulated disparity and peripheral blur interact during binocular fusion

We have developed a low-cost, practical gaze-contingent display in which natural images are presented to the observer with dioptric blur and stereoscopic disparity that are dependent on the three-dimensional structure of natural scenes. Our system simulates a distribution of retinal blur and depth similar to that experienced in real-world viewing conditions by emmetropic observers. We implemented the system using light-field photographs taken with a plenoptic camera which supports digital refocusing anywhere in the images. We coupled this capability with an eye-tracking system and stereoscopic rendering. With this display, we examine how the time course of binocular fusion depends on depth cues from blur and stereoscopic disparity in naturalistic images. Our results show that disparity and peripheral blur interact to modify eye-movement behavior and facilitate binocular fusion, and the greatest benefit was gained by observers who struggled most to achieve fusion. Even though plenoptic images do not replicate an individual’s aberrations, the results demonstrate that a naturalistic distribution of depth-dependent blur may improve 3-D virtual reality, and that interruptions of this pattern (e.g., with intraocular lenses) which flatten the distribution of retinal blur may adversely affect binocular fusion.

Highly informative natural scene regions increase microsaccade production during visual scanning

Classical image statistics, such as contrast, entropy, and the correlation between central and nearby pixel intensities, are thought to guide ocular fixation targeting. However, these statistics are not necessarily task relevant and therefore do not provide a complete picture of the relationship between informativeness and ocular targeting. Moreover, it is not known whether either informativeness or classical image statistics affect microsaccade production; thus, the role of microsaccades in information acquisition is also unknown. The objective quantification of the informativeness of a scene region is a major challenge, because it can vary with both image features and the task of the viewer. Thus, previous definitions of informativeness suffered from subjectivity and inconsistency across studies. Here we developed an objective measure of informativeness based on fixation consistency across human observers, which accounts for both bottom-up and top-down influences in ocular targeting. We then analyzed fixations in more versus less informative image regions in relation to classical statistics. Observers generated more microsaccades on more informative than less informative image regions, and such regions also exhibited low redundancy in their classical statistics. Increased microsaccade production was not explained by increased fixation duration, suggesting that the visual system specifically uses microsaccades to heighten information acquisition from informative regions.

Visual discomfort and depth-of-field

Visual discomfort has been reported for certain visual stimuli and under particular viewing conditions, such as stereoscopic viewing. In stereoscopic viewing, visual discomfort can be caused by a conflict between accommodation and convergence cues that may specify different distances in depth. Earlier research has shown that depth-of-field, which is the distance range in depth in the scene that is perceived to be sharp, influences both the perception of egocentric distance to the focal plane, and the distance range in depth between objects in the scene. Because depth-of-field may also be in conflict with convergence and the accommodative state of the eyes, we raised the question of whether depth-of-field affects discomfort when viewing stereoscopic photographs. The first experiment assessed whether discomfort increases when depth-of-field is in conflict with coherent accommodation-convergence cues to distance in depth. The second experiment assessed whether depth-of-field influences discomfort from a pre-existing accommodation-convergence conflict. Results showed no effect of depth-of-field on visual discomfort. These results suggest therefore that depth-of-field can be used as a cue to depth without inducing discomfort in the viewer, even when cue conflicts are large.

Symmetry in context: salience of mirror symmetry in natural patterns

Symmetry is a biologically relevant, mathematically involving, and aesthetically compelling visual phenomenon. Mirror symmetry detection is considered particularly rapid and efficient, based on experiments with random noise. Symmetry detection in natural settings, however, is often accomplished against structured backgrounds. To measure salience of symmetry in diverse contexts, we assembled mirror symmetric patterns from 101 natural textures. Temporal thresholds for detecting the symmetry axis ranged from 28 to 568 ms indicating a wide range of salience (1/Threshold). We built a model for estimating symmetry-energy by connecting pairs of mirror-symmetric filters that simulated cortical receptive fields. The model easily identified the axis of symmetry for all patterns. However, symmetry-energy quantified at this axis correlated weakly with salience. To examine context effects on symmetry detection, we used the same model to estimate approximate symmetry resulting from the underlying texture throughout the image. Magnitudes of approximate symmetry at flanking and orthogonal axes showed strong negative correlations with salience, revealing context interference with symmetry detection. A regression model that included the context-based measures explained the salience results, and revealed why perceptual symmetry can differ from mathematical characterizations. Using natural patterns thus produces new insights into symmetry perception and its possible neural circuits.

Small Saccades and Image Complexity during Free Viewing of Natural Images in Schizophrenia

In schizophrenia, patients display dysfunctions during the execution of simple visual tasks such as antisaccade or smooth pursuit. In more ecological scenarios, such as free viewing of natural images, patients appear to make fewer and longer visual fixations and display shorter scanpaths. It is not clear whether these measurements reflect alterations in their proficiency to perform basic eye movements, such as saccades and fixations, or are related to high-level mechanisms, such as exploration or attention. We utilized free exploration of natural images of different complexities as a model of an ecological context where normally operative mechanisms of visual control can be accurately measured. We quantified visual exploration as Euclidean distance, scanpaths, saccades, and visual fixation, using the standard SR-Research eye tracker algorithm (SR). We then compared this result with a computation that includes microsaccades (EM). We evaluated eight schizophrenia patients and corresponding healthy controls (HC). Next, we tested whether the decrement in the number of saccades and fixations, as well as their increment in duration reported previously in schizophrenia patients, resulted from the increasing occurrence of undetected microsaccades. We found that when utilizing the standard SR algorithm, patients displayed shorter scanpaths as well as fewer and shorter saccades and fixations. When we employed the EM algorithm, the differences in these parameters between patients and HC were no longer significant. On the other hand, we found that image complexity plays an important role in exploratory behaviors, demonstrating that this factor explains most of differences between eye-movement behaviors in schizophrenia patients. These results help elucidate the mechanisms of visual motor control that are affected in schizophrenia and contribute to the finding of adequate markers for diagnosis and treatment for this condition.

Repetition blindness for natural images of objects with viewpoint changes

When stimuli are repeated in a rapid serial visual presentation (RSVP), observers sometimes fail to report the second occurrence of a target. This phenomenon is referred to as “repetition blindness” (RB). We report an RSVP experiment with photographs in which we manipulated object viewpoints between the first and second occurrences of a target (0°, 45°, or 90° changes), and spatial frequency (SF) content. Natural images were spatially filtered to produce low, medium, or high SF stimuli. RB was observed for all filtering conditions. Surprisingly, for full-spectrum (FS) images, RB increased significantly as the viewpoint reached 90°. For filtered images, a similar pattern of results was found for all conditions except for medium SF stimuli. These findings suggest that object recognition in RSVP are subtended by viewpoint-specific representations for all spatial frequencies except medium ones.

Low-level contrast statistics are diagnostic of invariance of natural textures

Texture may provide important clues for real world object and scene perception. To be reliable, these clues should ideally be invariant to common viewing variations such as changes in illumination and orientation. In a large image database of natural materials, we found textures with low-level contrast statistics that varied substantially under viewing variations, as well as textures that remained relatively constant. This led us to ask whether textures with constant contrast statistics give rise to more invariant representations compared to other textures. To test this, we selected natural texture images with either high (HV) or low (LV) variance in contrast statistics and presented these to human observers. In two distinct behavioral categorization paradigms, participants more often judged HV textures as "different" compared to LV textures, showing that textures with constant contrast statistics are perceived as being more invariant. In a separate electroencephalogram (EEG) experiment, evoked responses to single texture images (single-image ERPs) were collected. The results show that differences in contrast statistics correlated with both early and late differences in occipital ERP amplitude between individual images. Importantly, ERP differences between images of HV textures were mainly driven by illumination angle, which was not the case for LV images: there, differences were completely driven by texture membership. These converging neural and behavioral results imply that some natural textures are surprisingly invariant to illumination changes and that low-level contrast statistics are diagnostic of the extent of this invariance.

Contrast-independent biologically inspired motion detection

Optic flow, i.e., retinal image movement resulting from ego-motion, is a crucial source of information used for obstacle avoidance and course control in flying insects. Optic flow analysis may prove promising for mobile robotics although it is currently not among the standard techniques. Insects have developed a computationally cheap analysis mechanism for image motion. Detailed computational models, the so-called elementary motion detectors (EMDs), describe motion detection in insects. However, the technical application of EMDs is complicated by the strong effect of local pattern contrast on their motion response. Here we present augmented versions of an EMD, the (s)cc-EMDs, which normalise their responses for contrast and thereby reduce the sensitivity to contrast changes. Thus, velocity changes of moving natural images are reflected more reliably in the detector response. The (s)cc-EMDs can easily be implemented in hardware and software and can be a valuable novel visual motion sensor for mobile robots.

Bayesian reconstruction of natural images from human brain activity

Recent studies have used fMRI signals from early visual areas to reconstruct simple geometric patterns. Here, we demonstrate a new Bayesian decoder that uses fMRI signals from early and anterior visual areas to reconstruct complex natural images. Our decoder combines three elements: a structural encoding model that characterizes responses in early visual areas, a semantic encoding model that characterizes responses in anterior visual areas, and prior information about the structure and semantic content of natural images. By combining all these elements, the decoder produces reconstructions that accurately reflect both the spatial structure and semantic category of the objects contained in the observed natural image. Our results show that prior information has a substantial effect on the quality of natural image reconstructions. We also demonstrate that much of the variance in the responses of anterior visual areas to complex natural images is explained by the semantic category of the image alone.

Contrast sensitivity in natural scenes depends on edge as well as spatial frequency structure

The contrast sensitivity function is routinely measured in the laboratory with sine-wave gratings presented on homogenous gray backgrounds; natural images are instead composed of a broad range of spatial and temporal structures. In order to extend channel-based models of visual processing to more natural conditions, we examined how contrast sensitivity varies with the context in which it is measured. We report that contrast sensitivity is quite different under laboratory than natural viewing conditions: adaptation or masking with natural scenes attenuates contrast sensitivity at low spatial and temporal frequencies. Expressed another way, viewing stimuli presented on homogenous screens overcomes chronic adaptation to the natural environment and causes a sharp, unnatural increase in sensitivity to low spatial and temporal frequencies. Consequently, the standard contrast sensitivity function is a poor indicator of sensitivity to structure in natural scenes. The magnitude of masking by natural scenes is relatively independent of local contrast but depends strongly on the density of edges even though neither greatly affects the local amplitude spectrum. These results suggest that sensitivity to spatial structure in natural scenes depends on the distribution of local edges as well as the local amplitude spectrum.

Do we know what the early visual system does?

We can claim that we know what the visual system does once we can predict neural responses to arbitrary stimuli, including those seen in nature. In the early visual system, models based on one or more linear receptive fields hold promise to achieve this goal as long as the models include nonlinear mechanisms that control responsiveness, based on stimulus context and history, and take into account the nonlinearity of spike generation. These linear and nonlinear mechanisms might be the only essential determinants of the response, or alternatively, there may be additional fundamental determinants yet to be identified. Research is progressing with the goals of defining a single "standard model" for each stage of the visual pathway and testing the predictive power of these models on the responses to movies of natural scenes. These predictive models represent, at a given stage of the visual pathway, a compact description of visual computation. They would be an invaluable guide for understanding the underlying biophysical and anatomical mechanisms and relating neural responses to visual perception.

The receptive-field organization of simple cells in primary visual cortex of ferrets under natural scene stimulation

The responses of simple cells in primary visual cortex to sinusoidal gratings can primarily be predicted from their spatial receptive fields, as mapped using spots or bars. Although this quasilinearity is well documented, it is not clear whether it holds for complex natural stimuli. We recorded from simple cells in the primary visual cortex of anesthetized ferrets while stimulating with flashed digitized photographs of natural scenes. We applied standard reverse-correlation methods to quantify the average natural stimulus that invokes a neuronal response. Although these maps cannot be the receptive fields, we find that they still predict the preferred orientation of grating for each cell very well (r = 0.91); they do not predict the spatial-frequency tuning. Using a novel application of the linear reconstruction method called regularized pseudoinverse, we were able to recover high-resolution receptive-field maps from the responses to a relatively small number of natural scenes. These receptive-field maps not only predict the optimum orientation of each cell (r = 0.96) but also the spatial-frequency optimum (r = 0.89); the maps also predict the tuning bandwidths of many cells. Therefore, our first conclusion is that the tuning preferences of the cells are primarily linear and constant across stimulus type. However, when we used these maps to predict the actual responses of the cells to natural scenes, we did find evidence of expansive output nonlinearity and nonlinear influences from outside the classical receptive fields, orientation tuning, and spatial-frequency tuning.