Temporal integration of spatially filtered visual images

Backward masking reveals coarse-to-fine dynamics in human V1

Natural images exhibit luminance variations aligned across a broad spectrum of spatial frequencies (SFs). It has been proposed that, at early stages of processing, the coarse signals carried by the low SF (LSF) of the visual input are sent rapidly from primary visual cortex (V1) to ventral, dorsal and frontal regions to form a coarse representation of the input, which is later sent back to V1 to guide the processing of fine-grained high SFs (i.e., HSF). We used functional resonance imaging (fMRI) to investigate the role of human V1 in the coarse-to-fine integration of visual input. We disrupted the processing of the coarse and fine content of full-spectrum human face stimuli via backward masking of selective SF ranges (LSFs: <1.75cpd and HSFs: >1.75cpd) at specific times (50, 83, 100 or 150ms). In line with coarse-to-fine proposals, we found that (1) the selective masking of stimulus LSF disrupted V1 activity in the earliest time window, and progressively decreased in influence, while (2) an opposite trend was observed for the masking of stimulus' HSF. This pattern of activity was found in V1, as well as in ventral (i.e. the Fusiform Face area, FFA), dorsal and orbitofrontal regions. We additionally presented subjects with contrast negated stimuli. While contrast negation significantly reduced response amplitudes in the FFA, as well as coupling between FFA and V1, coarse-to-fine dynamics were not affected by this manipulation. The fact that V1 response dynamics to strictly identical stimulus sets differed depending on the masked scale adds to growing evidence that V1 role goes beyond the early and quasi-passive transmission of visual information to the rest of the brain. It instead indicates that V1 may yield a 'spatially registered common forum' or 'blackboard' that integrates top-down inferences with incoming visual signals through its recurrent interaction with high-level regions located in the inferotemporal, dorsal and frontal regions.

Impact of glaucoma on the spatial frequency processing of scenes in central vision

Glaucoma is an eye disease characterized by a progressive vision loss usually starting in peripheral vision. However, a deficit for scene categorization is observed even in the preserved central vision of patients with glaucoma. We assessed the processing and integration of spatial frequencies in the central vision of patients with glaucoma during scene categorization, considering the severity of the disease, in comparison to age-matched controls. In the first session, participants had to categorize scenes filtered in low-spatial frequencies (LSFs) and high-spatial frequencies (HSFs) as a natural or an artificial scene. Results showed that the processing of spatial frequencies was impaired only for patients with severe glaucoma, in particular for HFS scenes. In the light of proactive models of visual perception, we investigated how LSF could guide the processing of HSF in a second session. We presented hybrid scenes (combining LSF and HSF from two scenes belonging to the same or different semantic category). Participants had to categorize the scene filtered in HSF while ignoring the scene filtered in LSF. Surprisingly, results showed that the semantic influence of LSF on HSF was greater for patients with early glaucoma than controls, and then disappeared for the severe cases. This study shows that a progressive destruction of retinal ganglion cells affects the spatial frequency processing in central vision. This deficit may, however, be compensated by increased reliance on predictive mechanisms at early stages of the disease which would however decline in more severe cases.

The Effects of Normal Aging, Subjective Cognitive Decline, Mild Cognitive Impairment, or Alzheimer’s Disease on Visual Search

Background: Alzheimer’s disease (AD) has been confirmed as an influencing factor of visual impairment, but potential concomitant effects on visual and cognitive performance are not well understood. Objective: To provide a new method for early screening of Alzheimer’s disease and further explore the theoretical mechanism of the decline of whole visual and cognitive performance in AD. Methods: We studied 60 individuals without dementia as normal control (NC), 74 individuals with subjective cognitive decline (SCD), 60 individuals with amnesia mild cognitive impairment (aMCI), and 75 patients with AD on a battery of tests designed to measure multiple aspects of basic and higher-order visual perception and cognition. All subjects performed on same visual and cognitive test batteries. Results: The results showed both of four groups, with the stimulus-presentation time being longer, the visual-search performance improved, and both the eye interest-area first fixation duration and the interest-area-fixation count increased. Particularly under the noise-masking condition, the AD group performed the worst at stimulus-presentation times between 300 and 900 ms. The aMCI group, but not the SCD group, performed worse than the NC group at the stimulus-presentation time of either 300 or 500 ms. The interest-area-fixation count was higher in all the patient groups than that in the NC group, and distinguishable between participants with AD and those with SCD or aMCI. Conclusion: The visual-search performance combined with eye-movement tracking under the noise-masking condition can be used for distinguishing AD from normal aging, SCD, and aMCI.

Coarse-to-Fine(r) Automatic Familiar Face Recognition in the Human Brain

At what level of spatial resolution can the human brain recognize a familiar face in a crowd of strangers? Does it depend on whether one approaches or rather moves back from the crowd? To answer these questions, 16 observers viewed different unsegmented images of unfamiliar faces alternating at 6 Hz, with spatial frequency (SF) content progressively increasing (i.e., coarse-to-fine) or decreasing (fine-to-coarse) in different sequences. Variable natural images of celebrity faces every sixth stimulus generated an objective neural index of single-glanced automatic familiar face recognition (FFR) at 1 Hz in participants' electroencephalogram (EEG). For blurry images increasing in spatial resolution, the neural FFR response over occipitotemporal regions emerged abruptly with additional cues at about 6.3-8.7 cycles/head width, immediately reaching amplitude saturation. When the same images progressively decreased in resolution, the FFR response disappeared already below 12 cycles/head width, thus providing no support for a predictive coding hypothesis. Overall, these observations indicate that rapid automatic recognition of heterogenous natural views of familiar faces is achieved from coarser visual inputs than generally thought, and support a coarse-to-fine FFR dynamics in the human brain.

Semantic and Physical Properties of Peripheral Vision Are Used for Scene Categorization in Central Vision

Theories of visual recognition postulate that our ability to understand our visual environment at a glance is based on the extraction of the gist of the visual scene, a first global and rudimentary visual representation. Gist perception would be based on the rapid analysis of low spatial frequencies in the visual signal and would allow a coarse categorization of the scene. We aimed to study whether the low spatial resolution information available in peripheral vision could modulate the processing of visual information presented in central vision. We combined behavioral measures (Experiments 1 and 2) and fMRI measures (Experiment 2). Participants categorized a scene presented in central vision (artificial vs. natural categories) while ignoring another scene, either semantically congruent or incongruent, presented in peripheral vision. The two scenes could either share the same physical properties (similar amplitude spectrum and spatial configuration) or not. Categorization of the central scene was impaired by a semantically incongruent peripheral scene, in particular when the two scenes were physically similar. This semantic interference effect was associated with increased activation of the inferior frontal gyrus. When the two scenes were semantically congruent, the dissimilarity of their physical properties impaired the categorization of the central scene. This effect was associated with increased activation in occipito-temporal areas. In line with the hypothesis of predictive mechanisms involved in visual recognition, results suggest that semantic and physical properties of the information coming from peripheral vision would be automatically used to generate predictions that guide the processing of signal in central vision.

A Neural Network Model With Gap Junction for Topological Detection

Visual information processing in the brain goes from global to local. A large volume of experimental studies has suggested that among global features, the brain perceives the topological information of an image first. Here, we propose a neural network model to elucidate the underlying computational mechanism. The model consists of two parts. The first part is a neural network in which neurons are coupled through gap junctions, mimicking the neural circuit formed by alpha ganglion cells in the retina. Gap junction plays a key role in the model, which, on one hand, facilitates the synchronized firing of a neuron group covering a connected region of an image, and on the other hand, staggers the firing moments of different neuron groups covering disconnected regions of the image. These two properties endow the network with the capacity of detecting the connectivity and closure of images. The second part of the model is a read-out neuron, which reads out the topological information that has been converted into the number of synchronized firings in the retina network. Our model provides a simple yet effective mechanism for the neural system to detect the topological information of images in ultra-speed.

Perception of the difference between past and present stimulus: A rare orientation illusion may indicate incidental access to prediction error-like signals

A popular model for sensory processing, known as predictive coding, proposes that incoming signals are iteratively compared with top-down predictions along a hierarchical processing scheme. At each step, error signals arising from differences between actual input and prediction are forwarded and recurrently minimized by updating internal models to finally be “explained away”. However, the neuronal mechanisms underlying such computations and their limitations in processing speed are largely unknown. Further, it remains unclear at which step of cortical processing prediction errors are explained away, if at all. In the present study, human subjects briefly viewed the superposition of two orthogonally oriented gratings followed by abrupt removal of one orientation after either 33 or 200 milliseconds. Instead of strictly seeing the remaining orientation, observers report rarely but highly significantly an illusory percept of the arithmetic difference between previous and actual orientations. Previous findings in cats using the identical paradigm suggest that such difference signals are inherited from first steps of visual cortical processing. In light of early modeling accounts of predictive coding, in which visual neurons were interpreted as residual error detectors signaling the difference between actual input and its temporal prediction based on past input, our data may indicate continued access to residual errors. Such strategy permits time-critical perceptual decision making across a spectrum of competing internal signals up to the highest levels of processing. Thus, the occasional appearance of a prediction error-like illusory percept may uncover maintained flexibility at perceptual decision stages when subjects cope with highly dynamic and ambiguous visual stimuli.

Leader-Based Multi-Scale Attention Deep Architecture for Person Re-Identification

Person re-identification (re-id) aims to match people across non-overlapping camera views in a public space. This is a challenging problem because the people captured in surveillance videos often wear similar clothing. Consequently, the differences in their appearance are typically subtle and only detectable at particular locations and scales. In this paper, we propose a deep re-id network (MuDeep) that is composed of two novel types of layers - a multi-scale deep learning layer, and a leader-based attention learning layer. Specifically, the former learns deep discriminative feature representations at different scales, while the latter utilizes the information from multiple scales to lead and determine the optimal weightings for each scale. The importance of different spatial locations for extracting discriminative features is learned explicitly via our leader-based attention learning layer. Extensive experiments are carried out to demonstrate that the proposed MuDeep outperforms the state-of-the-art on a number of benchmarks and has a better generalization ability under a domain generalization setting.

Coarse-to-fine information integration in human vision

Coarse-to-fine theories of vision propose that the coarse information carried by the low spatial frequencies (LSF) of visual input guides the integration of finer, high spatial frequency (HSF) detail. Whether and how LSF modulates HSF processing in naturalistic broad-band stimuli is still unclear. Here we used multivariate decoding of EEG signals to separate the respective contribution of LSF and HSF to the neural response evoked by broad-band images. Participants viewed images of human faces, monkey faces and phase-scrambled versions that were either broad-band or filtered to contain LSF or HSF. We trained classifiers on EEG scalp-patterns evoked by filtered scrambled stimuli and evaluated the derived models on broad-band scrambled and intact trials. We found reduced HSF contribution when LSF was informative towards image content, indicating that coarse information does guide the processing of fine detail, in line with coarse-to-fine theories. We discuss the potential cortical mechanisms underlying such coarse-to-fine feedback.

Potential downside of high initial visual acuity

Children who are treated for congenital cataracts later exhibit impairments in configural face analysis. This has been explained in terms of a critical period for the acquisition of normal face processing. Here, we consider a more parsimonious account according to which deficits in configural analysis result from the abnormally high initial retinal acuity that children treated for cataracts experience, relative to typical newborns. According to this proposal, the initial period of low retinal acuity characteristic of normal visual development induces extended spatial processing in the cortex that is important for configural face judgments. As a computational test of this hypothesis, we examined the effects of training with high-resolution or blurred images, and staged combinations, on the receptive fields and performance of a convolutional neural network. The results show that commencing training with blurred images creates receptive fields that integrate information across larger image areas and leads to improved performance and better generalization across a range of resolutions. These findings offer an explanation for the observed face recognition impairments after late treatment of congenital blindness, suggest an adaptive function for the acuity trajectory in normal development, and provide a scheme for improving the performance of computational face recognition systems.

Are Spatial Frequencies Integrated from Coarse to Fine?

The existence of a temporal anisotropy in the integration of spatial frequencies, such that spatial frequencies are integrated more effectively if they are available from low to high through time, has been examined in a series of experiments. In the first experiment, the first three harmonics of a square wave were presented in a low-to-high or a high-to-low sequence in a temporal two-interval forced-choice experiment. Subjects were asked to indicate which sequence appeared to resemble a square wave more. A high-to-low sequence of spatial frequencies was judged to more resemble the target than the low-to-high sequence. These results support a temporal anisotropy in the integration of spatial frequencies of exactly the opposite form to that suggested from previous results. Further experiments established that this was not due to task differences or to subjects basing their decision on the final spatial frequency shown. An interpretation is offered in which an isotropic mechanism for spatial-frequency integration is combined with a recency bias.

Spatio-temporal properties of letter crowding

Crowding between adjacent letters has been investigated primarily as a spatial effect. The purpose of this study was to investigate the spatio-temporal properties of letter crowding. Specifically, we examined the systematic changes in the degradation effects in letter identification performance when adjacent letters were presented with a temporal asynchrony, as a function of letter separation and between the fovea and the periphery. We measured proportion-correct performance for identifying the middle target letter in strings of three lowercase letters at the fovea and 10° in the inferior visual field, for a range of center-to-center letter separations and a range of stimulus onset asynchronies (SOA) between the target and flanking letters (positive SOAs: target preceded flankers). As expected, the accuracy for identifying the target letters reduces with decreases in letter separation. This crowding effect shows a strong dependency on SOAs, such that crowding is maximal between 0 and ∼100 ms (depending on conditions) and diminishes for larger SOAs (positive or negative). Maximal crowding does not require the target and flanking letters to physically coexist for the entire presentation duration. Most importantly, crowding can be minimized even for closely spaced letters if there is a large temporal asynchrony between the target and flankers. The reliance of letter identification performance on SOAs and how it changes with letter separations imply that the crowding effect can be traded between space and time. Our findings are consistent with the notion that crowding should be considered as a spatio-temporal, and not simply a spatial, effect.

The Neural Bases of the Semantic Interference of Spatial Frequency-based Information in Scenes

Current models of visual perception suggest that during scene categorization, low spatial frequencies (LSF) are processed rapidly and activate plausible interpretations of visual input. This coarse analysis would then be used to guide subsequent processing of high spatial frequencies (HSF). The present fMRI study examined how processing of LSF may influence that of HSF by investigating the neural bases of the semantic interference effect. We used hybrid scenes as stimuli by combining LSF and HSF from two different scenes, and participants had to categorize the HSF scene. Categorization was impaired when LSF and HSF scenes were semantically dissimilar, suggesting that the LSF scene was processed automatically and interfered with categorization of the HSF scene. fMRI results revealed that this semantic interference effect was associated with increased activation in the inferior frontal gyrus, the superior parietal lobules, and the fusiform and parahippocampal gyri. Furthermore, a connectivity analysis (psychophysiological interaction) revealed that the semantic interference effect resulted in increasing connectivity between the right fusiform and the right inferior frontal gyri. Results support influential models suggesting that, during scene categorization, LSF information is processed rapidly in the pFC and activates plausible interpretations of the scene category. These coarse predictions would then initiate top–down influences on recognition-related areas of the inferotemporal cortex, and these could interfere with the categorization of HSF information in case of semantic dissimilarity to LSF.

Primary visual cortex represents the difference between past and present

The visual system is confronted with rapidly changing stimuli in everyday life. It is not well understood how information in such a stream of input is updated within the brain. We performed voltage-sensitive dye imaging across the primary visual cortex (V1) to capture responses to sequences of natural scene contours. We presented vertically and horizontally filtered natural images, and their superpositions, at 10 or 33 Hz. At low frequency, the encoding was found to represent not the currently presented images, but differences in orientation between consecutive images. This was in sharp contrast to more rapid sequences for which we found an ongoing representation of current input, consistent with earlier studies. Our finding that for slower image sequences, V1 does no longer report actual features but represents their relative difference in time counteracts the view that the first cortical processing stage must always transfer complete information. Instead, we show its capacities for change detection with a new emphasis on the role of automatic computation evolving in the 100-ms range, inevitably affecting information transmission further downstream.

Spatial frequency processing in scene-selective cortical regions

Visual analysis begins with the parallel extraction of different attributes at different spatial frequencies. Low spatial frequencies (LSF) convey coarse information and are characterized by high luminance contrast, while high spatial frequencies (HSF) convey fine details and are characterized by low luminance contrast. In the present fMRI study, we examined how scene-selective regions-the parahippocampal place area (PPA), the retrosplenial cortex (RSC) and the occipital place area (OPA)-responded to spatial frequencies when contrast was either equalized or not equalized across spatial frequencies. Participants performed a categorization task on LSF, HSF and non-filtered scenes belonging to two different categories (indoors and outdoors). We either left contrast across scenes untouched, or equalized it using a root-mean-square contrast normalization. We found that when contrast remained unmodified, LSF and NF scenes elicited greater activation than HSF scenes in the PPA. However, when contrast was equalized across spatial frequencies, the PPA was selective to HFS. This suggests that PPA activity relies on an interaction between spatial frequency and contrast in scenes. In the RSC, LSF and NF elicited greater response than HSF scenes when contrast was not modified, while no effect of spatial frequencies appeared when contrast was equalized across filtered scenes, suggesting that the RSC is sensitive to high-contrast information. Finally, we observed selective activation of the OPA in response to HSF, irrespective of contrast manipulation. These results provide new insights into how scene-selective areas operate during scene processing.

Behavioral assessment of emotional and motivational appraisal during visual processing of emotional scenes depending on spatial frequencies

Previous studies performed on visual processing of emotional stimuli have revealed preference for a specific type of visual spatial frequencies (high spatial frequency, HSF; low spatial frequency, LSF) according to task demands. The majority of studies used a face and focused on the appraisal of the emotional state of others. The present behavioral study investigates the relative role of spatial frequencies on processing emotional natural scenes during two explicit cognitive appraisal tasks, one emotional, based on the self-emotional experience and one motivational, based on the tendency to action. Our results suggest that HSF information was the most relevant to rapidly identify the self-emotional experience (unpleasant, pleasant, and neutral) while LSF was required to rapidly identify the tendency to action (avoidance, approach, and no action). The tendency to action based on LSF analysis showed a priority for unpleasant stimuli whereas the identification of emotional experience based on HSF analysis showed a priority for pleasant stimuli. The present study confirms the interest of considering both emotional and motivational characteristics of visual stimuli.

Orientation enhancement in early visual processing can explain time course of brightness contrast and White's illusion

Dynamics of orientation tuning in V1 indicates that computational model of V1 should not only comprise of bank of static spatially oriented filters but also include the contribution for dynamical response facilitation or suppression along orientation. Time evolution of orientation response in V1 can emerge due to time- dependent excitation and lateral inhibition in the orientation domain. Lateral inhibition in the orientation domain suggests that Ernst Mach's proposition can be applied for the enhancement of initial orientation distribution that is generated due to interaction of visual stimulus with spatially oriented filters and subcortical temporal filter. Oriented spatial filtering that appears much early (<70 ms) in the sequence of visual information processing can account for many of the brightness illusions observed at steady state. It is therefore expected that time evolution of orientation response might be reflecting in the brightness percept over time. Our numerical study suggests that only spatio-temporal filtering at early phase can explain experimentally observed temporal dynamics of brightness contrast illusion. But, enhancement of orientation response at early phase of visual processing is the key mechanism that can guide visual system to predict the brightness by "Max-rule" or "Winner Takes All" (WTA) estimation and thus producing White's illusions at any exposure.

The color "fruit": object memories defined by color

Most fruits and other highly color-diagnostic objects have color as a central aspect of their identity, which can facilitate detection and visual recognition. It has been theorized that there may be a large amount of overlap between the neural representations of these objects and processing involved in color perception. In accordance with this theory we sought to determine if the recognition of highly color diagnostic fruit objects could be facilitated by the visual presentation of their known color associates. In two experiments we show that color associate priming is possible, but contingent upon multiple factors. Color priming was found to be maximally effective for the most highly color diagnostic fruits, when low spatial-frequency information was present in the image, and when determination of the object's specific identity, not merely its category, was required. These data illustrate the importance of color for determining the identity of certain objects, and support the theory that object knowledge involves sensory specific systems.

The neural signature of spatial frequency-based information integration in scene perception

Spatial frequency-based information plays an important role in visual perception. By combining behavioral and electroencephalogram (EEG) measurements, we investigated the mechanisms of the interaction and information integration between different spatial frequency bands. The observers performed a scene categorization task on hybrid images that were generated by combining the low spatial frequency (LSF) component of one image with the high spatial frequency (HSF) component of another image. The results showed that the recognition of the HSF component was interfered by the non-attended LSF component at semantic level. The strength of the semantic interference was modulated by the physical similarity between the LSF and HSF components. Analyses of the EEG data revealed an early anterior N1 component (122 ms from stimulus onset) that was related to the observed interaction of the semantic and physical information between the LSF and HSF components. These findings demonstrate that the semantic information from different spatial frequency bands can be integrated at early stage of the perceptual processing. This early integration is likely to occur at frontal areas in order to initiate top-down facilitation.

Is coarse-to-fine strategy sensitive to normal aging?

Theories on visual perception agree that visual recognition begins with global analysis and ends with detailed analysis. Different results from neurophysiological, computational, and behavioral studies all indicate that the totality of visual information is not immediately conveyed, but that information analysis follows a predominantly coarse-to-fine processing sequence (low spatial frequencies are extracted first, followed by high spatial frequencies). We tested whether such processing continues to occur in normally aging subjects. Young and aged participants performed a categorization task (indoor vs. outdoor scenes), using dynamic natural scene stimuli, in which they resorted to either a coarse-to-fine (CtF) sequence or a reverse fine-to-coarse sequence (FtC). The results show that young participants categorized CtF sequences more quickly than FtC sequences. However, sequence processing interacts with semantic category only for aged participants. The present data support the notion that CtF categorization is effective even in aged participants, but is constrained by the spatial features of the scenes, thus highlighting new perspectives in visual models.

The L/M-opponent channel provides a distinct and time-dependent contribution towards visual recognition

The visual pathway has been successfully modelled as containing separate channels consisting of one achromatically opponent mechanism and two chromatically opponent mechanisms. However, little is known about how time affects the processing of chromatic information. Here, parametrically defined objects were generated. Reduced-colour objects were interleaved with full-colour objects and measures of recognition performance (d') were compared by the continuous serial recognition paradigm. Measures were taken at multiple delay intervals (1, 4, 7, and 10 s). When chromatic variations were removed, recognition performance was impaired, but at the 1 s and 10 s intervals only. When luminance variations were removed, no impairment resulted. When only L/M-opponent modulations were removed, a deficit in performance was produced only at the 1 s and 10 s intervals, similar to the removal of chromatic variation. When only S-opponent modulations were removed, no impairment was observed. The results suggest that the L/M-opponent pathway provides a specialised contribution to visual recognition, but that its effect is modulated by time. A three-stage process model is proposed to explain the data.

The neural substrates and timing of top-down processes during coarse-to-fine categorization of visual scenes: a combined fMRI and ERP study

Spatial frequencies in an image influence visual analysis across a distributed, hierarchically organized brain network. Low spatial frequency (LSF) information may rapidly reach high-order areas to allow an initial coarse parsing of the visual scene, which could then be "retroinjected" through feedback into lower level visual areas to guide finer analysis on the basis of high spatial frequency (HSF). To test this "coarse-to-fine" processing scheme and to identify its neural substrates in the human brain, we presented sequences of two spatial-frequency-filtered scenes in rapid succession (LSF followed by HSF or vice versa) during fMRI and ERPs in the same participants. We show that for low-to-high sequences (but not for high-to-low sequences), LSF produces a first increase of activity in prefrontal and temporo-parietal areas, followed by enhanced responses to HSF in primary visual cortex. This pattern is consistent with retroactive influences on low-level areas that process HSF after initial activation of higher order areas by LSF.

From coarse to fine? Spatial and temporal dynamics of cortical face processing

Primary vision segregates information along 2 main dimensions: orientation and spatial frequency (SF). An important question is how this primary visual information is integrated to support high-level representations. It is generally assumed that the information carried by different SF is combined following a coarse-to-fine sequence. We directly addressed this assumption by investigating how the network of face-preferring cortical regions processes distinct SF over time. Face stimuli were flashed during 75, 150, or 300 ms and masked. They were filtered to preserve low SF (LSF), middle SF (MSF), or high SF (HSF). Most face-preferring regions robustly responded to coarse LSF, face information in early stages of visual processing (i.e., until 75 ms of exposure duration). LSF processing decayed as a function of exposure duration (mostly until 150 ms). In contrast, the processing of fine HSF, face information became more robust over time in the bilateral fusiform face regions and in the right occipital face area. The present evidence suggests the coarse-to-fine strategy as a plausible modus operandi in high-level visual cortex.

How spatial frequencies and visual awareness interact during face processing

In vision, high and low spatial frequencies have been dissociated at the cognitive and neural levels. Usually, high spatial frequency (HSF) is associated with slow analysis along the ventral cortical stream, and low spatial frequency (LSF) is associated with fast and automatic processing. These findings suggest a specific relation between spatial-frequency processing and visual awareness. We investigated this issue using masked-face priming with hybrid prime images of variable visibility. We found subliminal priming for both LSF and HSF information, along with a strong interaction between spatial frequency and visibility: HSF-related priming increased with stimulus visibility, whereas LSF influences remained unchanged. We argue that the results limit the validity of the coarse-to-fine model of vision and of models equating ventral-stream activity with perceptual awareness. Interpreting our results in light of the diagnostic approach suggests a close relation between awareness and diagnosticity.

Holistic processing of faces happens at a glance

Holistic processing (HP) of faces can be inferred from failure to selectively attend to part of a face. We explored how encoding time affects HP of faces by manipulating exposure duration of the study or test face in a sequential matching composite task. HP was observed for exposure as rapid as 50 ms, and was unaffected by whether exposure of the study or test face was limited. Holistic effects emerge as soon as performance is above chance, and are not larger at rapid exposure durations. Limiting exposure at study vs. test did have differential effects on response biases at the fastest exposure durations. These findings provide key constraints for understanding mechanisms of face recognition. These results are first to demonstrate that HP of faces emerges for very briefly presented faces, and that limited perceptual encoding time affects response biases and overall level of performance but not whether faces are processed holistically.

The time-course of visual categorizations: you spot the animal faster than the bird

Background

Since the pioneering study by Rosch and colleagues in the 70s, it is commonly agreed that basic level perceptual categories (dog, chair…) are accessed faster than superordinate ones (animal, furniture…). Nevertheless, the speed at which objects presented in natural images can be processed in a rapid go/no-go visual superordinate categorization task has challenged this “basic level advantage”.

Principal Findings

Using the same task, we compared human processing speed when categorizing natural scenes as containing either an animal (superordinate level), or a specific animal (bird or dog, basic level). Human subjects require an additional 40–65 ms to decide whether an animal is a bird or a dog and most errors are induced by non-target animals. Indeed, processing time is tightly linked with the type of non-targets objects. Without any exemplar of the same superordinate category to ignore, the basic level category is accessed as fast as the superordinate category, whereas the presence of animal non-targets induces both an increase in reaction time and a decrease in accuracy.

Conclusions and Significance

These results support the parallel distributed processing theory (PDP) and might reconciliate controversial studies recently published. The visual system can quickly access a coarse/abstract visual representation that allows fast decision for superordinate categorization of objects but additional time-consuming visual analysis would be necessary for a decision at the basic level based on more detailed representations.

Backward and common-onset masking of vibrotactile stimuli

To inform the design of haptic information displays for noisy environments, we investigated two mechanisms for temporal masking of vibrotactile stimuli (backwards and common-onset) using a commodity display. We used a two-channel setup, presenting stimuli to the middle and ring finger of a participant's right hand. The stimuli consisted of 250 Hz sinusoidal waveforms displayed at a fixed amplitude in various combinations of duration (0, 30 or 300 ms) and stimulus onset asynchrony (0 or 30 ms). In anticipation of future embedded applications where signals are deliberately masked but levels cannot be individualized, signals were standardized at conservative (harder to mask) levels. Our results confirm the existence of a statistically significant masking effect for both forms of haptic masking explored, with common-onset exhibiting a significantly larger masking effect than backwards. However, an analysis of confidence in response levels shows no difference between the two successful masking techniques. We discuss mechanisms that could be responsible for these results, which have implications for the design of user interfaces that rely on tactile transmission of information.

Fixational eye movements, natural image statistics, and fine spatial vision

Perception and motor control are often regarded as two separate branches of neuroscience. Like most species, however, humans are not passively exposed to the incoming flow of sensory data, but actively seek useful information. By shaping input signals in ways that simplify perceptual tasks, behavior might play an important role in establishing efficient sensory representations in the brain. Under natural viewing conditions, the main source of motion of the stimulus on the retina is not the scene but our own behavior. The retinal image is never still, even during visual fixation, when small eye movements combine with movements of the head and body to continually perturb the location of gaze. This article examines the impact of the fixational motion of the retinal image on the statistics of visual input and the neural encoding of visual information. Building upon recent theoretical and experimental results, it is argued that an unstable fixation constitutes an efficient strategy for acquiring information from natural scenes. According to this theory, the fluctuations of luminance caused by the incessant motion of the eye equalize the power present at different spatial frequencies in the spatiotemporal stimulus on the retina. This phenomenon yields compact neural representations, emphasizes fine spatial detail, and might enable a temporal multiplexing of visual information from the retina to the cortex. This theory posits motor contributions to early visual representations and suggests that perception and behavior are more intimately tied than commonly thought.

A model of the dynamics of retinal activity during natural visual fixation

During visual fixation, small eye movements keep the retinal image continuously in motion. It is known that neurons in the visual system are sensitive to the spatiotemporal modulations of luminance resulting from this motion. In this study, we examined the influence of fixational eye movements on the statistics of neural activity in the macaque's retina during the brief intersaccadic periods of natural visual fixation. The responses of parvocellular (P) and magnocellular (M) ganglion cells in different regions of the visual field were modeled while their receptive fields scanned natural images following recorded traces of eye movements. Immediately after the onset of fixation, wide ensembles of coactive ganglion cells extended over several degrees of visual angle, both in the central and peripheral regions of the visual field. Following this initial pattern of activity, the covariance between the responses of pairs of P and M cells and the correlation between the responses of pairs of M cells dropped drastically during the course of fixation. Cell responses were completely uncorrelated by the end of a typical 300-ms fixation. This dynamic spatial decorrelation of retinal activity is a robust phenomenon independent of the specifics of the model. We show that it originates from the interaction of three factors: the statistics of natural scenes, the small amplitudes of fixational eye movements, and the temporal sensitivities of ganglion cells. These results support the hypothesis that fixational eye movements, by shaping the statistics of retinal activity, are an integral component of early visual representations.

Dynamics of Receptive Field Size in Primary Visual Cortex

Recent studies have shown that the initial responses evoked by a stimulus in neurons of primary visual cortex are dominated by low spatial frequency information in the image, whereas finer spatial scales dominate later in the response. Such phenomena could arise from the dynamics of receptive field (RF) size at early stages of cortical processing. We measured changes in RF size in simple cells recorded from the primary visual cortex of anesthetized macaques by measuring their first-order spatio-temporal kernels and fitting them with two-dimensional Gabor functions at different time slices. We found that the width and length of the RF envelope and the period of the carrier tend to decrease during the time-course of the response. The most pronounced changes are seen in the width and spatial period of the RFs, which decrease by 15% during the central 20 ms of the response. These results show a novel form of spatio-temporal inseparability in simple cells and are consistent with the notion of a coarse-to-fine processing of information in early visual cortex.

Building the gist of a scene: the role of global image features in recognition

Humans can recognize the gist of a novel image in a single glance, independent of its complexity. How is this remarkable feat accomplished? On the basis of behavioral and computational evidence, this paper describes a formal approach to the representation and the mechanism of scene gist understanding, based on scene-centered, rather than object-centered primitives. We show that the structure of a scene image can be estimated by the mean of global image features, providing a statistical summary of the spatial layout properties (Spatial Envelope representation) of the scene. Global features are based on configurations of spatial scales and are estimated without invoking segmentation or grouping operations. The scene-centered approach is not an alternative to local image analysis but would serve as a feed-forward and parallel pathway of visual processing, able to quickly constrain local feature analysis and enhance object recognition in cluttered natural scenes.

Faces are "spatial"--holistic face perception is supported by low spatial frequencies

Faces are perceived holistically, a phenomenon best illustrated when the processing of a face feature is affected by the other features. Here, the authors tested the hypothesis that the holistic perception of a face mainly relies on its low spatial frequencies. Holistic face perception was tested in two classical paradigms: the whole-part advantage (Experiment 1) and the composite face effect (Experiments 2-4). Holistic effects were equally large or larger for low-pass filtered faces as compared to full-spectrum faces and significantly larger than for high-pass filtered faces. The disproportionate composite effect found for low-pass filtered faces was not observed when holistic perception was disrupted by inversion (Experiment 3). Experiment 4 showed that the composite face effect was enhanced only for low spatial frequencies, but not for intermediate spatial frequencies known be critical for face recognition. These findings indicate that holistic face perception is largely supported by low spatial frequencies. They also suggest that holistic processing precedes the analysis of local features during face perception.

Face perception: An integrative review of the role of spatial frequencies

The aim of this article is to reinterpret the results obtained from the research analyzing the role played by spatial frequencies in face perception. Two main working lines have been explored in this body of research: the critical bandwidth of spatial frequencies that allows face recognition to take place (the masking approach), and the role played by different spatial frequencies while the visual percept is being developed (the microgenetic approach). However, results obtained to date are not satisfactory in that no single explanation accounts for all the data obtained from each of the approaches. We propose that the main factor for understanding the role of spatial frequencies in face perception depends on the interaction between the demands of the task and the information in the image (the diagnostic recognition approach). Using this new framework, we review the most significant research carried out since the early 1970s to provide a reinterpretation of the data obtained.

The coarse-to-fine hypothesis revisited: evidence from neuro-computational modeling

The human perceptual system seems to be driven by a coarse-to-fine integration of visual information. Different results have shown a faster integration of low-spatial frequency compared with high-spatial frequency (HSF) information, starting at early retinal processes. The difference in spatial scale decomposition remains throughout the lateral geniculate nucleus (Hubel & Wiesel, 1977) and V1 (Tootell, Silverman, & De Valois, 1981). During the last decade, a debate has emerged concerning the origin of the coarse-to-fine integration. Is it a constant, perceptually driven integration (Parker et al., 1992 and Parker et al., 1996)? Instead, the flexible use hypothesis suggests that different spatial frequency channels could be enhanced depending on the requirement of the task for high-level cognitive processes like categorization (Oliva and Schyns, 1997 and Schyns and Oliva, 1999). In two connectionist simulations, we have shown that global categorization performance could actually be better performed with HSF information when the amount of information is normalized across the different spatial frequency channels. Those results suggest that high-level requirement alone could not explain the coarse-to-fine bias toward LSF information. A hypothesis is proposed concerning the possible implication of the amount of data provided by different spatial frequency channel that might provide the perceptual bias toward LSF information.

Hemispheric specialization of human inferior temporal cortex during coarse-to-fine and fine-to-coarse analysis of natural visual scenes

Recent models of visual recognition have suggested that perceptual analysis may start with a parallel extraction of different spatial frequencies (SF), using a preferential coarse-to-fine (low-to-high SF) sequence of processing. A rapid extraction of low spatial frequency (LSF) information may thus provide an initial and crude parsing of the visual scene, subsequently refined by slow but more detailed high spatial frequency (HSF) information. However, the sequence of SF analysis could be flexible, a high-to-low (HtL) being sometimes preferred to a low-to-high (LtH) SF sequence depending on task demands. Furthermore, it has also been suggested that the right vs. left hemisphere might be differentially specialized in LSF vs. HSF analysis, respectively. By manipulating the temporal succession of LSF and HSF stimuli, the present fMRI study investigated whether such hemispheric specialization may underlie the flexible use of different time-course in SF analysis. Participants performed a matching task between two successive images of natural scenes (LSF or HSF) that were displayed either in an LtH (LSF scene presented first and HSF scene second) or in a reverse HtL sequence. A direct inter-hemispheric comparison of the neural responses evoked by each SF sequence revealed greater activations within the right occipito-temporal cortex for the LtH sequence and within the left occipito-temporal cortex for the HtL sequence. These fMRI results suggest that the hemisphere preferentially engaged during the sequential processing of different SF might be determined by the initial SF-band appearing in this sequence, and that both a coarse-to-fine and fine-to-coarse analysis might independently take place in the two hemispheres.

Early computational processing in binocular vision and depth perception

Stereoscopic depth perception is a fascinating ability in its own right and also a useful model of perception. In recent years, considerable progress has been made in understanding the early cortical circuitry underlying this ability. Inputs from left and right eyes are first combined in primary visual cortex (V1), where many cells are tuned for binocular disparity. Although the observation of disparity tuning in V1, combined with psychophysical evidence that stereopsis must occur early in visual processing, led to initial suggestions that V1 was the neural correlate of stereoscopic depth perception, more recent work indicates that this must occur in higher visual areas. The firing of cells in V1 appears to depend relatively simply on the visual stimuli within local receptive fields in each retina, whereas the perception of depth reflects global properties of the stimulus. However, V1 neurons appear to be specialized in a number of respects to encode ecologically relevant binocular disparities. This suggests that they carry out essential pre-processing underlying stereoscopic depth perception in higher areas. This article reviews recent progress in developing accurate models of the computations carried out by these neurons. We seem close to achieving a mathematical description of the initial stages of the brain's stereo algorithm. This is important in itself--for instance, it may enable improved stereopsis in computer vision--and paves the way for a full understanding of how depth perception arises.

Forward masking of faces by spatially quantized random and structured masks: On the roles of wholistic configuration, local features, and spatial-frequency spectra in perceptual identification

The forward masking of faces by spatially quantized masking images was studied. Masks were used in order to exert different types of degrading effects on the early representations in facial information processing. Three types of source images for masks were used: Same-face images (with regard to targets), different-face images, and random Gaussian noise that was spectrally similar to facial images. They were all spatially quantized over the same range of quantization values. Same-face masks had virtually no masking effect at any of the quantization values. Different-face masks had strong masking effects only with fine-scale quantization, but led to the same efficiency of recognition as in the same-face mask condition with the coarsest quantization. Moreover, compared with the noise-mask condition, coarsely quantized different-face masks led to a relatively facilitated level of recognition efficiency. The masking effect of the noise mask did not vary significantly with the coarseness of quantization. The results supported neither a local feature processing account, nor a generalized spatial-frequency processing account, but were consistent with the microgenetic configuration-processing theory of face recognition. Also, the suitability of a spatial quantization technique for image configuration processing research has been demonstrated.

Color sensitivity of cells responsive to complex stimuli in the temporal cortex

The inferotemporal (IT) cortex of the monkey lies at the head of the ventral visual pathway and is known to mediate object recognition and discrimination. It is often assumed that color plays a minor role in the recognition of objects and faces because discrimination remains highly accurate with black-and-white images. Furthermore it has been suggested that for rapid presentation and reaction tasks, object classification may be based on a first wave of feedforward visual information, which is coarse and achromatic. The fine detail and color information follows later, allowing similar stimuli to be discriminated. To allow these theories to be tested, this study investigates whether the presence of color affects the response of IT neurons to complex stimuli, such as faces, and whether color information is delayed with respect to information about stimulus form in these cells. Color, achromatic, and false-color versions of effective stimuli were presented using a rapid serial visual presentation paradigm, and responses recorded from single cells in IT of the adult monkey. Achromatic images were found to evoke significantly reduced responses compared with color images in the majority of neurons (70%) tested. Differential activity for achromatic and colored stimuli was evident from response onset with no evidence to support the hypothesis that information about object color is delayed with respect to object form. A negative correlation (P < 0.01) was found between cell latency and color sensitivity, with the most color-sensitive cells tending to respond earliest. The results of this study suggest a strong role for color in familiar object recognition and provide no evidence to support the idea of a first wave of form processing in the ventral stream based on purely achromatic information.

On the delay in processing high spatial frequency visual information: reaction time and VEP latency study of the effect of local intensity of stimulation

Saleh and Bonnet [Fechner Day 98, p. 344] have shown that, upon parafoveal stimulation and up to 6.5 c/deg, reaction time (RT) is a function of grating contrast multiplied by grating period. The present experiments extend these findings to foveal stimulation within a wider spatial-frequency (SF) range and to stimuli of different duration. Both RT and latency of visually evoked potentials (VEP) were measured. The findings might be explained by the following assumption: Most RT and VEP latency variations across the SF range are a result of local intensity factors (retinal contrast and width of grating bars). Residual RT variations were found that might be due to processing of high SFs by slower mechanisms than those processing low and medium SFs.

Usage of spatial scales for the categorization of faces, objects, and scenes

The role of spatial scales (or spatial frequencies) in the processing of faces, objects, and scenes has recently seen a surge of research activity. In this review, we will critically examine two main theories of scale usage. The fixed theory proposes that spatial scales are used in a fixed, perceptually determined order (coarse to fine). The flexible theory suggests instead that usage of spatial scales is flexible, depending on the requirements of visual information for the categorization task at hand. The implications of the theories are examined for face, object, and scene categorization, attention, perception, and representation.

Nonhomogeneous resolution of images of natural scenes

The aim of this research is to model and simulate the loss of visual resolution as a function of retinal eccentricity in the perception of natural scenes. The model of visual resolution is based on a space-variant low-pass filter, having a variable convolution kernel according to retinal eccentricity. The parameters of the model are computed from psychophysical measures of visual acuity as a function of retinal eccentricity. The implementation of the model allowed us to generate images of scenes with nonhomogeneous space-variant resolution, simulating the filtering executed by the eye. These scenes are used to test and optimise the model by means of experiments in static vision (through tachistoscopic presentations) and in dynamic vision where the resolution of the scene is computed, in real-time, as a function of the location of gaze.