Time course of the integration of spatial frequency-based information in natural scenes

The time-course of real-world scene perception: Spatial and semantic processing

Real-world scene perception unfolds remarkably quickly, yet the underlying visual processes are poorly understood. Space-centered theory maintains that a scene's spatial structure (e.g., openness, mean depth) can be rapidly recovered from low-level image statistics. In turn, the statistical relationship between a scene's spatial properties and semantic content allows for semantic identity to be inferred from its layout. We tested this theory by investigating (1) the temporal dynamics of spatial and semantic perception in real-world scenes, and (2) dependencies between spatial and semantic judgments. Participants viewed backward-masked images for 13.3 to 106.7 ms, and identified the semantic (e.g., beach, road) or spatial structure (e.g., open, closed-off) category. We found no temporal precedence of spatial discrimination relative to semantic discrimination. Computational analyses further suggest that, instead of using spatial layout to infer semantic categories, humans exploit semantic information to discriminate spatial structure categories. These findings challenge traditional 'bottom-up' views of scene perception.

Grounding Context in Embodied Cognitive Robotics

Biological agents are context-dependent systems that exhibit behavioral flexibility. The internal and external information agents process, their actions, and emotions are all grounded in the context within which they are situated. However, in the field of cognitive robotics, the concept of context is far from being clear with most studies making little to no reference to it. The aim of this paper is to provide an interpretation of the notion of context and its core elements based on different studies in natural agents, and how these core contextual elements have been modeled in cognitive robotics, to introduce a new hypothesis about the interactions between these contextual elements. Here, global context is categorized as agent-related, environmental, and task-related context. The interaction of their core elements, allows agents to first select self-relevant tasks depending on their current needs, or for learning and mastering their environment through exploration. Second, to perform a task and continuously monitor its performance. Third, to abandon a task in case its execution is not going as expected. Here, the monitoring of prediction error, the difference between sensorimotor predictions and incoming sensory information, is at the core of behavioral flexibility during situated action cycles. Additionally, monitoring prediction error dynamics and its comparison with the expected reduction rate should indicate the agent its overall performance on executing the task. Sensitivity to performance evokes emotions that function as the driving element for autonomous behavior which, at the same time, depends on the processing of the interacting core elements. Taking all these into account, an interactionist model of contexts and their core elements is proposed. The model is embodied, affective, and situated, by means of the processing of the agent-related and environmental core contextual elements. Additionally, it is grounded in the processing of the task-related context and the associated situated action cycles during task execution. Finally, the model proposed here aims to guide how artificial agents should process the core contextual elements of the agent-related and environmental context to give rise to the task-related context, allowing agents to autonomously select a task, its planning, execution, and monitoring for behavioral flexibility.

Influence of field of view in visual prostheses design: Analysis with a VR system

OBJECTIVE

Visual prostheses are designed to restore partial functional vision in patients with total vision loss. Retinal visual prostheses provide limited capabilities as a result of low resolution, limited field of view and poor dynamic range. Understanding the influence of these parameters in the perception results can guide prostheses research and design.

APPROACH

In this work, we evaluate the influence of field of view with respect to spatial resolution in visual prostheses, measuring the accuracy and response time in a search and recognition task. Twenty-four normally sighted participants were asked to find and recognize usual objects, such as furniture and home appliance in indoor room scenes. For the experiment, we use a new simulated prosthetic vision system that allows simple and effective experimentation. Our system uses a virtual-reality environment based on panoramic scenes. The simulator employs a head-mounted display which allows users to feel immersed in the scene by perceiving the entire scene all around. Our experiments use public image datasets and a commercial head-mounted display. We have also released the virtual-reality software for replicating and extending the experimentation.

MAIN RESULTS

Results show that the accuracy and response time decrease when the field of view is increased. Furthermore, performance appears to be correlated with the angular resolution, but showing a diminishing return even with a resolution of less than 2.3 phosphenes per degree.

SIGNIFICANCE

Our results seem to indicate that, for the design of retinal prostheses, it is better to concentrate the phosphenes in a small area, to maximize the angular resolution, even if that implies sacrificing field of view.

The Role of Low-Spatial Frequency Components in the Processing of Deceptive Faces: A Study Using Artificial Face Models

Interpreting another's true emotion is important for social communication, even in the face of deceptive facial cues. Because spatial frequency components provide important clues for recognizing facial expressions, we investigated how we use spatial frequency information from deceptive faces to interpret true emotion. We conducted two different tasks: a face-generating experiment in which participants were asked to generate deceptive and genuine faces by tuning the intensity of happy and angry expressions (Experiment 1) and a face-classification task in which participants had to classify presented faces as either deceptive or genuine (Experiment 2). Low- and high-spatial frequency (LSF and HSF) components were varied independently. The results showed that deceptive happiness (i.e., anger is the hidden expression) involved different intensities for LSF and HSF. These results suggest that we can identify hidden anger by perceiving unbalanced intensities of emotional expression between LSF and HSF information contained in deceptive faces.

Modulation of microsaccades by spatial frequency during object categorization

The organization of visual processing into a coarse-to-fine information processing based on the spatial frequency properties of the input forms an important facet of the object recognition process. During visual object categorization tasks, microsaccades occur frequently. One potential functional role of these eye movements is to resolve high spatial frequency information. To assess this hypothesis, we examined the rate, amplitude and speed of microsaccades in an object categorization task in which participants viewed object and non-object images and classified them as showing either natural objects, man-made objects or non-objects. Images were presented unfiltered (broadband; BB) or filtered to contain only low (LSF) or high spatial frequency (HSF) information. This allowed us to examine whether microsaccades were modulated independently by the presence of a high-level feature - the presence of an object - and by low-level stimulus characteristics - spatial frequency. We found a bimodal distribution of saccades based on their amplitude, with a split between smaller and larger microsaccades at 0.4° of visual angle. The rate of larger saccades (⩾0.4°) was higher for objects than non-objects, and higher for objects with high spatial frequency content (HSF and BB objects) than for LSF objects. No effects were observed for smaller microsaccades (<0.4°). This is consistent with a role for larger microsaccades in resolving HSF information for object identification, and previous evidence that more microsaccades are directed towards informative image regions.

Studies on cognitively driven attention suggest that late vision is cognitively penetrated, whereas early vision is not

Firestone & Scholl (F&S) examine, among other possible cognitive influences on perception, the effects of peripheral attention and conclude that these effects do not entail cognition directly affecting perception. Studies in neuroscience with other forms of attention, however, suggest that a stage of vision, namely late vision, is cognitively penetrated mainly through the effects of cognitively driven spatial and object-centered attention.

The neural bases of spatial frequency processing during scene perception

Theories on visual perception agree that scenes are processed in terms of spatial frequencies. Low spatial frequencies (LSF) carry coarse information whereas high spatial frequencies (HSF) carry fine details of the scene. However, how and where spatial frequencies are processed within the brain remain unresolved questions. The present review addresses these issues and aims to identify the cerebral regions differentially involved in low and high spatial frequency processing, and to clarify their attributes during scene perception. Results from a number of behavioral and neuroimaging studies suggest that spatial frequency processing is lateralized in both hemispheres, with the right and left hemispheres predominantly involved in the categorization of LSF and HSF scenes, respectively. There is also evidence that spatial frequency processing is retinotopically mapped in the visual cortex. HSF scenes (as opposed to LSF) activate occipital areas in relation to foveal representations, while categorization of LSF scenes (as opposed to HSF) activates occipital areas in relation to more peripheral representations. Concomitantly, a number of studies have demonstrated that LSF information may reach high-order areas rapidly, allowing an initial coarse parsing of the visual scene, which could then be sent back through feedback into the occipito-temporal cortex to guide finer HSF-based analysis. Finally, the review addresses spatial frequency processing within scene-selective regions areas of the occipito-temporal cortex.

The neural signature of spatial frequency-based information integration in scene perception

Spatial frequency-based information plays an important role in visual perception. By combining behavioral and electroencephalogram (EEG) measurements, we investigated the mechanisms of the interaction and information integration between different spatial frequency bands. The observers performed a scene categorization task on hybrid images that were generated by combining the low spatial frequency (LSF) component of one image with the high spatial frequency (HSF) component of another image. The results showed that the recognition of the HSF component was interfered by the non-attended LSF component at semantic level. The strength of the semantic interference was modulated by the physical similarity between the LSF and HSF components. Analyses of the EEG data revealed an early anterior N1 component (122 ms from stimulus onset) that was related to the observed interaction of the semantic and physical information between the LSF and HSF components. These findings demonstrate that the semantic information from different spatial frequency bands can be integrated at early stage of the perceptual processing. This early integration is likely to occur at frontal areas in order to initiate top-down facilitation.

Attention-free integration of spatial frequency-based information in natural scenes

The integration of visual image information provided by low and high spatial frequency channels is critical to rapid perception of natural scenes. However, little is known about the role of attention in integrating this information. In two experiments, using attention-demanding tasks, we examined the advantage of integration, i.e. the superior categorization accuracies for images, using a wide range of different spatial frequencies. In Experiment 1, a spatially central to-be-identified letter and a peripheral filtered image of a natural scene appeared simultaneously. In Experiment 2, the letter and the image were presented sequentially at the same spatial location. In both experiments results consistently showed an advantage of integration in categorization behavior that was not influenced by attention-demanding tasks. This finding suggests that the integration of frequency-based information in natural scenes is attention-free.

Late vision: processes and epistemic status

In this paper, I examine the processes that occur in late vision and address the problem of whether late vision should be construed as a properly speaking perceptual stage, or as a thought-like discursive stage. Specifically, I argue that late vision, its (partly) conceptual nature notwithstanding, neither is constituted by nor does it implicate what I call pure thoughts, that is, propositional structures that are formed in the cognitive areas of the brain through, and participate in, discursive reasoning and inferences. At the same time, the output of late vision, namely an explicit belief concerning the identity and category membership of an object (that is, a recognitional belief) or its features, eventually enters into discursive reasoning. Using Jackendoff’s distinction between visual awareness, which characterizes perception, and visual understanding, which characterizes pure thought, I claim that the contents of late vision belong to visual awareness and not to visual understanding and that although late vision implicates beliefs, either implicit or explicit, these beliefs are hybrid visual/conceptual constructs and not pure thoughts. Distinguishing between these hybrid representations and pure thoughts and delineating the nature of the representations of late vision lays the ground for examining, among other things, the process of conceptualization that occurs in visual processing and the way concepts modulate perceptual content affecting either its representational or phenomenal character. I also do not discuss the epistemological relations between the representations of late vision and the perceptual judgments they “support” or “guide” or “render possible” or “evidence” or “entitle.” However, the specification of the epistemology of late vision lays the ground for attacking that problem as well.

[The integration of spatial frequency-based information in scene perception: evidence from spatially incongruent images]

Humans can recognize a complex natural scene even when it appears only briefly. The rapid recognition of natural scenes is accomplished by parallel processing of information based on multiple spatial frequencies and integration of this information. Previous studies have revealed the time course of integration of frequency-based information. However, it is still unclear how frequency-based information is integrated. There are two possible levels for the integration: One involves spatial integration of images and constructs a unified image, and the other entails semantic integration associated with the scene context level irrespective of spatial arrangements. We investigated the categorization accuracy of the low + high-pass images, in which a left-right mirror reversed low-pass image was superimposed on a nonreversed high-pass image or vice versa. In this context, the low+high-pass images were semantically integrable but spatially incongruent. The results indicated that accuracy of the low+high-pass images did not exceed the expected accuracy level estimated from separate presentations. This finding suggests that frequency-based information can be integrated spatially.