The coarse-to-fine hypothesis revisited: evidence from neuro-computational modeling

High spatial frequencies disrupt conscious visual recognition: evidence from an attentional blink paradigm

In this article, we tested the respective importance of low spatial frequencies (LSF) and high spatial frequencies (HSF) for conscious visual recognition of emotional stimuli by using an attentional blink paradigm. Thirty-eight participants were asked to identify and report two targets (happy faces) embedded in a rapid serial visual presentation of distractors (angry faces). During attentional blink, conscious perception of the second target (T2) is usually altered when the lag between the two targets is short (200-500 ms) but is restored at longer lags. The distractors between T1 and T2 were either non-filtered (broad spatial frequencies, BSF), low-pass filtered (LSF), or high-pass filtered (HSF). Assuming that prediction abilities could be at the root of conscious visual recognition, we expected that LSF distractors could result in a greater disturbance of T2 reporting than HSF distractors. Results showed that both LSF and HSF play a role in the emergence of exogenous consciousness in the visual system. Furthermore, HSF distractors strongly affected T1 and T2 reporting irrespective of the lag between targets, suggesting their role for facial emotion processing. We discuss these results with regards to other models of visual recognition. .

Flexibility of emerging face categorization at different levels of abstraction

Categorization of visual stimuli at different levels of abstraction relies on the encoding of relevant diagnostic features present at different spatial scales. We used the Eidolon Factory, an image-manipulation algorithm that introduces random disarray fields across spatial scales, to study how such a process flexibly combines perceptual information to perform successful categorization depending on task demands. Images of animal faces, human faces, and everyday objects were disarrayed coherently (random fields correlated) or incoherently (random fields randomized) to create a family of 50 eidolons per stimulus image with increasing disarray. Participants (N = 243) viewed each family of eidolons in a smooth sequence from maximum disarray to no disarray and performed a category verification task either at the superordinate (any face type) or basic (human face only) levels at two levels of uncertainty: participants in one group used their gut feeling to respond, whereas another group had to be sure of their decision. When participants used their gut feeling to respond, we observed a superordinate-level advantage. When they were sure of their response, we observed a basic-level advantage. Coherently disarrayed sequences impaired target detection compared to incoherently disarrayed sequences for both levels of response certainty. Furthermore, participants' sensitivity in the Any Face condition increased when they observed coherently disarrayed sequences and had to be sure of their response. These results suggest that the visual system does not strictly adhere to feedforward processing but flexibly adjusts to the relevant perceptual information depending on task context.

Rapid scene categorization: From coarse peripheral vision to fine central vision

Studies on scene perception have shown that the rapid extraction of low spatial frequencies (LSF) allows a coarse parsing of the scene, prior to the analysis of high spatial frequencies (HSF) containing details. Many studies suggest that scene gist recognition can be achieved with only the low resolution of peripheral vision. Our study investigated the advantage of peripheral vision on central vision during a scene categorization task (indoor vs. outdoor). In Experiment 1, we used large scene photographs from which we built one central disk and four circular rings of different eccentricities. The central disk either contained or not an object semantically related to the scene category. Results showed better categorization performances for the peripheral rings, despite the presence of an object in central vision that was semantically related to the scene category that significantly improved categorization performances. In Experiment 2, the central disk and rings were assembled from Central to Peripheral vision (CtP sequence) or from Peripheral to Central vision (PtC sequence). Results revealed better performances for PtC than CtP sequences, except when no central object was present under rapid categorization constraints. As Experiment 3 suggested that the PtC advantage was not explained by a reduction of the visibility of the object in the central disk by the surrounding peripheral rings (CtP sequence), results are interpreted in the context of a predominant coarse-to-fine processing during scene categorization, with greater efficiency and utility of coarse peripheral vision relative to fine central vision during rapid scene categorization.

Leader-Based Multi-Scale Attention Deep Architecture for Person Re-Identification

Person re-identification (re-id) aims to match people across non-overlapping camera views in a public space. This is a challenging problem because the people captured in surveillance videos often wear similar clothing. Consequently, the differences in their appearance are typically subtle and only detectable at particular locations and scales. In this paper, we propose a deep re-id network (MuDeep) that is composed of two novel types of layers - a multi-scale deep learning layer, and a leader-based attention learning layer. Specifically, the former learns deep discriminative feature representations at different scales, while the latter utilizes the information from multiple scales to lead and determine the optimal weightings for each scale. The importance of different spatial locations for extracting discriminative features is learned explicitly via our leader-based attention learning layer. Extensive experiments are carried out to demonstrate that the proposed MuDeep outperforms the state-of-the-art on a number of benchmarks and has a better generalization ability under a domain generalization setting.

Effects of Experience on Spatial Frequency Tuning in the Visual System: Behavioral, Visuocortical, and Alpha-band Responses

Using electrophysiology and a classic fear conditioning paradigm, this work examined adaptive visuocortical changes in spatial frequency tuning in a sample of 50 undergraduate students. High-density EEG was recorded while participants viewed 400 total trials of individually presented Gabor patches of 10 different spatial frequencies. Patches were flickered to produce sweep steady-state visual evoked potentials (ssVEPs) at a temporal frequency of 13.33 Hz, with stimulus contrast ramping up from 0% to 41% Michelson over the course of each 2800-msec trial. During the final 200 trials, a selected range of Gabor stimuli (either the lowest or highest spatial frequencies, manipulated between participants) were paired with an aversive 90-dB white noise auditory stimulus. Changes in spatial frequency tuning from before to after conditioning for paired and unpaired gratings were evaluated at the behavioral and electrophysiological level. Specifically, ssVEP amplitude changes were evaluated for lateral inhibition and generalization trends, whereas change in alpha band (8-12 Hz) activity was tested for a generalization trend across spatial frequencies, using permutation-controlled F contrasts. Overall time courses of the sweep ssVEP amplitude envelope and alpha-band power were orthogonal, and ssVEPs proved insensitive to spatial frequency conditioning. Alpha reduction (blocking) was most pronounced when viewing fear-conditioned spatial frequencies, with blocking decreasing along the gradient of spatial frequencies preceding conditioned frequencies, indicating generalization across spatial frequencies. Results suggest that alpha power reduction-conceptually linked to engagement of attention and alertness/arousal mechanisms-to fear-conditioned stimuli operates independently of low-level spatial frequency processing (indexed by ssVEPs) in primary visual cortex.

Learning Nonclassical Receptive Field Modulation for Contour Detection

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks.

Spatial frequency processing in scene-selective cortical regions

Visual analysis begins with the parallel extraction of different attributes at different spatial frequencies. Low spatial frequencies (LSF) convey coarse information and are characterized by high luminance contrast, while high spatial frequencies (HSF) convey fine details and are characterized by low luminance contrast. In the present fMRI study, we examined how scene-selective regions-the parahippocampal place area (PPA), the retrosplenial cortex (RSC) and the occipital place area (OPA)-responded to spatial frequencies when contrast was either equalized or not equalized across spatial frequencies. Participants performed a categorization task on LSF, HSF and non-filtered scenes belonging to two different categories (indoors and outdoors). We either left contrast across scenes untouched, or equalized it using a root-mean-square contrast normalization. We found that when contrast remained unmodified, LSF and NF scenes elicited greater activation than HSF scenes in the PPA. However, when contrast was equalized across spatial frequencies, the PPA was selective to HFS. This suggests that PPA activity relies on an interaction between spatial frequency and contrast in scenes. In the RSC, LSF and NF elicited greater response than HSF scenes when contrast was not modified, while no effect of spatial frequencies appeared when contrast was equalized across filtered scenes, suggesting that the RSC is sensitive to high-contrast information. Finally, we observed selective activation of the OPA in response to HSF, irrespective of contrast manipulation. These results provide new insights into how scene-selective areas operate during scene processing.

Affective and contextual values modulate spatial frequency use in object recognition

Visual object recognition is of fundamental importance in our everyday interaction with the environment. Recent models of visual perception emphasize the role of top-down predictions facilitating object recognition via initial guesses that limit the number of object representations that need to be considered. Several results suggest that this rapid and efficient object processing relies on the early extraction and processing of low spatial frequencies (LSF). The present study aimed to investigate the SF content of visual object representations and its modulation by contextual and affective values of the perceived object during a picture-name verification task. Stimuli consisted of pictures of objects equalized in SF content and categorized as having low or high affective and contextual values. To access the SF content of stored visual representations of objects, SFs of each image were then randomly sampled on a trial-by-trial basis. Results reveal that intermediate SFs between 14 and 24 cycles per object (2.3-4 cycles per degree) are correlated with fast and accurate identification for all categories of objects. Moreover, there was a significant interaction between affective and contextual values over the SFs correlating with fast recognition. These results suggest that affective and contextual values of a visual object modulate the SF content of its internal representation, thus highlighting the flexibility of the visual recognition system.

The neural bases of spatial frequency processing during scene perception

Theories on visual perception agree that scenes are processed in terms of spatial frequencies. Low spatial frequencies (LSF) carry coarse information whereas high spatial frequencies (HSF) carry fine details of the scene. However, how and where spatial frequencies are processed within the brain remain unresolved questions. The present review addresses these issues and aims to identify the cerebral regions differentially involved in low and high spatial frequency processing, and to clarify their attributes during scene perception. Results from a number of behavioral and neuroimaging studies suggest that spatial frequency processing is lateralized in both hemispheres, with the right and left hemispheres predominantly involved in the categorization of LSF and HSF scenes, respectively. There is also evidence that spatial frequency processing is retinotopically mapped in the visual cortex. HSF scenes (as opposed to LSF) activate occipital areas in relation to foveal representations, while categorization of LSF scenes (as opposed to HSF) activates occipital areas in relation to more peripheral representations. Concomitantly, a number of studies have demonstrated that LSF information may reach high-order areas rapidly, allowing an initial coarse parsing of the visual scene, which could then be sent back through feedback into the occipito-temporal cortex to guide finer HSF-based analysis. Finally, the review addresses spatial frequency processing within scene-selective regions areas of the occipito-temporal cortex.

Behavioral assessment of emotional and motivational appraisal during visual processing of emotional scenes depending on spatial frequencies

Previous studies performed on visual processing of emotional stimuli have revealed preference for a specific type of visual spatial frequencies (high spatial frequency, HSF; low spatial frequency, LSF) according to task demands. The majority of studies used a face and focused on the appraisal of the emotional state of others. The present behavioral study investigates the relative role of spatial frequencies on processing emotional natural scenes during two explicit cognitive appraisal tasks, one emotional, based on the self-emotional experience and one motivational, based on the tendency to action. Our results suggest that HSF information was the most relevant to rapidly identify the self-emotional experience (unpleasant, pleasant, and neutral) while LSF was required to rapidly identify the tendency to action (avoidance, approach, and no action). The tendency to action based on LSF analysis showed a priority for unpleasant stimuli whereas the identification of emotional experience based on HSF analysis showed a priority for pleasant stimuli. The present study confirms the interest of considering both emotional and motivational characteristics of visual stimuli.

Is coarse-to-fine strategy sensitive to normal aging?

Theories on visual perception agree that visual recognition begins with global analysis and ends with detailed analysis. Different results from neurophysiological, computational, and behavioral studies all indicate that the totality of visual information is not immediately conveyed, but that information analysis follows a predominantly coarse-to-fine processing sequence (low spatial frequencies are extracted first, followed by high spatial frequencies). We tested whether such processing continues to occur in normally aging subjects. Young and aged participants performed a categorization task (indoor vs. outdoor scenes), using dynamic natural scene stimuli, in which they resorted to either a coarse-to-fine (CtF) sequence or a reverse fine-to-coarse sequence (FtC). The results show that young participants categorized CtF sequences more quickly than FtC sequences. However, sequence processing interacts with semantic category only for aged participants. The present data support the notion that CtF categorization is effective even in aged participants, but is constrained by the spatial features of the scenes, thus highlighting new perspectives in visual models.

Perceptual Factors Affecting the Ability to Assess Facial Resemblance between Parents and Newborns in Humans

People undeniably pay attention to faces, and facial resemblance may act as a kinship cue. However, previous studies have shown that the ability to detect kinship through facial resemblance is limited, and it has been suggested that this may be due to several types of perceptual factors. To further understand the processes that underpin kinship judgment, it is important to investigate which perceptual factors predict the probability of parent–child pairs being detected as related. To this end, we performed two experiments. In the first, we evaluated the ability of human observers to match newborns with one of their parents. In the second, we explored three perceptual factors that may have influenced kinship detection (gender discrimination, facial attractiveness, and perceptual similarity). Results showed that the participants were able to match newborns with one of their parents, even though the task was perceived as difficult. Moreover, our study goes further than previous findings, showing that the perceptual factors investigated may significantly contribute to kinship detection.

A model of contour extraction including multiple scales, flexible inhibition and attention

A mathematical model of contextual integration and contour extraction in the primary visual cortex developed in a recent work [Ursino, M., & La Cara, G. E. (2004). A model of contextual interactions and contour detection in primary visual cortex. Neural Networks, 17, 719-735] has been significantly improved to include two fundamental additional aspects, i.e., multi-scale decomposition and attention. The model incorporates two independent paths for visual processing corresponding to two different scales. Attention from higher hierarchical levels works by modifying different properties of the network: by selecting the portion of the image to be scrutinized and the appropriate scale, by modulating the threshold of a gating mechanism, and by modifying the width and/or strength of lateral inhibition. Through computer simulations of real complex and noisy black-and-white images, we demonstrate that appropriate selection of the above factors allows accurate analysis of image contours at different levels, from global perception of the overall objects without details, down to a fine examination of minute particulars (such as the lips in a face or the fingers of a hand). Attentive reconfiguration of lateral inhibition plays a key role in the analysis of images at different detail levels.