View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds

Visual search in scenes involves selective and nonselective pathways

How does one find objects in scenes? For decades, visual search models have been built on experiments in which observers search for targets, presented among distractor items, isolated and randomly arranged on blank backgrounds. Are these models relevant to search in continuous scenes? This article argues that the mechanisms that govern artificial, laboratory search tasks do play a role in visual search in scenes. However, scene-based information is used to guide search in ways that had no place in earlier models. Search in scenes might be best explained by a dual-path model: a 'selective' path in which candidate objects must be individually selected for recognition and a 'nonselective' path in which information can be extracted from global and/or statistical information.

Visual streams and shifting attention

Understanding the relationship between bottom-up and top-down processing in visual perception and attention is challenging. An important part of that challenge is studying the roles the parvocellular (P) and magnocellular (M) retino-geniculo-cortical pathways play in visual processing and attention. The P pathway provides the dominant initial input to the ventral stream which plays an important role in object processing and is assumed to be relatively more involved in object-based attention. The faster responding M pathway provides the dominant initial input to the dorsal stream which plays an important role in processing movement and spatial location information and is assumed to be relatively more involved in space-based attention. To gain insight into the relationship between M/dorsal and P/ventral activity and deploying visual attention, we used a covert cuing paradigm to manipulate attention while bottom-up and top-down perceptual stimulus variables created M/dorsal and P/ventral-biased conditions. One study examined the object advantage, where responses are faster for within-relative to equidistant between-object shifts of attention. Visual stream contributions to object- and spaced-based attention were revealed using psychophysically equiluminant conditions expected to reduce M/dorsal activity. Other studies investigating visual stream contributions to location-based inhibition of return (IOR) used IOR magnitude as an indicator of the ease or difficulty of deploying spatial attention. Greater IOR was found under P/ventral-biased conditions. Less IOR was found under M/dorsal-biased conditions. The results support the use of M/dorsal and P/ventral-biased conditions as a valuable strategy for studying the relationship between visual stream activity and shifting attention.

A neural model of the temporal dynamics of figure-ground segregation in motion perception

How does the visual system manage to segment a visual scene into surfaces and objects and manage to attend to a target object? Based on psychological and physiological investigations, it has been proposed that the perceptual organization and segmentation of a scene is achieved by the processing at different levels of the visual cortical hierarchy. According to this, motion onset detection, motion-defined shape segregation, and target selection are accomplished by processes which bind together simple features into fragments of increasingly complex configurations at different levels in the processing hierarchy. As an alternative to this hierarchical processing hypothesis, it has been proposed that the processing stages for feature detection and segregation are reflected in different temporal episodes in the response patterns of individual neurons. Such temporal epochs have been observed in the activation pattern of neurons as low as in area V1. Here, we present a neural network model of motion detection, figure-ground segregation and attentive selection which explains these response patterns in an unifying framework. Based on known principles of functional architecture of the visual cortex, we propose that initial motion and motion boundaries are detected at different and hierarchically organized stages in the dorsal pathway. Visual shapes that are defined by boundaries, which were generated from juxtaposed opponent motions, are represented at different stages in the ventral pathway. Model areas in the different pathways interact through feedforward and modulating feedback, while mutual interactions enable the communication between motion and form representations. Selective attention is devoted to shape representations by sending modulating feedback signals from higher levels (working memory) to intermediate levels to enhance their responses. Areas in the motion and form pathway are coupled through top-down feedback with V1 cells at the bottom end of the hierarchy. We propose that the different temporal episodes in the response pattern of V1 cells, as recorded in recent experiments, reflect the strength of modulating feedback signals. This feedback results from the consolidated shape representations from coherent motion patterns and the attentive modulation of responses along the cortical hierarchy. The model makes testable predictions concerning the duration and delay of the temporal episodes of V1 cell responses as well as their response variations that were caused by modulating feedback signals.

Cortical and subcortical predictive dynamics and learning during perception, cognition, emotion and action

An intimate link exists between the predictive and learning processes in the brain. Perceptual/cognitive and spatial/motor processes use complementary predictive mechanisms to learn, recognize, attend and plan about objects in the world, determine their current value, and act upon them. Recent neural models clarify these mechanisms and how they interact in cortical and subcortical brain regions. The present paper reviews and synthesizes data and models of these processes, and outlines a unified theory of predictive brain processing.

A neural model of how the brain computes heading from optic flow in realistic scenes

Visually-based navigation is a key competence during spatial cognition. Animals avoid obstacles and approach goals in novel cluttered environments using optic flow to compute heading with respect to the environment. Most navigation models try either explain data, or to demonstrate navigational competence in real-world environments without regard to behavioral and neural substrates. The current article develops a model that does both. The ViSTARS neural model describes interactions among neurons in the primate magnocellular pathway, including V1, MT(+), and MSTd. Model outputs are quantitatively similar to human heading data in response to complex natural scenes. The model estimates heading to within 1.5 degrees in random dot or photo-realistically rendered scenes, and within 3 degrees in video streams from driving in real-world environments. Simulated rotations of less than 1 degrees /s do not affect heading estimates, but faster simulated rotation rates do, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.

Visual object cognition precedes but also temporally overlaps mental rotation

Two-dimensional, mental rotation of alphanumeric characters and geometric figures is related to linear increases in parietal negativity between 400 and 800 ms as rotation increases, similar to linear increases with rotation in response times. This suggests that the frontoparietal networks implicated in mental rotation are engaged after 400 ms. However, the time course of three-dimensional object mental rotation using the classic Shepard-Metzler task has not been studied, even though this is one of the most commonly used versions in behavioral and neuroimaging work. Using this task, this study replicated a prior neuroimaging version using event-related potentials. Results confirmed linear mental rotation effects on performance and parietal negativity. In addition, a frontocentral N350 complex that indexes visual object cognition processes was more negative with mental rotation and showed linear trends at frontopolar sites from 200 to 700 ms and centrofrontal sites from 400 to 500 ms. The centrofrontal negativity has been implicated in object working memory processes in ventrolateral prefrontal and occipitotemporal areas. The frontopolar N350 has been implicated in processes that compute the spatial relations among parts of objects to resolve visual differences between object representations and enable an accurate cognitive decision involving a network of ventrocaudal intraparietal, ventral premotor, and inferotemporal cortices. Overall, the time course indicates that visual object cognition processes precede (200-500 ms) but also overlap the initial phase of mental rotation (500-700 ms) indexed by parietal negativity.

Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance

Visually guided navigation through a cluttered natural scene is a challenging problem that animals and humans accomplish with ease. The ViSTARS neural model proposes how primates use motion information to segment objects and determine heading for purposes of goal approach and obstacle avoidance in response to video inputs from real and virtual environments. The model produces trajectories similar to those of human navigators. It does so by predicting how computationally complementary processes in cortical areas MT(-)/MSTv and MT(+)/MSTd compute object motion for tracking and self-motion for navigation, respectively. The model's retina responds to transients in the input stream. Model V1 generates a local speed and direction estimate. This local motion estimate is ambiguous due to the neural aperture problem. Model MT(+) interacts with MSTd via an attentive feedback loop to compute accurate heading estimates in MSTd that quantitatively simulate properties of human heading estimation data. Model MT(-) interacts with MSTv via an attentive feedback loop to compute accurate estimates of speed, direction and position of moving objects. This object information is combined with heading information to produce steering decisions wherein goals behave like attractors and obstacles behave like repellers. These steering decisions lead to navigational trajectories that closely match human performance.