Finding and recognizing objects in natural scenes: complementary computations in the dorsal and ventral visual systems

Reduced spatial attentional distribution in older adults

Older adults show decline in visual search performance, but the underlying cause remains unclear. It has been suggested that older adults' altered performance may be related to reduced spatial attention to peripheral visual information compared with younger adults. In this study, 18 younger (M = 21.6 years) and 16 older (M = 69.1 years) participants performed pop-out and serial visual search tasks with variously sized gaze-contingent artificial central scotomas (3°, 5°, or 7° diameter). By occluding central vision, we measured how attention to the periphery was contributing to the search performance. We also tested the effect of target eccentricity on search times and eye movements. We hypothesized that, if attention is reduced primarily in the periphery in older adults, we would observe longer search times for more eccentric targets and with central occlusion. During the pop-out search, older adults showed a steeper decline in search performance with increasing eccentricity and central scotoma size compared with younger adults. In contrast, during the serial search, older adults had longer search times than younger adults overall, independent of target eccentricity and scotoma size. Longer search times were attributed to higher cost-per-item slopes, indicating increased difficulty in simultaneously processing complex symbols made up of separable features in aging, possibly stemming from challenges in spatially binding individual features. Altogether, our findings point to fewer attentional resources of simultaneous visual processing to distribute over space or separable features of objects, consistent with decreased dorsal visual stream functioning in aging.

Topography of inputs into the hippocampal formation of a food-caching bird

The mammalian hippocampal formation (HF) is organized into domains associated with different functions. These differences are driven in part by the pattern of input along the hippocampal long axis, such as visual input to the septal hippocampus and amygdalar input to the temporal hippocampus. HF is also organized along the transverse axis, with different patterns of neural activity in the hippocampus and the entorhinal cortex. In some birds, a similar organization has been observed along both of these axes. However, it is not known what role inputs play in this organization. We used retrograde tracing to map inputs into HF of a food-caching bird, the black-capped chickadee. We first compared two locations along the transverse axis: the hippocampus and the dorsolateral hippocampal area (DL), which is analogous to the entorhinal cortex. We found that pallial regions predominantly targeted DL, while some subcortical regions like the lateral hypothalamus (LHy) preferentially targeted the hippocampus. We then examined the hippocampal long axis and found that almost all inputs were topographic along this direction. For example, the anterior hippocampus was preferentially innervated by thalamic regions, while the posterior hippocampus received more amygdalar input. Some of the topographies we found bear a resemblance to those described in the mammalian brain, revealing a remarkable anatomical similarity of phylogenetically distant animals. More generally, our work establishes the pattern of inputs to HF in chickadees. Some of these patterns may be unique to chickadees, laying the groundwork for studying the anatomical basis of these birds' exceptional hippocampal memory.

Using Natural Scenes to Enhance our Understanding of the Cerebral Cortex's Role in Visual Search

Using natural scenes is an approach to studying the visual and eye movement systems approximating how these systems function in everyday life. This review examines the results from behavioral and neurophysiological studies using natural scene viewing in humans and monkeys. The use of natural scenes for the study of cerebral cortical activity is relatively new and presents challenges for data analysis. Methods and results from the use of natural scenes for the study of the visual and eye movement cortex are presented, with emphasis on new insights that this method provides enhancing what is known about these cortical regions from the use of conventional methods.

Emotion, motivation, decision-making, the orbitofrontal cortex, anterior cingulate cortex, and the amygdala

The orbitofrontal cortex and amygdala are involved in emotion and in motivation, but the relationship between these functions performed by these brain structures is not clear. To address this, a unified theory of emotion and motivation is described in which motivational states are states in which instrumental goal-directed actions are performed to obtain rewards or avoid punishers, and emotional states are states that are elicited when the reward or punisher is or is not received. This greatly simplifies our understanding of emotion and motivation, for the same set of genes and associated brain systems can define the primary or unlearned rewards and punishers such as sweet taste or pain. Recent evidence on the connectivity of human brain systems involved in emotion and motivation indicates that the orbitofrontal cortex is involved in reward value and experienced emotion with outputs to cortical regions including those involved in language, and is a key brain region involved in depression and the associated changes in motivation. The amygdala has weak effective connectivity back to the cortex in humans, and is implicated in brainstem-mediated responses to stimuli such as freezing and autonomic activity, rather than in declarative emotion. The anterior cingulate cortex is involved in learning actions to obtain rewards, and with the orbitofrontal cortex and ventromedial prefrontal cortex in providing the goals for navigation and in reward-related effects on memory consolidation mediated partly via the cholinergic system.

Hippocampal spatial view cells for memory and navigation, and their underlying connectivity in humans

Hippocampal and parahippocampal gyrus spatial view neurons in primates respond to the spatial location being looked at. The representation is allocentric, in that the responses are to locations "out there" in the world, and are relatively invariant with respect to retinal position, eye position, head direction, and the place where the individual is located. The underlying connectivity in humans is from ventromedial visual cortical regions to the parahippocampal scene area, leading to the theory that spatial view cells are formed by combinations of overlapping feature inputs self-organized based on their closeness in space. Thus, although spatial view cells represent "where" for episodic memory and navigation, they are formed by ventral visual stream feature inputs in the parahippocampal gyrus in what is the parahippocampal scene area. A second "where" driver of spatial view cells are parietal inputs, which it is proposed provide the idiothetic update for spatial view cells, used for memory recall and navigation when the spatial view details are obscured. Inferior temporal object "what" inputs and orbitofrontal cortex reward inputs connect to the human hippocampal system, and in macaques can be associated in the hippocampus with spatial view cell "where" representations to implement episodic memory. Hippocampal spatial view cells also provide a basis for navigation to a series of viewed landmarks, with the orbitofrontal cortex reward inputs to the hippocampus providing the goals for navigation, which can then be implemented by hippocampal connectivity in humans to parietal cortex regions involved in visuomotor actions in space. The presence of foveate vision and the highly developed temporal lobe for object and scene processing in primates including humans provide a basis for hippocampal spatial view cells to be key to understanding episodic memory in the primate and human hippocampus, and the roles of this system in primate including human navigation.

Topography of inputs into the hippocampal formation of a food-caching bird

The mammalian hippocampal formation (HF) is organized into domains associated with different functions. These differences are driven in part by the pattern of input along the hippocampal long axis, such as visual input to the septal hippocampus and amygdalar input to temporal hippocampus. HF is also organized along the transverse axis, with different patterns of neural activity in the hippocampus and the entorhinal cortex. In some birds, a similar organization has been observed along both of these axes. However, it is not known what role inputs play in this organization. We used retrograde tracing to map inputs into HF of a food-caching bird, the black-capped chickadee. We first compared two locations along the transverse axis: the hippocampus and the dorsolateral hippocampal area (DL), which is analogous to the entorhinal cortex. We found that pallial regions predominantly targeted DL, while some subcortical regions like the lateral hypothalamus (LHy) preferentially targeted the hippocampus. We then examined the hippocampal long axis and found that almost all inputs were topographic along this direction. For example, the anterior hippocampus was preferentially innervated by thalamic regions, while posterior hippocampus received more amygdalar input. Some of the topographies we found bear resemblance to those described in the mammalian brain, revealing a remarkable anatomical similarity of phylogenetically distant animals. More generally, our work establishes the pattern of inputs to HF in chickadees. Some of these patterns may be unique to chickadees, laying the groundwork for studying the anatomical basis of these birds ’ exceptional hippocampal memory.

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

First, neurophysiological evidence for the learning of invariant representations in the inferior temporal visual cortex is described. This includes object and face representations with invariance for position, size, lighting, view and morphological transforms in the temporal lobe visual cortex; global object motion in the cortex in the superior temporal sulcus; and spatial view representations in the hippocampus that are invariant with respect to eye position, head direction, and place. Second, computational mechanisms that enable the brain to learn these invariant representations are proposed. For the ventral visual system, one key adaptation is the use of information available in the statistics of the environment in slow unsupervised learning to learn transform-invariant representations of objects. This contrasts with deep supervised learning in artificial neural networks, which uses training with thousands of exemplars forced into different categories by neuronal teachers. Similar slow learning principles apply to the learning of global object motion in the dorsal visual system leading to the cortex in the superior temporal sulcus. The learning rule that has been explored in VisNet is an associative rule with a short-term memory trace. The feed-forward architecture has four stages, with convergence from stage to stage. This type of slow learning is implemented in the brain in hierarchically organized competitive neuronal networks with convergence from stage to stage, with only 4-5 stages in the hierarchy. Slow learning is also shown to help the learning of coordinate transforms using gain modulation in the dorsal visual system extending into the parietal cortex and retrosplenial cortex. Representations are learned that are in allocentric spatial view coordinates of locations in the world and that are independent of eye position, head direction, and the place where the individual is located. This enables hippocampal spatial view cells to use idiothetic, self-motion, signals for navigation when the view details are obscured for short periods.

Spatial coordinate transforms linking the allocentric hippocampal and egocentric parietal primate brain systems for memory, action in space, and navigation

A theory and model of spatial coordinate transforms in the dorsal visual system through the parietal cortex that enable an interface via posterior cingulate and related retrosplenial cortex to allocentric spatial representations in the primate hippocampus is described. First, a new approach to coordinate transform learning in the brain is proposed, in which the traditional gain modulation is complemented by temporal trace rule competitive network learning. It is shown in a computational model that the new approach works much more precisely than gain modulation alone, by enabling neurons to represent the different combinations of signal and gain modulator more accurately. This understanding may have application to many brain areas where coordinate transforms are learned. Second, a set of coordinate transforms is proposed for the dorsal visual system/parietal areas that enables a representation to be formed in allocentric spatial view coordinates. The input stimulus is merely a stimulus at a given position in retinal space, and the gain modulation signals needed are eye position, head direction, and place, all of which are present in the primate brain. Neurons that encode the bearing to a landmark are involved in the coordinate transforms. Part of the importance here is that the coordinates of the allocentric view produced in this model are the same as those of spatial view cells that respond to allocentric view recorded in the primate hippocampus and parahippocampal cortex. The result is that information from the dorsal visual system can be used to update the spatial input to the hippocampus in the appropriate allocentric coordinate frame, including providing for idiothetic update to allow for self-motion. It is further shown how hippocampal spatial view cells could be useful for the transform from hippocampal allocentric coordinates to egocentric coordinates useful for actions in space and for navigation.

Non-accidental properties, metric invariance, and encoding by neurons in a model of ventral stream visual object recognition, VisNet

When objects transform into different views, some properties are maintained, such as whether the edges are convex or concave, and these non-accidental properties are likely to be important in view-invariant object recognition. The metric properties, such as the degree of curvature, may change with different views, and are less likely to be useful in object recognition. It is shown that in a model of invariant visual object recognition in the ventral visual stream, VisNet, non-accidental properties are encoded much more than metric properties by neurons. Moreover, it is shown how with the temporal trace rule training in VisNet, non-accidental properties of objects become encoded by neurons, and how metric properties are treated invariantly. We also show how VisNet can generalize between different objects if they have the same non-accidental property, because the metric properties are likely to overlap. VisNet is a 4-layer unsupervised model of visual object recognition trained by competitive learning that utilizes a temporal trace learning rule to implement the learning of invariance using views that occur close together in time. A second crucial property of this model of object recognition is, when neurons in the level corresponding to the inferior temporal visual cortex respond selectively to objects, whether neurons in the intermediate layers can respond to combinations of features that may be parts of two or more objects. In an investigation using the four sides of a square presented in every possible combination, it was shown that even though different layer 4 neurons are tuned to encode each feature or feature combination orthogonally, neurons in the intermediate layers can respond to features or feature combinations present is several objects. This property is an important part of the way in which high capacity can be achieved in the four-layer ventral visual cortical pathway. These findings concerning non-accidental properties and the use of neurons in intermediate layers of the hierarchy help to emphasise fundamental underlying principles of the computations that may be implemented in the ventral cortical visual stream used in object recognition.

The visual development of hand-centered receptive fields in a neural network model of the primate visual system trained with experimentally recorded human gaze changes

Neurons have been found in the primate brain that respond to objects in specific locations in hand-centered coordinates. A key theoretical challenge is to explain how such hand-centered neuronal responses may develop through visual experience. In this paper we show how hand-centered visual receptive fields can develop using an artificial neural network model, VisNet, of the primate visual system when driven by gaze changes recorded from human test subjects as they completed a jigsaw. A camera mounted on the head captured images of the hand and jigsaw, while eye movements were recorded using an eye-tracking device. This combination of data allowed us to reconstruct the retinal images seen as humans undertook the jigsaw task. These retinal images were then fed into the neural network model during self-organization of its synaptic connectivity using a biologically plausible trace learning rule. A trace learning mechanism encourages neurons in the model to learn to respond to input images that tend to occur in close temporal proximity. In the data recorded from human subjects, we found that the participant's gaze often shifted through a sequence of locations around a fixed spatial configuration of the hand and one of the jigsaw pieces. In this case, trace learning should bind these retinal images together onto the same subset of output neurons. The simulation results consequently confirmed that some cells learned to respond selectively to the hand and a jigsaw piece in a fixed spatial configuration across different retinal views.

The Development of Hand-Centered Visual Representations in the Primate Brain: A Computer Modeling Study Using Natural Visual Scenes

Neurons that respond to visual targets in a hand-centered frame of reference have been found within various areas of the primate brain. We investigate how hand-centered visual representations may develop in a neural network model of the primate visual system called VisNet, when the model is trained on images of the hand seen against natural visual scenes. The simulations show how such neurons may develop through a biologically plausible process of unsupervised competitive learning and self-organization. In an advance on our previous work, the visual scenes consisted of multiple targets presented simultaneously with respect to the hand. Three experiments are presented. First, VisNet was trained with computerized images consisting of a realistic image of a hand and a variety of natural objects, presented in different textured backgrounds during training. The network was then tested with just one textured object near the hand in order to verify if the output cells were capable of building hand-centered representations with a single localized receptive field. We explain the underlying principles of the statistical decoupling that allows the output cells of the network to develop single localized receptive fields even when the network is trained with multiple objects. In a second simulation we examined how some of the cells with hand-centered receptive fields decreased their shape selectivity and started responding to a localized region of hand-centered space as the number of objects presented in overlapping locations during training increases. Lastly, we explored the same learning principles training the network with natural visual scenes collected by volunteers. These results provide an important step in showing how single, localized, hand-centered receptive fields could emerge under more ecologically realistic visual training conditions.

Invariant visual object recognition: biologically plausible approaches

Key properties of inferior temporal cortex neurons are described, and then, the biological plausibility of two leading approaches to invariant visual object recognition in the ventral visual system is assessed to investigate whether they account for these properties. Experiment 1 shows that VisNet performs object classification with random exemplars comparably to HMAX, except that the final layer C neurons of HMAX have a very non-sparse representation (unlike that in the brain) that provides little information in the single-neuron responses about the object class. Experiment 2 shows that VisNet forms invariant representations when trained with different views of each object, whereas HMAX performs poorly when assessed with a biologically plausible pattern association network, as HMAX has no mechanism to learn view invariance. Experiment 3 shows that VisNet neurons do not respond to scrambled images of faces, and thus encode shape information. HMAX neurons responded with similarly high rates to the unscrambled and scrambled faces, indicating that low-level features including texture may be relevant to HMAX performance. Experiment 4 shows that VisNet can learn to recognize objects even when the view provided by the object changes catastrophically as it transforms, whereas HMAX has no learning mechanism in its S-C hierarchy that provides for view-invariant learning. This highlights some requirements for the neurobiological mechanisms of high-level vision, and how some different approaches perform, in order to help understand the fundamental underlying principles of invariant visual object recognition in the ventral visual stream.