Visual navigation with a neural network

Decoding self-motion from visual image sequence predicts distinctive features of reflexive motor responses to visual motion

Visual motion analysis is crucial for humans to detect external moving objects and self-motion which are informative for planning and executing actions for various interactions with environments. Here we show that the image motion analysis trained to decode the self-motion during human natural movements by a convolutional neural network exhibits similar specificities with the reflexive ocular and manual responses induced by a large-field visual motion, in terms of stimulus spatiotemporal frequency tuning. The spatiotemporal frequency tuning of the decoder peaked at high-temporal and low-spatial frequencies, as observed in the reflexive ocular and manual responses, but differed significantly from the frequency power of the visual image itself and the density distribution of self-motion. Further, artificial manipulations of the learning data sets predicted great changes in the specificity of the spatiotemporal tuning. Interestingly, despite similar spatiotemporal frequency tunings in the vertical-axis rotational direction and in the transversal direction to full-field visual stimuli, the tunings for center-masked stimuli were different between those directions, and the specificity difference is qualitatively similar to the discrepancy between ocular and manual responses, respectively. In addition, the representational analysis demonstrated that head-axis rotation was decoded by relatively simple spatial accumulation over the visual field, while the transversal motion was decoded by more complex spatial interaction of visual information. These synthetic model examinations support the idea that visual motion analyses eliciting the reflexive motor responses, which are critical in interacting with the external world, are acquired for decoding self-motion.

Competitive Dynamics in MSTd: A Mechanism for Robust Heading Perception Based on Optic Flow

Human heading perception based on optic flow is not only accurate, it is also remarkably robust and stable. These qualities are especially apparent when observers move through environments containing other moving objects, which introduce optic flow that is inconsistent with observer self-motion and therefore uninformative about heading direction. Moving objects may also occupy large portions of the visual field and occlude regions of the background optic flow that are most informative about heading perception. The fact that heading perception is biased by no more than a few degrees under such conditions attests to the robustness of the visual system and warrants further investigation. The aim of the present study was to investigate whether recurrent, competitive dynamics among MSTd neurons that serve to reduce uncertainty about heading over time offer a plausible mechanism for capturing the robustness of human heading perception. Simulations of existing heading models that do not contain competitive dynamics yield heading estimates that are far more erratic and unstable than human judgments. We present a dynamical model of primate visual areas V1, MT, and MSTd based on that of Layton, Mingolla, and Browning that is similar to the other models, except that the model includes recurrent interactions among model MSTd neurons. Competitive dynamics stabilize the model's heading estimate over time, even when a moving object crosses the future path. Soft winner-take-all dynamics enhance units that code a heading direction consistent with the time history and suppress responses to transient changes to the optic flow field. Our findings support recurrent competitive temporal dynamics as a crucial mechanism underlying the robustness and stability of perception of heading.

Motions add, orientations don't, in the human visual system

Humans can distinguish between contours of similar orientation, and between directions of visual motion. There is consensus that both of these capabilities depend on selective activation of tuned neural channels. The bandwidths of these tuned channels are estimated here by modelling previously published empirical data. Human subjects were presented with a rapid stream of randomly oriented gratings, or randomly directed motions, and asked to respond when they saw a target stimulus. For the orientation task, subjects were less likely to respond when two preceding orientations were close to the target orientation but differed from each other, presumably due to a failure of summation. For the motion data, by contrast, subjects were more likely to respond when the vector sum of two previous directions was in the target direction. Fitting a cortical signal-processing model to these data showed that the direction bandwidth of motion sensors is about three times the bandwidth of orientation sensors, and that it is the large bandwidth that allows the summation of motion stimuli. The differing bandwidths of orientation and motion sensors presumably equip them for differing tasks, such as orientation discrimination and estimation of heading, respectively.

Cortical neurons combine visual cues about self-movement

Visual cues about self-movement are derived from the patterns of optic flow and the relative motion of discrete objects. We recorded dorsal medial superior temporal (MSTd) cortical neurons in monkeys that held centered visual fixation while viewing optic flow and object motion stimuli simulating the self-movement cues seen during translation on a circular path. Twenty stimulus configurations presented naturalistic combinations of optic flow with superimposed objects that simulated either earth-fixed landmark objects or independently moving animate objects. Landmarks and animate objects yield the same response interactions with optic flow; mainly additive effects, with a substantial number of sub- and super-additive responses. Sub- and super-additive interactions reflect each neuron's local and global motion sensitivities: Local motion sensitivity is based on the spatial arrangement of directions created by object motion and the surrounding optic flow. Global motion sensitivity is based on the temporal sequence of self-movement headings that define a simulated path through the environment. We conclude that MST neurons' spatio-temporal response properties combine object motion and optic flow cues to represent self-movement in diverse, naturalistic circumstances.

A neural model of how the brain computes heading from optic flow in realistic scenes

Visually-based navigation is a key competence during spatial cognition. Animals avoid obstacles and approach goals in novel cluttered environments using optic flow to compute heading with respect to the environment. Most navigation models try either explain data, or to demonstrate navigational competence in real-world environments without regard to behavioral and neural substrates. The current article develops a model that does both. The ViSTARS neural model describes interactions among neurons in the primate magnocellular pathway, including V1, MT(+), and MSTd. Model outputs are quantitatively similar to human heading data in response to complex natural scenes. The model estimates heading to within 1.5 degrees in random dot or photo-realistically rendered scenes, and within 3 degrees in video streams from driving in real-world environments. Simulated rotations of less than 1 degrees /s do not affect heading estimates, but faster simulated rotation rates do, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.

Cortical neuronal responses to optic flow are shaped by visual strategies for steering

We hypothesized that neuronal responses to virtual self-movement would be enhanced during steering tasks. We recorded the activity of medial superior temporal (MSTd) neurons in monkeys trained to steer a straight-ahead course, using optic flow. We found smaller optic flow responses during active steering than during the passive viewing of the same stimuli. Behavioral analysis showed that the monkeys had learned to steer using local motion cues. Retraining the monkeys to use the global pattern of optic flow reversed the effects of the active-steering task: active steering then evoked larger responses than passive viewing. We then compared the responses of neurons during active steering by local motion and by global patterns: Local motion trials promoted the use of local dot movement near the center of the stimulus by occluding the peripheral visual field midway through the trial. Global pattern trials promoted the use of radial pattern movement by occluding the central visual field midway through the trial. In this study, identical full-field optic-flow stimuli evoked larger responses in global-pattern trials than in local motion trials. We conclude that the selection of specific visual cues reflects strategies for active steering and alters MSTd neuronal responses to optic flow.

Behavioral Influences on Cortical Neuronal Responses to Optic Flow

Optic flow selectively activates neurons in medial superior temporal (MST) cortex. We find that many MST neurons yield larger and more selective responses when the optic flow guides a subsequent eye movement. Smaller, less selective responses are seen when optic flow is preceded by a flashed precue that guides eye movements. Selectivity can decrease by a third (32%) after a flashed precue is presented at a peripheral location as a small spot specifying the target location of the eye movement. Smaller decreases in selectivity (18%) occur when the precue is presented centrally with its shape specifying the target location. Shape precues presented centrally, but not linked to specific target locations, do not appear to alter optic flow selectivity. The effects of spatial precueing can be reversed so that the precue leads to larger and more selective optic flow responses: A flashed precue presented as a distracter before behaviorally relevant optic flow is associated with larger optic flow responses and a 45% increase in selectivity. Together, these findings show that spatial precues can decrease or increase the size and selectivity of optic flow responses depending on the associated behavioral contingencies.

Factors affecting curved versus straight path heading perception

Displays commonly used for testing heading judgments in the presence of rotations are ambiguous to observers. They can be interpreted equally well as motion in a straight line while rotating the eyes or as motion on a curved path. This has led to conflicting results from studies that use these displays. In this study, we tested several factors that might influence which of these two interpretations observers see. These factors included the size of the field of view, the duration of the stimulus, textured scenes versus random-dot displays, and whether or not observers were given a description of their path. The only factor that had a significant effect on path perception was whether or not observers were given instructions describing their path of motion. Under all conditions without instructions, we found that observers responded in a way that was consistent with the perception of motion on a curved path.

Resolution of complex motion detectors in the central and peripheral visual field

We examine how local direction signals are combined to compute the focus of radial motion (FRM) in random dot patterns and examine how this process changes across the visual field. Equivalent noise analysis showed that a loss in FRM accuracy was largely attributable to an increase in local motion detector noise with little or no change in efficiency across the visual field. The minimum separation for discriminating the foci of two overlapping optic flow patterns increased in the periphery faster than predicted from the resolution for a single FRM. This behavior requires that observers average numerous local velocities to estimate the FRM, which enables resistance to internal and external noise and endows the system with the property of position invariance. However, such pooling limits the precision with which multiple looming objects can be discriminated, especially in the peripheral visual field.

Global motion mechanisms compensate local motion deficits in a patient with a bilateral occipital lobe lesion

Successive stages of cortical processing encode increasingly more complex types of information. In the visual motion system this increasing complexity, complemented by an increase in spatial summation, has proven effective in characterizing the mechanisms mediating visual perception. Here we report psychophysical results from a motion-impaired stroke patient, WB, whose pattern of deficits over time reveals a systematic shift in spatial scale for processing speed. We show that following loss in sensitivity to low-level motion direction WB's representation of speed shifts to larger spatial scales, consistent with recruitment of intact high-level mechanisms. With the recovery of low-level motion processing WB's representation of speed shifts back to small spatial scales. These results support the recruitment of high-level visual mechanisms in cases where lower-level function is impaired and suggest that, as an experimental paradigm, spatial summation may provide an important avenue for investigating functional recovery in patients following damage to visually responsive cortex.

Measurement of rate of expansion in the perception of radial motion

Optic flow generated by rigid surface patches can be decomposed into a small number of elementary motion types. In these experiments, we show that the human visual system can evaluate expansion, one of these motion types, metrically. Moreover, we show that the discrimination of rates of expansion are spatially local. Because the estimation of the focus of expansion is somewhat imprecise, this locality sometimes produces predictable errors in the estimation of rate of expansion. One can make predictions like this with a model adapted from one previously developed for angular-velocity discrimination.

Spectrum analysis of motion parallax in a 3D cluttered scene and application to egomotion

Previous methods for estimating observer motion in a rigid 3D scene assume that image velocities can be measured at isolated points. When the observer is moving through a cluttered 3D scene such as a forest, however, pointwise measurements of image velocity are more challenging to obtain because multiple depths, and hence multiple velocities, are present in most local image regions. We introduce a method for estimating egomotion that avoids pointwise image velocity estimation as a first step. In its place, the direction of motion parallax in local image regions is estimated, using a spectrum-based method, and these directions are then combined to directly estimate 3D observer motion. There are two advantages to this approach. First, the method can be applied to a wide range of 3D cluttered scenes, including those for which pointwise image velocities cannot be measured because only normal velocity information is available. Second, the egomotion estimates can be used as a posterior constraint on estimating pointwise image velocities, since known egomotion parameters constrain the candidate image velocities at each point to a one-dimensional rather than a two-dimensional space.

A visual motion sensor based on the properties of V1 and MT neurons

The motion response properties of neurons increase in complexity as one moves from primary visual cortex (V1), up to higher cortical areas such as the middle temporal (MT) and the medial superior temporal area (MST). Many of the features of V1 neurons can now be replicated using computational models based on spatiotemporal filters. However until recently, relatively little was known about how the motion analysing properties of MT neurons could originate from the V1 neurons that provide their inputs. This has constrained the development of models of the MT-MST stages which have been linked to higher level motion processing tasks such as self-motion perception and depth estimation. I describe the construction of a motion sensor built up in stages from two spatiotemporal filters with properties based on V1 neurons. The resulting composite sensor is shown to have spatiotemporal frequency response profiles, speed and direction tuning responses that are comparable to MT neurons. The sensor is designed to work with digital images and can therefore be used as a realistic front-end to models of MT and MST neuron processing; it can be probed with the same two-dimensional motion stimuli used to test the neurons and has the potential to act as a building block for more complex models of motion processing.

A model using MT-like motion-opponent operators explains an illusory transformation in the optic flow field

Previous studies have shown that a physiologically based model using motion-opponent operators to compute heading performs accurately for simulated observer translations. Here we show how this model can explain an illusory shift in the perceived focus of expansion of a radial flow field that occurs when a field of laterally moving dots is superimposed on a field of radially moving dots. Furthermore, we can use the model to predict the perceptual shift of the focus of expansion for novel visual stimuli. These results support the hypothesis that this illusion results from motion subtraction during the processing of optic flow fields.

On motion detection through a multi-layer neural network architecture

A neural network model called lateral interaction in accumulative computation for detection of non-rigid objects from motion of any of their parts in indefinite sequences of images is presented. Some biological evidences inspire the model. After introducing the model, the complete multi-layer neural architecture is offered in this paper. The architecture consists of four layers that perform segmentation by gray level bands, accumulative charge computation, charge redistribution by gray level bands and moving object fusion. The lateral interaction in accumulative computation associated learning algorithm is also introduced. Some examples that explain the usefulness of the system we propose are shown at the end of this article.

A neural network model of spiral-planar motion tuning in MSTd

Neurophysiological studies in MSTd report the existence of motion pattern selective cells whose visual motion properties span a continuum of values, suggesting a role in estimates of self-motion from optic flow. Biologically motivated models of heading estimation support this view, having identified similar visual motion properties within their "neural" structures. While such models have addressed the computational sufficiency of their respective feed-forward designs they have not explicitly examined the underlying computational structures, particularly as they relate to the interaction between planar and spiral motion responses within MSTd. Here we use an expanded stimulus training set that includes planar motions to extend the range of neurophysiological properties identified within an existing network structure [Network: Comput. Neural Syst. 9 (1998) 467]. In doing so, we quantify the emergent planar motion properties within the network hidden layer and examine how they interact, functionally and computationally, with cardinal/spiral motion pattern responses. Throughout the hidden layer we demonstrate that the input activation associated with a unit's preferred planar motion is consistent with an overlapping gradient hypothesis [J. Neurophysiol. 65(6) (1991) 1346]. Together with the change to a peripheral excitation profile in the presence of a unit's preferred spiral motion these results suggest a more complex computational architecture in which the cell's 'classical' receptive field properties are dependent on the type of stimulus used to map them. Based on the computational model we propose an experimental paradigm to investigate the existence of equivalent computational structures in MSTd.

Computing heading in the presence of moving objects: a model that uses motion-opponent operators

Psychophysical experiments have shown that human heading judgments can be biased by the presence of moving objects. Here we present a theoretical argument that motion differences can account for the direction of bias seen in humans. We further examine the responses of a computer simulation of a model for computing heading that uses motion-opponent operators similar to cells in the primate middle temporal visual area. When moving objects are present, this model shows similar biases to those seen with humans, suggesting that such a model may underlie human heading computations.

Perceiving curvilinear heading in the presence of moving objects

Four experiments were directed at understanding the influence of multiple moving objects on curvilinear (i.e., circular and elliptical) heading perception. Displays simulated observer movement over a ground plane in the presence of moving objects depicted as transparent, opaque, or black cubes. Objects either moved parallel to or intersected the observer's path and either retreated from or approached the moving observer. Heading judgments were accurate and consistent across all conditions. The significance of these results for computational models of heading perception and for information in the global optic flow field about observer and object motion is discussed.

Heading and path percepts from visual flow and eye pursuit signals

The percept of self-motion through the environment is supported by visual motion signals and eye movement signals. The interaction between these signals by decoupling of the eye movement and the pattern of retinal motion during brief simulated ego-movement on straight or circular trajectories was studied. A new response method enabled subjects to report perceived destination and perceived curvature of their future path simultaneously. Various combinations of simulated gaze rotation in the retinal flow and eye pursuit were investigated. Simulated gaze rotation ranged from consistent and larger than, to opponent and larger than eye pursuit. It was found that the perceived destination shifts non-linearly with the mismatch between simulated gaze rotation and eye pursuit. The non-linearity is also revealed in the perceived tangent heading direction and perceived path curvature, although to different extent in different subjects. For the same retinal flow, eye pursuit that is consistent with the simulated gaze rotation reduces heading error and the perceived path straightens out. In contrast, perceived path and/or heading do not become more curved or more biased in the direction opposite to pursuit when the eye -in-head rotation is opposite to the simulated gaze rotation. These observations point to modulation of the effect of the extra-retinal pursuit signal by the visual evidence for eye rotation. In a second experiment, one presented to a stationary eye the sum of a component of simulated gaze rotation and radial flow. It was found that the bi-circular flow component, that characterizes the change in pattern of flow directions by the gaze rotation, induces a shift of perceived heading without appreciable perceived path curvature. Conversely, the complementary component of simulated gaze rotation (bi-radial flow) evokes a percept of motion on a curved path with a small tangent heading error. It was suggested that bi-circular and bi-radial flow components contribute primarily to percepts of heading and path curvature, respectively.

A model of human heading judgement in forward motion

We developed a new computational model of human heading judgement from retinal flow. The model uses two assumptions: a large number of sampling points in the flow field and a symmetric sampling region around the origin. The algorithm estimates self-rotation parameters by calculating statistics whose expectations correspond to the rotation parameters. After the rotational components are removed from the retinal flow, the heading direction is recovered from the flow field. Performance of the model was compared with human data in three psychophysical experiments. In the first experiment, we generated stimuli which simulated self-motion toward the ground, a cloud or a frontoparallel plane and found that the simulation results of the model were consistent with human performance. In the second and third experiment, we measured the slope of the perceived versus simulated heading function when a perturbation velocity weighted according to the distance relative to the fixation distance was added to the vertical velocity component under the cloud condition. It was found that as the magnitude of the perturbation was increased, the slope of the function increased. The characteristics observed in the experiments can be explained well by the proposed model.

Differential effects of shared attention on perception of heading and 3-D object motion

When a person moves in a straight line through a stationary environment, the images of object surfaces move in a radial pattern away from a single point. This point, known as the focus of expansion (FOE), corresponds to the person's direction of motion. People judge their heading from image motion quite well in this situation. They perform most accurately when they can see the region around the FOE, which contains the most useful information for this task. Furthermore, a large moving object in the scene has no effect on observer heading judgments unless it obscures the FOE. Therefore, observers may obtain the most accurate heading judgments by focusing their attention on the region around the FOE. However, in many situations (e.g., driving), the observer must pay attention to other moving objects in the scene (e.g., cars and pedestrians) to avoid collisions. These objects may be located far from the FOE in the visual field. We tested whether people can accurately judge their heading and the three-dimensional (3-D) motion of objects while paying attention to one or the other task. The results show that differential allocation of attention affects people's ability to judge 3-D object motion much more than it affects their ability to judge heading. This suggests that heading judgments are computed globally, whereas judgments about object motion may require more focused attention.

A self-organizing neural network architecture for navigation using optic flow

This article describes a self-organizing neural network architecture that transforms optic flow and eye position information into representations of heading, scene depth, and moving object locations. These representations are used to navigate reactively in simulations involving obstacle avoidance and pursuit of a moving target. The network's weights are trained during an action-perception cycle in which self-generated eye and body movements produce optic flow information, thus allowing the network to tune itself without requiring explicit knowledge of sensor geometry. The confounding effect of eye movement during translation is suppressed by learning the relationship between eye movement outflow commands and the optic flow signals that they induce. The remaining optic flow field is due to only observer translation and independent motion of objects in the scene. A self-organizing feature map categorizes normalized translational flow patterns, thereby creating a map of cells that code heading directions. Heading information is then recombined with translational flow patterns in two different ways to form maps of scene depth and moving object locations. Most of the learning processes take place concurrently and evolve through unsupervised learning. Mapping the learned heading representations onto heading labels or motor commands requires additional structure. Simulations of the network verify its performance using both noise-free and noisy optic flow information.

Mathematical analysis of motion-opponent mechanisms used in the determination of heading and depth

A mathematical analysis is presented of a model that uses motion-opponent operators similar to neurons found in the primate middle temporal visual area, to determine observer heading and depth from optical flow information. The response of these operators to depth changes in the form of a slanted plane or a step edge is analyzed, and the outputs of odd-symmetric operators are compared with that of circularly symmetric operators. The analysis shows sources of error from these operators in determining heading and depth and suggests how some of these errors can be mitigated. Simulations are presented that show that the model performs well for a variety of situations.

Human heading estimation during visually simulated curvilinear motion

Recent studies have suggested that humans cannot estimate their direction of forward translation (heading) from the resulting retinal motion (flow field) alone when rotation rates are higher than approximately 1 deg/sec. It has been argued that either oculomotor or static depth cues are necessary to disambiguate the rotational and translational components of the flow field and, thus, to support accurate heading estimation. We have re-examined this issue using visually simulated motion along a curved path towards a layout of random points as the stimulus. Our data show that, in this curvilinear motion paradigm, five of six observers could estimate their heading relatively accurately and precisely (error and uncertainty < approximately 4 deg), even for rotation rates as high as 16 deg/sec, without the benefit of either oculomotor or static depth cues signaling rotation rate. Such performance is inconsistent with models of human self-motion estimation that require rotation information from sources other than the flow field to cancel the rotational flow.

Planar directional contributions to optic flow responses in MST neurons

Many neurons in the dorsal region of the medial superior temporal area (MSTd) of monkey cerebral cortex respond to optic flow stimuli in which the center of motion is shifted off the center of the visual field. Each shifted-center-of-motion stimulus presents both different directions of planar motion throughout the visual field and a unique pattern of global motion across the visual field. We investigated the contribution of planar motion to the responses of these neurons in two experiments. In the first, we compared the responses of 243 neurons to planar motion and to shifted-center-of-motion stimuli created by vector summation of planar motion and radial or circular motion. We found that many neurons preferred the same directions of motion in the combined stimuli as in the planar stimuli, but other neurons did not. When we divided our sample into one group with stronger directionality to both planar and vector combination stimuli and one group with weaker directionality, we found that the neurons with the stronger directionality were those that showed the greatest similarity in the preferred direction of motion for both the planar and combined stimuli. In a second set of experiments, we overlapped planar motion and radial or circular motion to create transparent stimuli with the same motion components as the vector combination stimuli, but without the shifted centers of motion. We found that the neurons that responded most strongly to the planar motion when it was combined with radial or circular motion also responded best when the planar motion was overlapped by a transparent motion stimulus. We conclude that the responses of those neurons with stronger directional responses to both the motion of planar and vector combination stimuli are most readily understood as responding to the total planar motion in the stimulus, a planar motion mechanism. Other neurons that had weaker directional responses showed no such similarity in the preferred directions of planar motion in the vector combination and the transparent overlap stimuli and fit best with a mechanism dependent on the global motion pattern. We also found that neurons having significant responses to both radial and circular motion also responded to the spiral stimuli that result from a vector combination of radial and circular motion. The preferred planar-spiral vector combination stimulus was frequently the one containing that neurons' preferred direction of planar motion, which makes them similar to other MSTd neurons.

Human heading judgments in the presence of moving objects

When moving toward a stationary scene, people judge their heading quite well from visual information alone. Much experimental and modeling work has been presented to analyze how people judge their heading for stationary scenes. However, in everyday life, we often move through scenes that contain moving objects. Most models have difficulty computing heading when moving objects are in the scene, and few studies have examined how well humans perform in the presence of moving objects. In this study, we tested how well people judge their heading in the presence of moving objects. We found that people perform remarkably well under a variety of conditions. The only condition that affects an observer's ability to judge heading accurately consists of a large moving object crossing the observer's path. In this case, the presence of the object causes a small bias in the heading judgments. For objects moving horizontally with respect to the observer, this bias is in the object's direction of motion. These results present a challenge for computational models.

An illusory transformation in a model of optic flow processing

We present results from computer simulations of a biologically plausible model of heading detection in the visual motion pathway of higher mammals. These simulations are closely related to a recently discovered visual illusion in optic flow processing in humans. The model reproduces the results described for humans and suggests a possible explanation, namely that humans interpret the illusory stimuli in terms of egomotion. It provides further indication that the visual system makes use of visual information to cope with eye movement effects in dealing with optic flow.

Perceiving heading in the presence of moving objects

In most models of heading from optic flow a rigid environment is assumed, yet humans often navigate in the presence of independently moving objects. Simple spatial pooling of the flow field would yield systematic heading errors. Alternatively, moving objects could be segmented on the basis of relative motion, dynamic occlusion, or inconsistency with the global flow, and heading determined from the background flow. Displays simulated observer translation toward a frontal random-dot plane, with a 10 deg square moving independently in depth. The path of motion of the object was varied to create a secondary focus of expansion (FOE') 6 deg to the right or left of the actual heading point (FOE), which could bias the perceived heading. There was no effect when the FOE was visible, but when the object moved in front of it, perceived heading was biased toward the FOE' by approximately 1.9 degrees with a transparent object, and approximately 3.4 degrees with an opaque object. The results indicate that scene segmentation does not occur prior to heading estimation, which is consistent with spatial pooling weighted near the FOE. A simple template model based on large-field, center-weighted expansion units accounts for the data. This may actually represent an adaptive solution for navigation with respect to obstacles on the path ahead.

Motion anisotropies and heading detection

In motion-processing areas of the visual cortex in cats and monkeys, an anisotropic distribution of direction selectivities displays a preference for movements away from the fovea. This 'centrifugal bias' has been hypothetically linked to the processing of optic flow fields generated during forward locomotion. In this paper, we show that flow fields induced on the retina in many natural situations of locomotion of higher mammals are indeed qualitatively centrifugal in structure, even when biologically plausible eye movements to stabilize gaze on environmental targets are performed. We propose a network model of heading detection that carries an anisotropy similar to the one found in cat and monkey. In simulations, this model reproduces a number of psychophysical results of human heading detection. It suggests that a recently reported human disability to correctly identify the direction of heading from optic flow when a certain type of eye movement is simulated might be linked to the noncentrifugal structure of the resulting retinal flow field and to the neurophysiological anisotropies.

A model of self-motion estimation within primate extrastriate visual cortex

Perrone [(1992) Journal of the Optical Society of America A, 9, 177-194] recently proposed a template-based model of self-motion estimation which uses direction- and speed-tuned input sensors similar to neurons in area MT of primate visual cortex. Such an approach would generally require an unrealistically large number of templates (five continuous dimensions). However, because primates, including humans, have a number of oculomotor mechanisms which stabilize gaze during locomotion, we can greatly reduce the number of templates required (two continuous dimensions and one compressed and bounded dimension). We therefore refined the model to deal with the gaze-stabilization case and extended it to extract heading and relative depth simultaneously. The new model is consistent with previous human psychophysics and has the emergent property that its output detectors have similar response properties to neurons in area MST.

A Neural Network for the Processing of Optic Flow from Ego-Motion in Man and Higher Mammals

Interest in the processing of optic flow has increased recently in both the neurophysiological and the psychophysical communities. We have designed a neural network model of the visual motion pathway in higher mammals that detects the direction of heading from optic flow. The model is a neural implementation of the subspace algorithm introduced by Heeger and Jepson (1990). We have tested the network in simulations that are closely related to psychophysical and neurophysiological experiments and show that our results are consistent with recent data from both fields. The network reproduces some key properties of human ego-motion perception. At the same time, it produces neurons that are selective for different components of ego-motion flow fields, such as expansions and rotations. These properties are reminiscent of a subclass of neurons in cortical area MSTd, the triple-component neurons. We propose that the output of such neurons could be used to generate a computational map of heading directions in or beyond MST.

The role of central and peripheral vision in perceiving the direction of self-motion

Three experiments were performed to examine the role that central and peripheral vision play in the perception of the direction of translational self-motion, or heading, from optical flow. When the focus of radial outflow was in central vision, heading accuracy was slightly higher with central circular displays (10 degrees-25 degrees diameter) than with peripheral annular displays (40 degrees diameter), indicating that central vision is somewhat more sensitive to this information. Performance dropped rapidly as the eccentricity of the focus of outflow increased, indicating that the periphery does not accurately extract radial flow patterns. Together with recent research on vection and postural adjustments, these results contradict the peripheral dominance hypothesis that peripheral vision is specialized for perception of self-motion. We propose a functional sensitivity hypothesis--that self-motion is perceived on the basis of optical information rather than the retinal locus of stimulation, but that central and peripheral vision are differentially sensitive to the information characteristic of each retinal region.