Monocular depth perception from optical flow by space time signal processing

Optical flow estimation for a periodic image sequence

We propose a temporal modeling approach for determining image motion from a sequence of images wherein the inherent motion is periodic over time. To exploit the periodic nature of the motion, we use a Fourier harmonic representation to model the temporal evolution of the motion field for the entire sequence. We then determine the motion field simultaneously for the different image frames by estimating the parameters of this representation model, where the model order in the Fourier representation serves as a regularization parameter on the temporal coherence of the motion field. This approach can take advantage of the statistics of all the available data in the image sequence. In our experiments, we tested the proposed approach on several motion types at different noise levels, including translational motion, convergent/divergent motion, and cardiac motion. Our results demonstrate that this approach could lead to more robust estimation of the motion field in the presence of strong imaging noise compared to a frame-by-frame estimation approach.

Using hardware models to quantify sensory data acquisition across the rat vibrissal array

Our laboratory investigates how animals acquire sensory data to understand the neural computations that permit complex sensorimotor behaviors. We use the rat whisker system as a model to study active tactile sensing; our aim is to quantitatively describe the spatiotemporal structure of incoming sensory information to place constraints on subsequent neural encoding and processing. In the first part of this paper we describe the steps in the development of a hardware model (a 'sensobot') of the rat whisker array that can perform object feature extraction. We show how this model provides insights into the neurophysiology and behavior of the real animal. In the second part of this paper, we suggest that sensory data acquisition across the whisker array can be quantified using the complete derivative. We use the example of wall-following behavior to illustrate that computing the appropriate spatial gradients across a sensor array would enable an animal or mobile robot to predict the sensory data that will be acquired at the next time step.

A computational model of the analysis of some first-order and second-order motion patterns by simple and complex cells

Although spatio-temporal gradient schemes are widely used in the computation of image motion, algorithms are ill conditioned for particular classes of input. This paper addresses this problem. Motion is computed as the space-time direction in which the difference in image illuminance from the local mean is conserved. This method can reliably detect motion in first-order and some second-order motion stimuli. Components of the model can be identified with directionally asymmetric and directionally selective simple cells. A stage in which we compute spatial and temporal derivatives of the difference between image illuminance and the local mean illuminance using a truncated Taylor series gives rise to a phase-invariant output reminiscent of the response of complex cells.

(Non-)rigid motion interpretation : a regularized approach

Determining 3D motion from a time-varying 2D image is an ill-posed problem; unless we impose additional constraints, an infinite number of solutions is possible. The usual constraint is rigidity, but many naturally occurring motions are not rigid and not even piecewise rigid. A more general assumption is that the parameters (or some of the parameters) characterizing the motion are approximately (but not exactly) constant in any sufficiently small region of the image. If we know the shape of a surface we can uniquely recover the smoothest motion consistent with image data and the known structure of the object, through regularization. This paper develops a general paradigm for the analysis of nonrigid motion. The variational condition we obtain includes many previously studied constraints as ‘special cases’. Among them are isometry, rigidity and planarity. If the variational condition is applied at multiple scales of resolution, it can be applied to turbulent motion. Finally, it is worth noting that our theory does not require the computation of correspondence (optic flow or discrete displacements), and it is effective in the presence of motion discontinuities.

What happens to binocularity in primate strabismus?

Normal humans for whom the positions and movements of the two eyes are constrained to be yoked together are able to extract rich binocular sensory information from the environment. Humans with strabismus are deficient in extracting some of this information. Studies of strabismus in non-human primates can augment what has been learned from humans about relationships between strabismus and sensory binocular function. For example, speculation about the role of binocular vision in primate evolution can help us understand why it is that the advantages of sensory binocular function outweigh the disadvantages of having the positions of the two eyes yoked together. Physiological optics assessments of fixation patterns and accommodative responses in monkeys provide information about how the brain accomplishes and coordinates motor and sensory binocular functions, and sets the stage for determining underlying neural mechanisms responsible for this coordination. Finally, a developmental perspective, concerned with events that occur during an early sensitive period in the life span of an infant primate, can help us understand how nature and nurture interact to set up this complex neural system in normal individuals, and how this process is disrupted in conditions such as strabismus.

Probabilistic and sequential computation of optical flow using temporal coherence

In the computation of dense optical flow fields, spatial coherence constraints are commonly used to regularize otherwise ill-posed problem formulations, providing spatial integration of data. We present a temporal, multiframe extension of the dense optical flow estimation formulation proposed by Horn and Schunck (1981) in which we use a temporal coherence constraint to yield the optimal fusing of data from multiple frames of measurements. Conceptually, standard Kalman filtering algorithms are applicable to the resulting multiframe optical flow estimation problem, providing a solution that is sequential and recursive in time. Experiments are presented to demonstrate that the resulting multiframe estimates are more robust to noise than those provided by the original, single-frame formulation. In addition, we demonstrate cases where the aperture problem of motion vision cannot be resolved satisfactorily without the temporal integration of data enabled by the proposed formulation. Practically, the large matrix dimensions involved in the problem prohibit exact implementation of the optimal Kalman filter. To overcome this limitation, we present a computationally efficient, yet near-optimal approximation of the exact filtering algorithm. This approximation has a precise interpretation as the sequential estimation of a reduced-order spatial model for the optical flow estimation error process at each time step and arises from an estimation-theoretic treatment of the filtering problem. Experiments also demonstrate the efficacy of this near-optimal filter.

Scene segmentation from visual motion using global optimization

This paper presents results from computer experiments with an algorithm to perform scene disposition and motion segmentation from visual motion or optic flow. The maximum a posteriori (MAP) criterion is used to formulate what the best segmentation or interpretation of the scene should be, where the scene is assumed to be made up of some fixed number of moving planar surface patches. The Bayesian approach requires, first, specification of prior expectations for the optic flow field, which here is modeled as spatial and temporal Markov random fields; and, secondly, a way of measuring how well the segmentation predicts the measured flow field. The Markov random fields incorporate the physical constraints that objects and their images are probably spatially continuous, and that their images are likely to move quite smoothly across the image plane. To compute the flow predicted by the segmentation, a recent method for reconstructing the motion and orientation of planar surface facets is used. The search for the globally optimal segmentation is performed using simulated annealing.

Cable theory in neurons with active, linearized membranes

This investigation aims at exploring some of the functional consequences of single neurons containing active, voltage dependent channels for information processing. Assuming that the voltage change in the dendritic tree of these neurons does not exceed a few millivolts, it is possible to linearize the non-linear channel conductance. The membrane can then be described in terms of resistances, capacitances and inductances, as for instance in the small-signal analysis of the squid giant axon. Depending on the channel kinetics and the associated ionic battery the linearization yields two basic types of membrane: a membrane modeled by a collection of resistances and capacitances and membranes containing in addition to these components inductances. Under certain specified conditions the latter type of membrane gives rise to a membrane impedance that displays a prominent maximum at some nonzero resonant frequency fmax. We call this type of membrane quasi-active, setting it apart from the usual passive membrane. We study the linearized behaviour of active channels giving rise to quasi-active membranes in extended neuronal structures and consider several instances where such membranes may subserve neuronal function: 1. The resonant frequency of a quasi-active membrane increases with increasing density of active channels. This might be one of the biophysical mechanisms generating the large range over which hair cells in the vertebrate cochlea display frequency tuning. 2. The voltage recorded from a cable with a quasi-active membrane can be proportional to the temporal derivative of the injected current. 3. We modeled a highly branched dendritic tree (delta-ganglion cell of the cat retina) using a quasi-active membrane. The voltage attenuation from a given synaptic site to the soma decreases with increasing frequency up to the resonant frequency, in sharp contrast to the behaviour of passive membranes.(ABSTRACT TRUNCATED AT 400 WORDS)