3-d motion estimation, understanding, and prediction from noisy image sequences

Mobile Robot Localization from Large-Scale Appearance Mosaics

A new practical, high-performance mobile robot localization technique is described that is motivated by the fact that many man-made environments contain substantially flat, visually textured surfaces of persistent appearance. While the tracking of image regions is much studied in computer vision, appearance is still a largely unexploited localization resource in commercially relevant applications. We show how prior appearance models can be used to enable highly repeatable mobile robot guidance that, unlike commercial alternatives, is both infrastructure-free and free-ranging. Very large-scale mosaics are constructed and used to localize a mobile robot operating in the modeled environment. Straightforward techniques from vision-based localization and mosaicking are used to produce a field-relevant AGV guidance system based only on vision and odometry. The feasibility, design, implementation, and precommercial field qualification of such a guidance system are described.

Robust ego-motion estimation and 3-D model refinement using surface parallax

We present an iterative algorithm for robustly estimating the ego-motion and refining and updating a coarse depth map using parametric surface parallax models and brightness derivatives extracted from an image pair. Given a coarse depth map acquired by a range-finder or extracted from a digital elevation map (DEM), ego-motion is estimated by combining a global ego-motion constraint and a local brightness constancy constraint. Using the estimated camera motion and the available depth estimate, motion of the three-dimensional (3-D) points is compensated. We utilize the fact that the resulting surface parallax field is an epipolar field, and knowing its direction from the previous motion estimates, estimate its magnitude and use it to refine the depth map estimate. The parallax magnitude is estimated using a constant parallax model (CPM) which assumes a smooth parallax field and a depth based parallax model (DBPM), which models the parallax magnitude using the given depth map. We obtain confidence measures for determining the accuracy of the estimated depth values which are used to remove regions with potentially incorrect depth estimates for robustly estimating ego-motion in subsequent iterations. Experimental results using both synthetic and real data (both indoor and outdoor sequences) illustrate the effectiveness of the proposed algorithm.

An information theoretic criterion for evaluating the quality of 3-D reconstructions from video

Even though numerous algorithms exist for estimating the three-dimensional (3-D) structure of a scene from its video, the solutions obtained are often of unacceptable quality. To overcome some of the deficiencies, many application systems rely on processing more data than necessary, thus raising the question: how is the accuracy of the solution related to the amount of data processed by the algorithm? Can we automatically recognize situations where the quality of the data is so bad that even a large number of additional observations will not yield the desired solution? Previous efforts to answer this question have used statistical measures like second order moments. They are useful if the estimate of the structure is unbiased and the higher order statistical effects are negligible, which is often not the case. This paper introduces an alternative information-theoretic criterion for evaluating the quality of a 3-D reconstruction. The accuracy of the reconstruction is judged by considering the change in mutual information (MI) (termed as the incremental MI) between a scene and its reconstructions. An example of 3-D reconstruction from a video sequence using optical flow equations and known noise distribution is considered and it is shown how the MI can be computed from first principles. We present simulations on both synthetic and real data to demonstrate the effectiveness of the proposed criterion.

Parallel distributed detection of feature trajectories in multiple discontinuous motion image sequences

Concerns the 3D interpretation of image sequences showing multiple objects in motion. Each object exhibits smooth motion except at certain time instants when a motion discontinuity may occur. The objects are assumed to contain point features which are detected as the images are acquired. Estimating feature trajectories in the first two frames amounts to feature matching. As more images are acquired, existing trajectories are extended. Both initial detection and extension of trajectories are done by enforcing pertinent constraints from among the following: similarity of the image plane arrangement of neighboring features, smoothness of the 3D motion and smoothness of the image plane motion. The constraints are incorporated into energy functions which are minimized using 2D Hopfield networks. Wrong matches that result from convergence to local minima are eliminated using a 1D Hopfield-like network. Experimental results on several image sequences are shown.

On the problem of correspondence in range data and some inelastic uses for elastic nets

In this work, the authors propose a novel method to obtain correspondence between range data across image frames using neural like mechanisms. The method is computationally efficient and tolerant of noise and missing points. Elastic nets, which evolved out of research into mechanisms to establish ordered neural projections between structures of similar geometry, are used to cast correspondence as an optimization problem. This formulation is then used to obtain approximations to the motion parameters under the assumption of rigidity (inelasticity). These parameter scan be used to recover correspondence. Experimental results are presented to establish the veracity of the scheme and the method is compared to earlier attempts in this direction.

Tracking a dynamic set of feature points

We address the problems of tracking a set of feature points over a long sequence of monocular images as well as how to include and track new feature points detected in successive frames. Due to the 3-D movement of the camera, different parts of the images exhibit different image motion. Tracking discrete features can therefore be decomposed into several independent and local problems. Accordingly, we propose a localized feature tracking algorithm. The trajectory of each feature point is described by a 2-D kinematic model. Then to track a feature point, an interframe motion estimation scheme is designed to obtain the estimates of interframe motion parameters. Subsequently, using the estimates of motion parameters, corresponding points are identified to subpixel accuracy. Afterwards, the temporal information is processed to facilitate the tracking scheme. Since different feature points are tracked independently, the algorithm is able to handle the image motion arising from general 3-D camera movements. On the other hand, in addition to tracking feature points detected at the beginning, an efficient way to dynamically include new points extracted in subsequent frames is devised so that the information in a sequence is preserved. Experimental results for several image sequences are also reported.

Artificial neural networks for 3-D motion analysis. I. Rigid motion

Proposes an approach applying artificial neural net techniques to 3D rigid motion analysis based on sequential multiple time frames. The approach consists of two phases: (1) matching between every two consecutive frames and (2) estimating motion parameters based on the correspondences established. Phase 1 specifies the matching constraints to ensure a stable and coherent feature correspondence establishment between two sequential time frames and configures a 2D Hopfield neural net to enforce these constraints. Phase 2 constructs a 3-layer net to estimate parameters through supervised learning. The method performs motion analysis based on sequential multiple time frames. It represents an effective way to achieve optimal matching between two frames using neural net techniques. The energy function of the Hopfield net is designed to reflect the matching constraints and the minimization of this function leads to the optimal feature correspondence establishment. The approach introduces the learning concept to motion estimation. The structure of the net provides the flexibility in estimating motion parameters based on information from multiple frames.

Adaptive model-based motion estimation

A general discrete-time, adaptive, multidimensional framework is introduced for estimating the motion of one or several object features from their successive nonlinear projections on an image plane. The motion model consists of a set of linear difference equations with parameters estimated recursively from a nonlinear observation equation. The model dimensionality corresponds to that of the original, nonprojected motion space, thus allowing to compensate for variable projection characteristics such as paning and zooming of the camera. Extended recursive least-squares and linear-quadratic tracking algorithms are used to adaptively adjust the model parameters and minimize the errors of either smoothing, filtering or prediction of the object trajectories in the projection plane. Both algorithms are derived using a second order approximation of the projection nonlinearities. All the results presented here use a generalized vectorial notation suitable for motion estimation of any finite number of object features and various approximations of the nonlinear projection. The application of the model-based motion estimator for temporal decimation/interpolation in digital video sequence compression systems is presented.