Improvement of the Prediction Quality for Visual Servoing with a Switching Kalman Filter

The main control problem of visual servoing is to cope with the delay introduced by image acquisition and image processing. This delay is the main reason for limited tracking velocity and acceleration. Predictive algorithms are one solution to handle the delay. The drawback of prediction algorithms is the bad prediction behavior for the discontinuity in the target motion, for example, a velocity step. In this paper, a switching Kalman filter (SKF) is proposed to overcome this problem. The SKF introduces three cooperating components. A prediction monitor supervises the prediction quality of an adaptive Kalman filter (AKF). If a discontinuity is detected, a transition filter switches to an appropriate steady-state Kalman filter (αβ or αβγ), which handles a discontinuity better than the AKF. During this transition, an auxiliary controller ensures that overall control is continuous. This new prediction algorithm is able to achieve a good prediction quality for smooth and for discontinuous motions. It is evaluated using a pan/tilt unit to track a colored object. The SKF and its components are compared to the classical AKF with four different target motions.

Robust and efficient feature tracking for indoor navigation

Robust feature tracking is a requirement for many computer vision tasks such as indoor robot navigation. However, indoor scenes are characterized by poorly localizable features. As a result, indoor feature tracking without artificial markers is challenging and remains an attractive problem. We propose to solve this problem by constraining the locations of a large number of nondistinctive features by several planar homographies which are strategically computed using distinctive features. We experimentally show the need for multiple homographies and propose an illumination-invariant local-optimization scheme for motion refinement. The use of a large number of nondistinctive features within the constraints imposed by planar homographies allows us to gain robustness. Also, the lesser computation cost in estimating these nondistinctive features helps to maintain the efficiency of the proposed method. Our local-optimization scheme produces subpixel accurate feature motion. As a result, we are able to achieve robust and accurate feature tracking.

Survey of visual and force/tactile control of robots for physical interaction in Spain

Sensors provide robotic systems with the information required to perceive the changes that happen in unstructured environments and modify their actions accordingly. The robotic controllers which process and analyze this sensory information are usually based on three types of sensors (visual, force/torque and tactile) which identify the most widespread robotic control strategies: visual servoing control, force control and tactile control. This paper presents a detailed review on the sensor architectures, algorithmic techniques and applications which have been developed by Spanish researchers in order to implement these mono-sensor and multi-sensor controllers which combine several sensors.

Real-time decentralized articulated motion analysis and object tracking from videos

In this paper, we present two new articulated motion analysis and object tracking approaches: the decentralized articulated object tracking method and the hierarchical articulated object tracking method. The first approach avoids the common practice of using a high-dimensional joint state representation for articulated object tracking. Instead, we introduce a decentralized scheme and model the interpart interaction within an innovative Bayesian framework. Specifically, we estimate the interaction density by an efficient decomposed interpart interaction model. To handle severe self-occlusions, we further extend the first approach by modeling high-level interunit interaction and develop the second algorithm within a consistent hierarchical framework. Preliminary experimental results have demonstrated the superior performance of the proposed approaches on real-world videos in both robustness and speed compared with other articulated object tracking methods.

Recovering articulated pose: a comparison of two pre and postimposed constraint methods

We contrast the performance of two methods of imposing constraints during the tracking of articulated objects, the first method preimposing the kinematic constraints during tracking and, thus, using the minimum degrees of freedom, and the second imposing constraints after tracking and, hence, using the maximum. Despite their very different formulations, the methods recover the same pose change. Further comparisons are drawn in terms of computational speed and algorithmic simplicity and robustness, and it is the last area which is the most telling. The results suggest that using built-in constraints is well-suited to tracking individual articulated objects, whereas applying constraints afterward is most suited to problems involving contact and breakage between articulated (or rigid) objects, where the ability to test tracking performance quickly with constraints turned on or off is desirable.

Using an Extended Kalman Filter for Rigid Body Pose Estimation

Rigid body pose is commonly represented as the rigid body transformation from one (often reference) pose to another. This is usually computed for each frame of data without any assumptions or restrictions on the temporal change of the pose. The most common algorithm was proposed by Söderkvist and Wedin (1993, “Determining the Movements of the Skeleton Using Well-configured Markers,” J. Biomech., 26, pp. 1473–1477), and implies the assumption that measurement errors are isotropic and homogenous. This paper describes an alternative method based on a state space formulation and the application of an extended Kalman filter (EKF). State space models are formulated, which describe the kinematics of the rigid body. The state vector consists of six generalized coordinates (corresponding to the 6 degrees of freedom), and their first time derivatives. The state space models have linear dynamics, while the measurement function is a nonlinear relation between the state vector and the observations (marker positions). An analytical expression for the linearized measurement function is derived. Tracking the rigid body motion using an EKF enables the use of a priori information on the measurement noise and type of motion to tune the filter. The EKF is time variant, which allows for a natural way of handling temporarily missing marker data. State updates are based on all the information available at each time step, even when data from fewer than three markers are available. Comparison with the method of Söderkvist and Wedin on simulated data showed a considerable improvement in accuracy with the proposed EKF method when marker data was temporarily missing. The proposed method offers an improvement in accuracy of rigid body pose estimation by incorporating knowledge of the characteristics of the movement and the measurement errors. Analytical expressions for the linearized system equations are provided, which eliminate the need for approximate discrete differentiation and which facilitate a fast implementation.

A posture optimization algorithm for model-based motion capture of movement sequences

We have developed and evaluated a new optical motion capture approach that is suitable for a wide range of studies in neuroethology and motor control. Based on the stochastic search algorithm of Simulated Annealing (SA), it utilizes a kinematic body model that includes joint angle constraints to reconstruct posture from an arbitrary number of views. Rather than tracking marker trajectories in time, the algorithm minimizes an error function that compares predicted model projections to the recorded views. Thus, each video-frame is analyzed independently from other frames, enabling the system to recover from incorrectly analyzed postures. The system works with standard computer and video equipment. Its accuracy is evaluated using videos of animated locust leg movements, recorded by two orthogonal views. The resulting joint angle RMS errors range between 0.7 degrees and 4.9 degrees, limited by the pixel resolution of the digital video. 3D-movement reconstruction is possible even from a single view. In a real experimental application, stick insect walking sequences are analyzed with leg joint angle deviations between 0.5 degrees and 3.0 degrees. This robust and accurate performance is reached in spite of marker fusions and occlusions, simply by exploiting the natural constraints imposed by a kinematic chain and a known experimental setup.

Robust recovery of human motion from video using Kalman filters and virtual humans

In sport science, as in clinical gait analysis, optoelectronic motion capture systems based on passive markers are widely used to recover human movement. By processing the corresponding image points, as recorded by multiple cameras, the human kinematics is resolved through multistage processing involving spatial reconstruction, trajectory tracking, joint angle determination, and derivative computation. Key problems with this approach are that marker data can be indistinct, occluded or missing from certain cameras, that phantom markers may be present, and that both 3D reconstruction and tracking may fail. In this paper, we present a novel technique, based on state space filters, that directly estimates the kinematical variables of a virtual mannequin (biomechanical model) from 2D measurements, that is, without requiring 3D reconstruction and tracking. Using Kalman filters, the configuration of the model in terms of joint angles, first and second order derivatives is automatically updated in order to minimize the distances, as measured on TV-cameras, between the 2D measured markers placed on the subject and the corresponding back-projected virtual markers located on the model. The Jacobian and Hessian matrices of the nonlinear observation function are computed through a multidimensional extension of Stirling's interpolation formula. Extensive experiments on simulated and real data confirmed the reliability of the developed system that is robust against false matching and severe marker occlusions. In addition, we show how the proposed technique can be extended to account for skin artifacts and model inaccuracy.

Real-time human motion estimation using biomechanical models and non-linear state-space filters

In the field of sports biomechanics and rehabilitation engineering, the possibility of computing, in real time, the angular displacements and derivatives of human joints, from a video of motion sequences, represents an appealing goal. In particular, applications of biofeedback protocols in rehabilitation can benefit from this capability. The focus of the investigation was concerned with the application of biomechanical models, comprising of a kinematic chain and surface envelopes, and state-space filters, to the computation, in real time and with high accuracy, of the angular data and derivatives. By minimising the distances, measured with TV cameras, between the 2D marker projections and the corresponding back-projected markers located on the mannequin, the configuration of the biomechanical model was automatically updated. The use of state-space estimation allowed the computation of smooth derivatives of the orientation data. Owing to the non-linearity of the functions involved, the derivatives of the observation model were obtained through a multidimensional extension of Stirling's interpolation formula. Proper algorithms were developed to cope with the model calibration, initialisation and data labelling. Extensive experiments on real and simulated motions proved the reliability (maximum angular error less than 1 degree, maximum point reconstruction less than 1 mm) of the developed system, which is robust to false matching caused by marker occlusions. Moreover, orientation artifacts due to skin motion can be reduced by a factor of 50%.