Low bit rate video coding using robust motion vector regeneration in the decoder

Low-bit-rate video coding using dense motion field and uncovered background prediction

It is well known that accurate dense motion field can improve the video coding efficiency. This paper presents a novel Markov random field (MRF) model that estimates both the dense motion and uncovered background fields in image sequences, and the application of these estimates in H.263-based video coding framework.

An optimal quadtree-based motion estimation and motion-compensated interpolation scheme for video compression

We propose an optimal quadtree (QT)-based motion estimator for video compression. It is optimal in the sense that for a given bit budget for encoding the displacement vector field (DVF) and the QT segmentation, the scheme finds a DVF and a QT segmentation which minimizes the energy of the resulting displaced frame difference (DFD). We find the optimal QT decomposition and the optimal DVF jointly using the Lagrangian multiplier method and a multilevel dynamic program. We introduce a new, very fast convex search for the optimal Lagrangian multiplier lambda(*), which results in a very fast convergence of the Lagrangian multiplier method. The resulting DVF is spatially inhomogeneous, since large blocks are used in areas with simple motion and small blocks in areas with complex motion. We also propose a novel motion-compensated interpolation scheme which uses the same mathematical tools developed for the QT-based motion estimator. One of the advantages of this scheme is the globally optimal control of the tradeoff between the interpolation error energy and the DVF smoothness. Another advantage is that no interpolation of the DVF is required since we directly estimate the DVF and the QT-segmentation for the frame which needs to be interpolated. We present results with the proposed QT-based motion estimator which show that for the same DFD energy the proposed estimator uses about 25% fewer bits than the commonly used block matching algorithm. We also experimentally compare the interpolated frames using the proposed motion compensated interpolation scheme with the reconstructed original frames.

Simultaneous recursive displacement estimation and restoration of noisy-blurred image sequences

We develop a recursive model-based maximum a posteriori (MAP) estimator that simultaneously estimates the displacement vector field (DVF) and the intensity field from a noisy-blurred image sequence. Current motion-compensated spatio-temporal noise filters treat the estimation of the DVF as a preprocessing step. Generally, no attempt is made to verify the accuracy of these estimates prior to their use in the filter. By simultaneously estimating these two fields, we establish a link between the two estimators. It is through this link that the DVF estimate and its corresponding accuracy information are shared with the other intensity estimator, and vice versa. To model the DVF and the intensity field, we use coupled Gauss-Markov (CGM) models. A CGM model consists of two levels: an upper level, which is made up of several submodels with various characteristics, and a lower level or line field, which governs the transitions between the submodels. The CGM models are well suited for estimating the displacement and intensity fields since the resulting estimates preserve the boundaries between the stationary areas present in both fields. Detailed line fields are proposed for the modeling of these boundaries, which also take into account the correlations that exist between these two fields. A Kalman-type estimator results, followed by a decision criterion for choosing the appropriate set of line fields. Several experiments using noisy and noisy-blurred image sequences demonstrate the superior performance of the proposed algorithm with respect to prediction error and mean-square error.