The role of perspective information in the recovery of 3D structure-from-motion

Recovery of surface pose from texture orientation statistics under perspective projection

In a seminal paper, Witkin (1981) derived a model of surface slant and tilt recovery based on the statistics of the orientations of texture elements (texels) on a planar surface. This model made use of basic mathematical properties of probability distributions to formulate a posterior distribution on slant and tilt given a set of image orientations under orthographic projection. One problem with the Witkin model was that it produced a posterior distribution with multiple maxima, reflecting the inherent ambiguity in scene reconstruction under orthographic projection. In the present article, we extend Witkin's method to incorporate the effects of perspective projection. An identical approach is used to that of Witkin; however, the model now reflects the effects of perspective projection on texel orientation. Performance of the new model is compared against that of Witkin's model in a basic surface pose recovery task using both a maximum a posteriori (MAP) decision rule and a rule based on the expected value of the posterior distribution. The resultant posterior of the new model is shown to have only one maximum and thereby the ambiguity in scene interpretation is resolved. Furthermore, the model performs better than Witkin's model using both MAP and expected value decision rules. The results are discussed in the context of human slant estimation.

A new theory of structure-from-motion perception

Humans can recover 3-D structure from the projected 2D motion field of a rotating object, a phenomenon called structure from motion (SFM). Current models of SFM perception are limited to the case in which objects rotate about a frontoparallel axis. However, as our recent psychophysical studies showed, frontoparallel axes of rotation are not representative of the general case. Here we present the first model to address the problem of SFM perception for the general case of rotations around an arbitrary axis. The SFM computation is cast as a two-stage process. The first stage computes the structure perpendicular to the axis of rotation. The second stage corrects for the slant of the axis of rotation. For cylinders, the computed object shape is invariant with respect to the observer's viewpoint (that is, perceived shape doesn't change with a change in the direction of the axis of rotation). The model uses template matching to estimate global parameters such as the angular speed of rotation, which are then used to compute the local depth structure. The model provides quantitative predictions that agree well with current psychophysical data for both frontoparallel and non-frontoparallel rotations.

Combining cues while avoiding perceptual conflicts

A common assumption in cue combination models is that small discrepancies between cues are due to the limited resolution of the individual cues. Whenever this assumption holds, information from the separate cues can best be combined to give a single, more accurate estimate of the property of interest. We examined whether information about the discrepancy itself is lost when this is done. In our experiments, subjects were required to combine cues to match certain properties while avoiding perceptual conflicts. In part 1, they combined expansion and change in disparity to estimate motion in depth; and in part 2, they combined perspective and binocular disparities to estimate slant. We compared the pattern in the way that subjects set the two cues with the patterns predicted by models of cue combination with and without a loss of information about the discrepancy. From this comparison we conclude that little information about the discrepancies between cues is lost when the cues are combined.