Mach edges: Local features predicted by 3rd derivative spatial filtering

Covariance properties under natural image transformations for the generalised Gaussian derivative model for visual receptive fields

The property of covariance, also referred to as equivariance, means that an image operator is well-behaved under image transformations, in the sense that the result of applying the image operator to a transformed input image gives essentially a similar result as applying the same image transformation to the output of applying the image operator to the original image. This paper presents a theory of geometric covariance properties in vision, developed for a generalised Gaussian derivative model of receptive fields in the primary visual cortex and the lateral geniculate nucleus, which, in turn, enable geometric invariance properties at higher levels in the visual hierarchy. It is shown how the studied generalised Gaussian derivative model for visual receptive fields obeys true covariance properties under spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations. These covariance properties imply that a vision system, based on image and video measurements in terms of the receptive fields according to the generalised Gaussian derivative model, can, to first order of approximation, handle the image and video deformations between multiple views of objects delimited by smooth surfaces, as well as between multiple views of spatio-temporal events, under varying relative motions between the objects and events in the world and the observer. We conclude by describing implications of the presented theory for biological vision, regarding connections between the variabilities of the shapes of biological visual receptive fields and the variabilities of spatial and spatio-temporal image structures under natural image transformations. Specifically, we formulate experimentally testable biological hypotheses as well as needs for measuring population statistics of receptive field characteristics, originating from predictions from the presented theory, concerning the extent to which the shapes of the biological receptive fields in the primary visual cortex span the variabilities of spatial and spatio-temporal image structures induced by natural image transformations, based on geometric covariance properties.

Normative theory of visual receptive fields

This article gives an overview of a normative theory of visual receptive fields. We describe how idealized functional models of early spatial, spatio-chromatic and spatio-temporal receptive fields can be derived in a principled way, based on a set of axioms that reflect structural properties of the environment in combination with assumptions about the internal structure of a vision system to guarantee consistent handling of image representations over multiple spatial and temporal scales. Interestingly, this theory leads to predictions about visual receptive field shapes with qualitatively very good similarities to biological receptive fields measured in the retina, the LGN and the primary visual cortex (V1) of mammals.

Dynamic Cancellation of Perceived Rotation from the Venetian Blind Effect

Geometric differences between the images seen by each eye enable the perception of depth. Additionally, depth is produced in the absence of geometric disparities with binocular disparities in either the average luminance or contrast, which is known as the Venetian blind effect. The temporal dynamics of the Venetian blind effect are much slower (1.3 Hz) than those for geometric binocular disparities (4-5 Hz). Sine-wave modulations of luminance and contrast disparity, however, can be discriminated from square-wave modulations at 1 Hz, which suggests a non-linearity. To measure this non-linearity, a luminance or contrast disparity modulation was presented at a particular frequency and paired with a geometric disparity modulation that cancelled the perceived rotation induced by the luminance or contrast modulation. Phases between the luminance or contrast and the geometric modulation varied in 50 ms increments from -200 and 200 ms. When phases were aligned, observers perceived little or no rotation. When not aligned, a perceived rotation was induced by a contrast or luminance disparity that was then cancelled by the geometric disparity. This causes the perception of a slight jump. The Generalized Difference Model, which is linear in time, predicted a minimal probability in cases when luminance or contrast disparities occurred before the geometric disparities due to the slower dynamics of the Venetian blind effect. The Gated Generalized Difference Model, which is non-linear in time, predicted a minimal probability for offsets of 0 ms. Results followed the Gated model, which further suggests a non-linearity in time for the Venetian blind effect.

Features and the 'primal sketch'

This review is concerned primarily with psychophysical and physiological evidence relevant to the question of the existence of spatial features or spatial primitives in human vision. The review will be almost exclusively confined to features defined in the luminance domain. The emphasis will be on the experimental and computational methods that have been used for revealing features, rather than on a detailed comparison between different models of feature extraction. Color and texture fall largely outside the scope of the review, though the principles may be similar. Stereo matching and motion matching are also largely excluded because they are covered in other contributions to this volume, although both have addressed the question of the spatial primitives involved in matching. Similarities between different psychophysically-based model will be emphasized rather than minor differences. All the models considered in the review are based on the extraction of directional spatial derivatives of the luminance profile, typically the first and second, but in one case the third order, and all have some form of non-linearity, be it rectification or thresholding.