Computing with self-excitatory cliques: A model and an application to hyperacuity-scale computation in visual cortex

We present a model of visual computation based on tightly inter-connected cliques of pyramidal cells. It leads to a formal theory of cell assemblies, a specific relationship between correlated firing patterns and abstract functionality, and a direct calculation relating estimates of cortical cell counts to orientation hyperacuity. Our network architecture is unique in that (1) it supports a mode of computation that is both reliable and efficient; (2) the current-spike relations are modeled as an analog dynamical system in which the requisite computations can take place on the time scale required for an early stage of visual processing; and (3) the dynamics are triggered by the spatiotemporal response of cortical cells. This final point could explain why moving stimuli improve vernier sensitivity.

Evidence for boundary-specific grouping

A prerequisite for higher-level visual tasks such as object recognition is a segmentation of the image into distinct two-dimensional regions. While it has long been assumed that the human visual system jointly exploits region and boundary cues for image segmentation, we report the results of psychophysical experiments which suggest that the visual system relies on geometric properties of bounding contours such as closure and not on the texture of the two-dimensional regions they partition. These findings suggest that the visual system may code and links contours into coherent shapes before surface properties are conjoined.

Area and length minimizing flows for shape segmentation

A number of active contour models have been proposed that unify the curve evolution framework with classical energy minimization techniques for segmentation, such as snakes. The essential idea is to evolve a curve (in two dimensions) or a surface (in three dimensions) under constraints from image forces so that it clings to features of interest in an intensity image. The evolution equation has been derived from first principles as the gradient flow that minimizes a modified length functional, tailored to features such as edges. However, because the flow may be slow to converge in practice, a constant (hyperbolic) term is added to keep the curve/surface moving in the desired direction. We derive a modification of this term based on the gradient flow derived from a weighted area functional, with image dependent weighting factor. When combined with the earlier modified length gradient flow, we obtain a partial differential equation (PDE) that offers a number of advantages, as illustrated by several examples of shape segmentation on medical images. In many cases the weighted area flow may be used on its own, with significant computational savings.

Efficient Simplex-Like Methods for Equilibria of Nonsymmetric Analog Networks

What is the complexity of computing equilibria for physically implementable analog networks (Hopfield 1984; Sejnowski 1981) with arbitrary connectivity? We show that if the amplifiers are piecewise-linear, then such networks are instances of a game-theoretic model known as polymatrix games. In contrast with the usual gradient descent methods for symmetric networks, equilibria for polymatrix games may be computed by vertex pivoting algorithms similar to the simplex method for linear programming. Like the simplex method, these algorithms have characteristic low order polynomial behavior in virtually all practical cases, though not certain theoretical ones. While these algorithms cannot be applied to models requiring evolution from an initial point, they are applicable to “clamping” models whose input is expressed purely as a bias. Thus we have an a priori indication that such models are computationally tractable.

The discriminability of smooth stereoscopic surfaces

In this study the sensitivity of human vision to the smoothness of stereoscopic surface structure was investigated. In experiments 1 and 2 random-dot stereograms were used to evaluate the discrimination of smooth versus 'noisy' sinusoidal surfaces differing in the percentages of points on a single smooth surface. Fully coherent smooth surfaces were found to be much more discriminable than other less smooth randomly perturbed surfaces. In experiment 3 the discrimination between discontinuous triangle-wave surfaces and similarly shaped smoothly curved surfaces obtained from the addition of the fundamental and the third harmonic of the corresponding triangle-wave surface was evaluated. The triangle-wave surfaces were found to be more accurately discriminated from the smoothly curved surfaces than would be predicted from the detectability of the difference in their Fourier power spectra. This superior discriminability was attributed to differences between the curvature and/or discontinuity of the two surfaces. In experiment 3 the effects of incoherent 'noise' points on the discrimination between the two surface types were also evaluated. These randomly positioned noise points had a relatively small effect on the discrimination between the two surfaces. In general, the results of these experiments indicate that smooth surfaces are salient for stereopsis and that isolated local violations of smoothness are highly discriminable.

Coherent Compound Motion: Corners and Nonrigid Configurations

Consider two wire gratings, superimposed and moving across each other. Under certain conditions the two gratings will cohere into a single, compound pattern, which will appear to be moving in another direction. Such coherent motion patterns have been studied for sinusoidal component gratings, and give rise to percepts of rigid, planar motions. In this paper we show how to construct coherent motion displays that give rise to nonuniform, nonrigid, and nonplanar percepts. Most significantly, they also can define percepts with corners. Since these patterns are more consistent with the structure of natural scenes than rigid sinusoidal gratings, they stand as interesting stimuli for both computational and physiological studies. To illustrate, our display with sharp corners (tangent discontinuities or singularities) separating regions of coherent motion suggests that smoothing does not cross tangent discontinuities, a point that argues against existing (regularization) algorithms for computing motion. This leads us to consider how singularities can be confronted directly within optical flow computations, and we conclude with two hypotheses: (1) that singularities are represented within the motion system as multiple directions at the same retinotopic location; and (2) for component gratings to cohere, they must be at the same depth from the viewer. Both hypotheses have implications for the neural computation of coherent motion.

Two Stages of Curve Detection Suggest Two Styles of Visual Computation

The problem of detecting curves in visual images arises in both computer vision and biological visual systems. Our approach integrates constraints from these two sources and suggests that there are two different stages to curve detection, the first resulting in a local description, and the second in a global one. Each stage involves a different style of computation: in the first stage, hypotheses are represented explicitly and coarsely in a fixed, preconfigured architecture; in the second stage, hypotheses are represented implicitly and more finely in a dynamically constructed architecture. We also show how these stages could be related to physiology, specifying the earlier parts in a relatively fine-grained fashion and the later ones more coarsely.

Endstopping and curvature

Hypercomplex or endstopped visual cortical neurons are usually supposed to be concerned with length or end point analysis. However, recent evidence demonstrates that endstopped neurons are curvature-selective, a connection that we explore here in some detail. A model of endstopped simple cells is developed and a variety of computational simulations examine the connection of the model to the reported length and orientation responses of endstopped neurons. Even and odd versions of the model are described, both of which are shown to be curvature-selective. Even-symmetric instances of the model respond well to thin curves over a range of curve orientation and curvature, independent of sign of curvature. In contrast, odd-symmetric instances respond to both thin and thick curves while exhibiting a more complex curvature-sign dependence--responding in a sign-selective fashion to curved lines but not to curved edges. Finally, the response of the endstopped model to curve singularities is explored, and the possible role of nonendstopped and endstopped cells in building curve descriptions is discussed.

Texture fields and texture flows: sensitivity to differences

There are two large classes of textures, those with an overall orientation structure (texture flows) and those without (texture fields). We investigate human sensitivity to detecting a patch of texture field within a texture flow psychophysically by using random not Moiré patterns. The resultant sensitivity, as a function of patch-size and path-length, is then related to a computational model of orientation selection, which reveals a connection between texture structure and the estimation of curvature. Finally, the connection back to curvature is confirmed by demonstrating a similarity between the patch sensitivity data and previous data on sensitivity to corners in flow patterns.

Points and endpoints: a size/spacing constraint for dot grouping

One-dimensional arrangements of dots immediately group into contours. It is reported that, when these contours participate in certain larger arrangements, there is an abrupt point at which the percept changes as a function of dot spacing (or density along the contour). Closely spaced arrangements give rise to subjective effects involving apparent brightness and depth, whereas sparsely spaced ones do not. The effects are most clear in configurations that involve endpoints and possible occlusions. For these configurations, densely dotted contours are perceptually equivalent to solid ones, but sparse ones are not. This change in percept occurs abruptly and consistently at a dot to space ratio of 1:5, when the dot density is normalized by dot size, and this point is called the size/spacing constraint. It holds only for dots of the order of 1 min visual angle in diameter when small to modest contrast values are used. The subjective effects are not present for dotted contours (or even for solid ones) that are smaller (less than 0.5 min), and differ for contours that are larger (greater than 10 min). To demonstrate the significance of size/spacing constraints for early vision, a framework for grouping consisting of processes at many different levels is outlined, and the requirements for the earliest one (orientation selection) are sketched in greater detail. The size/spacing constraint follows directly from one of these requirements--receptive field structure--and seems to indicate a switch from early orientation-selection processes to later ones.

Endstopped neurons in the visual cortex as a substrate for calculating curvature

Neurons in the visual cortex typically respond selectively to the orientation, and velocity and direction of movement, of moving-bar stimuli. These responses are generally thought to provide information about the orientation and position of lines and edges in the visual field. Some cells are also endstopped, that is selective for bars of specific lengths. Hubel and Wiesel first observed that endstopped hypercomplex cells could respond to curved stimuli and suggested they might be involved in detection of curvature, but the exact relationship between endstopping and curvature has never been determined. We present here a mathematical model relating endstopping to curvature in which the difference in response of two simple cells gives rise to endstopping and varies in proportion to curvature. We also provide physiological evidence that endstopped cells in area 17 of the cat visual cortex are selective for curvature, whereas non-endstopped cells are not, and that some are selective for the sign of curvature. The prevailing view of edge and curve determination is that orientations are selected locally by the class of simple cortical cells and then integrated to form global curves. We have developed a computational theory of orientation selection which shows that measurements of orientation obtained by simple cells are not sufficient because there will be strong, incorrect responses from cells whose receptive fields (RFs) span distinct curves (Fig. 1). If estimates of curvature are available, however, these inappropriate responses can be eliminated. Curvature provides the key to structuring the network that underlies our theory and distinguishes it from previous lateral inhibition schemes.

The local structure of image discontinuities in one dimension

The detailed structure of intensities in the local neighborhood of an edge can often indicate the nature of the physical event givinig rise to that edge. We argue that the limit, as we approach arbitrarily close to either side of an edge, of such image parameters as type of texture, texture gradient, color, appropriate directional derivatives of intensity, etc., is a key aspect of this structure. However, the general problem of capturing this local structure is surprisingly complex. Thus, we restrict ourselves in this paper to a relatively simple domain¿one-dimensional cuts through idealized images modeled by piecewise smooth (C1) functions corrupted by Gaussian noise. Within this domain, we define local structure to be the limit of the uncorrupted intensity and of its derivatives as we approach arbitrarily close to either side of a discontinuity. We develop a technique that captures this local structure while simultaneously locating the discontinuities, and demonstrate that these tasks are in fact inseparable. The technique is an extension, using estimation theory, of the classical definition of discontinuity. It handles, in a consistent fashion, both jump discontinuities in the function and jump discontinuities in its first derivative (so-called step-edges are a special case of the former and roof-edges of the latter). It also integrates, again in a consistent fashion, information derived from a number of different neighborhood sizes.